OSCAR: A Collaborative Bandwidth Aggregation System

0.66 MB·

by  Karim Habak

OSCAR: A Collaborative Bandwidth Aggregation
System
Karim Habak Khaled A. Harras Moustafa Youssef
School of Computer Science Computer Science Department Department of CS. and Eng.
College of Computing School of Computer Science Egypt-Japan University of
Georgia Institute of Technology Carnegie Mellon University Science and Technology (E-JUST)
Email: [e-mail] Email: [e-mail] Email: [e-mail]
Abstract—The exponential increase in mobile data demand, and railway stations. Fortunately, the widespread deployment
coupled with growing user expectation to be connected in of various wireless technologiescoupled with the exponential
all places at all times, have introduced novel challenges for
increaseofmulti-interfaceenableddevicesareprovidingusers
researcherstoaddress.Fortunately, thewidespread deployment
with many alternatives for sending and receiving data.
of various network technologies and the increased adoption
of multi-interface enabled devices have enabled researchers to A potential approach for solving the data tsunami problem
develop solutions for those challenges. Such solutions aim to as well as expensive data roaming charges for mobile users
exploit available interfaces on such devices in both solitary and is exploiting all available communication interfaces available
collaborative forms.Thesesolutions,however, havefacedasteep
on modern mobile devices in both solitary and collaborative
deployment barrier.
forms. In the solitary form, the goal is to exploit any direct
Inthispaper,wepresentOSCAR,amulti-objective,incentive-
based, collaborative, and deployable bandwidth aggregation Internet connectivity on any of the available interfaces by
system. We present the OSCAR architecture that does not distributing applications data across them in order to achieve
introduce any intermediate hardware nor require changes to higherthroughput,minimizeenergyconsumption,and/ormin-
current applications or legacy servers. The OSCAR architec-
imizecost.Inthecollaborativeform,thegoalistoenableand
ture is designed to automatically estimate the system’s context,
incentivize mobile devices to utilize their neighbors’ under-
dynamically schedule various connections and/or packets to
different interfaces, be backwards compatible with the current utilized bandwidth in addition to their own direct Internet
Internet architecture, and provide the user with incentives for connections.
collaboration. We also formulate the OSCAR scheduler as a Utilizing the mobile devices interfaces in these two forms
multi-objective, multi-modal scheduler that maximizes system
have been investigated over the past few years [4]. The focus
throughput while minimizing energy consumption or financial
has largely been on the solitary form developing a variety
cost. We evaluate OSCAR via implementation on Linux, as well
as via simulation, and compare our results to the current opti- ofmulti-interfacebandwidthaggregationsolutionsatdifferent
mal achievable throughput, cost, and energy consumption. Our layersof the protocolstack [5]–[17].Other attempts to utilize
evaluation shows that, in the throughput maximization mode, theseinterfacesinthecollaborativeformwerealsointroduced
we provide up to 150% enhancement in throughput compared
[18]–[22]. These approaches either deal with a small scale
to current operating systems, without any changes to legacy
collaborative community managed by a single authority [19],
servers. Moreover, this performance gain further increases with
the availability of connection resume-supporting, or OSCAR- [20], or utilize proxy servers to handle and guarantee such
enabledservers, reachingthemaximumachievableupper-bound collaboration [18], [21]. Overall, despite the fact that current
throughput. smart phones, tablets, and other mobile devices are equipped
with multiple network interfaces, current operating systems
I. INTRODUCTION
mainly allow users to utilize only one interface at a time, or
The Federal Communications Commission (FCC) has in- enable limited bandwidth sharing options through tethering.
dicated an expected data tsunami problem predicting a 25- In other words, there has been a high deploymentbarrier and
50× increase in mobile data traffic by the year 2015 [1], solutions have focused on bandwidth maximization while not
[2]. This expected explosive demand for mobile data, along paying sufficient attention to energy and/or cost efficiency.
with expensive data roaming charges and users’ expectations In this paper, we present OSCAR, a multi-objective,
toremainconnectedinallplacesatalltime,arecreatingnovel incentive-based, collaborative, and deployable bandwidth ag-
challengesfor service providersand researchers to solve. The gregationsystem. OSCAR fulfills the followingrequirements:
recentdeploymentandadoptionofFON[3],asystemthaten- (1)Itis easily deployablewithoutrequiringchangesto legacy
ables users to share their home bandwidth for obtaining WiFi servers, applications, or network infrastructure (i.e. adding
coverageviaotherworldwideFONusers,reflectsarecentpush new hardware like proxies and routers); (2) It is able to
for novel solutions to help solve a subset of these challenges. seamlessly exploit available network interfaces in solitary
Other novel solutions are needed, however, when users are and collaborative forms; (3) It adapts to real-time Internet
on the run or in public places like malls, airports, or bus characteristics and the system parameters to achieve efficient
utilization of these interfaces; (4) It is equipped with an Suchtechniquesrequirechangestotheclientoperatingsystem,
incentive system that encourages users to share their band- legacy servers, and/or applications; and hence have a high
width with others; (5) It adopts an optimal multi-objective deploymentbarrier. Finally, the majority of the network layer
and multi-modal scheduler that maximizes the overall system solutionshidethevariationininterfacesfromtherunningTCP
throughput, while minimizing cost and energy consumption protocol [5]–[7]. Chebrolu et al. [5] require having a proxy
based on the user’s requirements and system’s status; (6) server that communicates with the client and is aware of the
It leverages incremental system adoption and deployment to client’s multiple interfaces. Others implement their system at
further enhance performance gains. both connection-ends, which makes their deployment rely on
Our contributions in this paper are: (1) Designing the updating the legacy servers [6]. On the other hand, MAR [7]
OSCAR system architecture fulfilling the stated requirements requires having a special router as well as an optional proxy
above.(2)FormulatingOSCAR’sdataschedulerasanoptimal server.
multi-objective,multi-modalschedulerthattakesuserrequire- Despite allthiswork,modernoperatingsystemsonlyallow
ments, device context information, as well as application users to use one of the available interfaces at a time, even if
requirements into consideration while distributing application multiple of them are connected to the Internet. This attests to
data across multiple interfaces. (3) Developing OSCAR com- the fact that all current proposals for bandwidth aggregation
munication protocols in order to enable efficient and secured face a steep deployment barrier. In addition, these solutions
communication between the collaborating nodes as well as have largely focused only on maximizing throughput while
OSCAR enabled servers. overlookingother user and system goals like minimizing cost
We evaluate OSCAR via implementation on Linux, as and energy consumption.
well as via simulation, and show its ability to increase the Recently, developing a deployable bandwidth aggregation
overallsystem throughputwhile achievingits cost and energy system has been investigated [14], [15], [23]. DBAS focuses
efficiency targets. The results show that, with no changes to only on maximizing the overallsystem throughput[15], [23].
currentInternetarchitectures, OSCAR reaches the throughput On the other hand, even though OPERETTA is deployable
upper-bound. It also provides up to 150% enhancement in and energy-aware, it does not utilize the interfaces to their
throughputcomparedtothecurrentOperatingSystemswithout maximum in case of non-OPERETTA enabled servers [14].
any change to legacy servers. Our results also demonstrate In addition, similar to previous solutions, they only focus
OSCAR’sabilitytomaintaincostandtheenergyconsumption on utilizing the available interfaces in the solitary form,
levels in the user-defined acceptable ranges. overlooking available connectivity at neighboring devices as
Theremainderofthispaperisorganizedasfollows.Section well as cost efficiency.
II discusses the related work. A motivational scenario along
B. Collaborative Solutions
with the intuition of our solutions is presented in Section
III. We then present the overall architecture of our system There exists some shy attempts to utilize the available
in Section IV. Section V formulates the OSCAR scheduling interfacesinthe collaborativeform[18]–[22].Thesesolutions
problem, after which we present OSCAR’s communication are motivated by the increase of the devices density and
protocolsin Section VI. In Section VII, we discuss our Linux are aimed at sharing neighboring devices connectivity. Un-
OS implementation and evaluate OSCAR’s performance and fortunately,thesesystemshavemanyshortcomings.Examples
compare it to the current solutions. Finally, Section VIII include requiring Internet infrastructure updates such as the
concludes the paper and provides directions for future work. existenceof proxyservers[18],[19],[21],updatingthe avail-
able applications to interface with the new API and specify
II. RELATEDWORK
their requirements [20], developing applications that are fully
A. Solitary Solutions responsible for handling and making use of the collaboration
Manysolutionshaveemergedtoaddresssingle-deviceband- for their own benefits [22]. In addition, all these solutions
width aggregation at different layers of the protocol stack focus only on maximizing throughput and usually ignore
[4].Applicationlayersolutionseitherassumethatapplications developing incentives systems that enable this collaboration.
are aware of the existing network interfaces and take the The closest system to OSCAR in terms of incentivized
responsibility of utilizing them for their needs [12] or rely bandwidth sharing is FON [3]. FON is a commercial system
on modifying the kernel socket handling functions to enable thatenablesuserstosharetheirhomebandwidthforobtaining
existing applications use multiple interfaces [12], [13]. Such WiFi coverage via other worldwide FON users. Basically,
modifications require changes to legacy servers in order to each user installs a Fonera router in her home network. This
supportthese new sockets.In addition,[13]requiresfeedback Fonerarouterisresponsibleforsharinga portionoftheuser’s
from the applications about their performance, and hence bandwidthwithotherFONsystemusers.Themaindifferences
is not backwards compatible with previous versions of the between FON and OSCAR are: (1) FON does not aggregate
applications. theavailablebandwidthneitherinadevicenorincollaborative
Manybandwidthaggregationtechniques,however,naturally devices.(2)FONusesspecialrouter(Fonera).(3)FONrouters
lieinthetransportlayer[8]–[11].ThesesolutionsreplaceTCP are static while OSCAR targets mobile devices. (4) The user
withmechanismsandprotocolsthathandlemultipleinterfaces. is unable to connect to more than one Fonera. (5) FON users
Virtual Bank John
Alice BVailratunacle MAlaicrek Applications:
WiFi Connection-Oriented
OSCAR OSCAR Client
Bluetooth
Context Awareness Packet-Oriented
Incentives Manager
Legacy Server Managger Scheduling Manager
The Internet
3G Packet-Oriented OSCAR User Interface MMooddee DDeetteeccttiioonn Tendering Price
John Resume Module Module Determiner
Supporting
WiFi Mark LSeegravecyr ACpphalicr.a Etiosntim Taratoffric Resume Module aTnedn Fdieltresr iCngo llMecotdiounle
3G
OSCAR-Enabled Server Neighbor Discovery Contracts
Module Assessment Module
Connection #1 Packets John’s Incentives Packets
CCoonnnneeccttiioonn ##23 PPaacckkeettss MAalicrke’’ss IInncceennttiivveess PPaacckkeettss NCethwaorr. kE Isnttiemrafatocers Balance Handler
Scheduler
Battery Sensor Reservation Manager
Fig.1. OSCARscenario
are incetivized by the need of being connected to the Internet Backward Compatibility Manager
Packet Reordering Network Address Connection
while being away from their home network. OSCAR builds
Module Translation Module Redirection Module
on this needas well, butalso addsa tangiblefinancialpricing
system to further encourage collaboration.
Network Interfaces:
III. MOTIVATING SCENARIO AND SOLUTION
Most mobile devices today are equipped with multiple Fig.2. OSCARclient architecture
heterogeneousnetworkinterfaces.Forexample,John’smobile
resumefunctionalitytoschedulethedataatafinergranularity
deviceisequippedwithWifi,Bluetoothand3G.WhenJohnis
in order to enhance the overall system performance. It would
sitting in a mall, waiting for a train at the station, or having a
also be equipped with an incentive mechanism in order to
meal,hetypicallywatchesyoutubevideos,listenstopodcasts,
convince John’s neighbors to share their connectivity with
uses facebook to get his social network feeds, and opens his
him.Finally,thissystem wouldadoptan efficientandsecured
favoritenewscites.Heconnectshismobiledevicetoavailable
communication protocol to make the collaboration between
Wifi hotspots while being connected to his 3G network and
the different users feasible.
leaving his Bluetooth idle. In such places John has a lot of
Figure 1 shows a scenario, based on OSCAR, in which
people around him such as Mark who is not using his 3G
John’s device is able utilize all the available connectivity
Internet connectivity and Alice who is connected to a high
optionsincludingMark’sand Alice’s under-utilizedinterfaces
bandwidth WiFi network but is not currently using it.
to achieve John’s requirements and enhances his experience.
Unfortunately, John’s current operating system assigns all
In this scenario, we notice that while communicating with
his connections to Wifi. Consequently, John has to wait until
the legacyservers the device schedules the connectionsto the
theyoutubevideoisbufferedinordertowatchitcontinuously
paths in a connection-orientedmode, in which all the packets
without disruption; meanwhile other applications are slowly
belonging to the same connection utilizes the same path.
retrieving their content. With the high contention on the
However,whilebeingconnectedtoanOSCARenabledserver
Wifi interface, the available bandwidth is degraded, and John
orresume-supportinglegacyserver,itusesfine-grainedpacket-
disconnects his device from Wifi to utilize his 3G connec-
oriented scheduling for further performance enhancements.
tion. Although, this increases his throughput, the available
We also observe that there is also a virtual bank, which is
bandwidth is not sufficient to smoothly retrieve the content,
part of the incentive system, that John uses to pay for the
he incurs the added overhead of restarting his non-resumable
bandwidth he shared from Mark and Alice’s. We now move
connections, and pays a higher cost in battery and money for
ontodiscussingthearchitectureandsystemforenablingsuch
using 3G.
scenario.
John’s needs can be satisfied by concurrently utilizing his
available network interfaces as well as the under-utilized IV. THE OSCAR SYSTEM
connections of Mark and Alice. John’s applications traffic In this section, we start by describing our system assump-
can be scheduled across different interfaces and neighbors tions. We then provide an overview of OSCAR’s architecture
in order to enhance his user experience. Such a scheduler followed by a detailed description of its main components.
should take the network interface characteristics, bandwidth
A. Assumptions
heterogeneity, and different application traffic characteristics
into account when scheduling different connections to the OSCAR works in a distributed environment, where a node
availableinterfaces.Furthermore,theschedulershouldexploit can share and use the bandwidth available from its OSCAR-
servers supportfor the resume functionalityin order to seam- enabled neighbors to connect to both legacy and OSCAR-
lessly migrate connections assigned to an interface that got enabled servers. For this, we assume that all OSCAR com-
disconnected to another. The scheduler can also leverage this ponentsare trusted. Handling security aspects of the software
modulesis outside the scopeof this paper.We furtherassume (implementedthroughthe BackwardCompatibilityManager);
that there is a trusted entity, called the virtual bank, that is which enables OSCAR’s clients to make use of the available
responsibleforauthenticatingOSCARusersandkeepingtrack connectivity opportunities without requiring any changes to
of the virtual currency exchanged by users for the bandwidth legacy servers, network infrastructure, or legacy applications.
(as detailed in sections IV-F and IV-I). Bandwidth sharing Third,leveragingconnectionresume-supportinglegacyservers
agreement between two nodes is performed in rounds, each andOSCAR-enabledserversforfurtherperformanceenhance-
withafixeddurationdeterminedatthebeginningoftheround. ments (implemented through the Packet-Oriented Scheduling
To determine the cost of the bandwidth at the selling node, Manager). Fourth, Handling the user incentives and collabo-
OSCAR has a module that suggests an initial cost based rating with the trusted bank module in handling the payment
on different factors (as detailed in Section IV-F1). The user, transactions implemented through the (Incentive Manager).
however,canmanuallyadjustthecost,e.g.toofferbandwidth Finally, optimally scheduling the connections and/or packets
sharingforherowndevicesforfree.Oncethepriceisfixed,it onthedifferentpathsinordertoachievetheuserrequirements
cannot be negotiated until the next round. The buyer collects (implemented in the OSCAR Scheduler).
the different offers from the neighboring buyers and selects Note that, OSCAR server architecture is a subset of this
the ones that best fits its needs. A buyer can re-sell her clientarchitecturecontainingthefollowingmodules:(1)Mode
bandwidth to other nodes, allowing for multi-hop bandwidth Detection Module, (2) Packet Reordering Module, (3) Sched-
sharing. OSCAR modulesaddress the cost and loopingissues uler,(4) Network InterfacesCharacteristics Estimator,and (5)
associated with bandwidth re-selling. Network Address Translation Module. For the balance of this
section,wedescribeeachofthesecomponentsinmoredetails.
B. Communicating Entities
D. Context Awareness Manager
Figure 1 shows that we have five main communicating
entities in our architecture. Firstly, client devices equipped One of the key factors that enables OSCAR to efficiently
with multiple network interfaces varying in their available utilize all available connectivity options is its ability to de-
bandwidth, energy consumption rates and cost per unit data. termine, store, and utilize the mobile device’s context infor-
Each interface can be used to connect directly to the Inter- mation accurately and efficiently. This is the responsibility
net and/or through neighboring client devices sharing their of the context-awareness manager. This component consists
connectivity options. These options vary in their usage cost of the following five modules: the User Interface Module,
and available bandwidth to be shared. We call each of these the Application Traffic Characteristics Estimator, the Neigh-
possible ways to connectto the Interneta path. Each of these bor Discovery Module, the Network Interface Characteristics
devicesrunsmultipleapplicationsthatvaryin their character- Estimator, and the Battery Sensor.
istics. Secondly, a virtual bank which is a trusted third party 1) UserInterfaceModule: OSCARprovidesauser-friendly
that handles the payment transactions between the different interface that enables users to set their requirements, prefer-
devices as well as the authentication related aspects. Thirdly, ences,andschedulingpolicies.Thisalsoenablesuserstoinput
legacyserverswhicharetypicalnon-modifiedInternetservers. other contextual parameters like the interface usage cost (e.g.
The clients use the connection-orientedmode to connectwith cost of using 3G or the hotel Wifi). Finally, this module is
them as its default mode of operation. In this mode, OSCAR also used to express willingness to share bandwidth, set the
schedules different connections to the available paths such amount to be shared, and define the needed revenue in return
that a connection can be assigned to only one path. Once for consuming the device’s battery to aid others.
assigned, all the packets belonging to this connection utilize 2) Application Traffic Characteristics Estimator: Knowing
thesamepath.Fourthly,connectionresume-supportinglegacy theapplicationtrafficcharacteristicssignificantlyimpactsOS-
servers, e.g.FTPandHTTPserversthatsupportresumingthe CAR’s performance.However,to be fully deployable,we can
connections. OSCAR leverages these servers to enhance the not require any changes to existing applications in order to
performance by switching to a packet-oriented mode, where determine this crucial information. Our approach is to auto-
each packet or group of packets can be independently sched- matically estimate application characteristics based on their
uled on a different path. Finally, OSCAR-enabled servers, operational history on given port numbers. These estimation
running our software modules, which OSCAR leverages for techniques enable us to accurately determine the application
enabling highly efficient packet-oriented scheduling. traffic characteristics since most popular applications have
specific ports reserved for their communication and exhibit
C. Architecture Overview relatively consistent behavior. We estimate the average con-
Figure 2 shows the OSCAR’s client architecture. The nection data demand in bytes for each port as follows:
architecture implements five core functionalities: First, con- After a connection is terminated, the module updates the
text awareness (implemented through the Context Awareness estimated values of connection data demand (Cdemand) as:
Manager). It enables OSCAR to sense its environment; in-
Cdemand(i)=(1−ρ)Cdemand(i−1)+ρTCdemand(i) (1)
cluding user requirements, applications traffic characteristics,
attached interfaces characteristics, battery status, and nearby where TCdemand(i) is the number of bytes transmitted by
OSCAR-enabled devices. Second, backward compatibility the ith connection that uses this port number which has just
TABLEI such opportunities is the responsibility of the packet-oriented
POWERCONSUMPTIONTABLE[25] scheduling manager. This module exploits the availability of
OSCAR-enabledserversorserversthatsupporttheconnection
Interface Tech. ∆Power1 Datarate
Netgear MA701 Wifi 726mW 11Mbps resumptiontoenhancetheperformanceofOSCAR.Itcontains
LinksysWCF12 Wifi 634mW 11Mbps two sub-modules:the mode detection module and the resume
BlueCore3 Bluetooth 95mW 723.2Kbps
module.
finished,Cdemand(i)isthenewestimateoftheconnectiondata 1) Mode Detection Module: This module detects whether
demand,andρisasmoothingcoefficient,takenequalto0.125 the end point of the connection supports OSCAR or not
toevaluatetheequationefficiently.Thisway,theestimate can in order to enable the optional packet-oriented mode. It is
be obtained by an efficient shift operation. implemented as a service that monitors the exchanged data
3) Neighbor Discovery Module: This module is responsi- packetsbetweenthetwo endpointsandaltersthe TCPpacket
ble for discovering the existence of nearby OSCAR-enabled header. When the client starts a new connection, the module
devices. It selects the interfaces that are going to be used in alters the TCP header adding OSCAR existence flag in the
the discovery process, determines the frequency of sending optionpartoftheTCPheader,iftheserverrespondssimilarly,
neighbor discovery queries, maintains the lists of the discov- this indicates that OSCAR is supported on the server. Hence,
ered neighbors, and communicates with the virtual bank to packet-oriented mode can be utilized. If so, OSCAR will
authenticate these neighbors. schedulethepacketsthatbelongtothisTCPconnectiononthe
4) NetworkInterfaceCharacteristicsEstimator: Thismod- available paths. If OSCAR is not supported on the server, the
ule is responsible for estimating the characteristics of each client resorts to the default connection-oriented mode unless
network interface. In particular, it estimates the available the connection is resumable.
bandwidth, energy consumption rates, the used physical data This approach enables OSCAR to efficiently determine the
rates,andtheachievablemaximumdatarateforeachinterface. operation mode without incurring extra packet transmission
To estimate the available bandwidth, it periodically connects overhead. It also enables us to avoid extra delay since imme-
through each interface to various geographically dispersed diate scheduling in connection-orientedmode occurs until we
serversthat supportthe packet-pairestimation technique[24]. determine the need to switch to packet-oriented mode.
On the other hand, when communicating with OSCAR- 2) Resume Module: This module is utilized when com-
enabledservers,themoduleperiodicallysendsthedatapackets municating with legacy servers, with the purpose of enabling
in pairs and marks them in order to be used for estimating packet-oriented scheduling even if the server is not OSCAR-
theavailablebandwidthusingthepacket-pairtechnique.These enabled. To detect whether a particular server supports the
estimates are sufficient since bandwidth bottlenecks typically resume functionalityor not, we build a web service maintain-
exist at the client’s end not the server’s, which is typically ing a list of protocols that support the resume functionality.
connectedto high bandwidthlinks and designed to scale with OSCAR periodically connects to this web service to update
the number of clients. and cache this list of protocols. Each protocol is represented
Tocharacterizetheenergyconsumption,TableIshowshow by a template that uniquely identifies the protocol, e.g. its
the power consumptionof a networkinterface dependson the port number, as well as how to check with the end server
NIC, the technology used, and the physical transmission data that it supports the resume functionality, e.g. checking for
rate[25].Hence,inordertomakeOSCARestimatetheenergy the “Accept-Ranges” header option in HTTP. The OSCAR
consumptionforitsinterfaces,webuildanonlineservicewith Resume Module uses such protocol information to determine
a database containing energy consumption rates for various whether the server supports the resume functionality or not,
networkinterfacesatdifferentphysicaltransmissiondatarates. and accordingly enables the optional packet-oriented mode.
Once OSCAR runs, it queries the database for the energy The resume module is implemented as an in-device
consumption rates of each network interface attached to the proxy responsible for identifying the resumable connections,
device. scheduling sub-connections across different device interfaces,
Estimatingthephysicaltransmissiondatarateisthesimplest and closing these sub-connections once they terminate. To
task since it can be queried using the supported OS API. It minimize the latency and buffer size the application needs
is used in additionto the queriedenergyconsumptionrates to to wait for the in-order delivery of data, the module uses
determinetheinterface’scurrentenergyconsumptionrate.We twoapproaches:(1)itdividestheconnectionintofixedlength
usetheestimatedphysicaltransmissionrateasanestimatefor units, where a unit with an earlier offset from the beginning
the maximum achievable data rate for the interface. of the connection is retrieved over the multiple interfaces
5) Battery Sensor: This module senses whether the device before the next unit is retrieved. Each interface is assigned
ispluggedto apowersourceorrunningonbatteryalongwith a portion of the unit to retrieve based on weights decided
the available battery level. by the scheduler. (2) It sorts the interfaces in a descending
order according to their scheduling weight and starts the
E. Packet-Oriented Scheduling Manager
sub-connection with the larger weight first. This way, larger
Utilizing the opportunities of packet-oriented scheduling is amounts of data will arrive in order from the beginning of
one of the key features introduced by OSCAR. Detecting the connection. For further optimization, if the resumable
connection supports range requests, it does not close the sub- status2 and other system parameters. The device owner can
connection once the load is finished but reuses it for the next agree with this initial price suggested by the system or
load assigned to that interface. This optimization technique preemptivelychangeitthroughtheUser InterfaceModule.To
avoids the extra delay of increasing the congestion windows determine an initial price, multiple cost parameters are taken
to its optimal size introduced by TCP slow start. into account including the service provider cost, the energy
On the other hand, this module also handles interface consumptioncost,andthemarketstatus. Thiscostisfixedfor
disconnectionwhichincreasesinthecollaborativemodedueto a fixed amount of time and needs to be renewed in order to
devicemobility.Itdoessobymonitoringeachsub-connection handle mobility and disconnections of devices.
ensuring it will receive its portion. If one or more of the sub- a. Service Provider Cost: Since a node may be paying for
connections’ paths became unavailable because of disconnec- its own Internetaccess (either through an Internetprovideror
tion,itre-schedulesthissub-connectionononeoftheavailable a neighboringsharing node),this node needs to at least cover
runningpaths.Theschedulerisresponsiblefordeterminingthe such cost in order to avoid incurring a loss. Hence, the first
appropriate path to be used in this case. portion of the incentives price will be ci which is the service
provider cost per unit data of using interface i.
b. Energy Consumption: Sharing bandwidth by enabling
F. Incentives Manager multiple network interfaces will consume extra power, which
isanextremelycrucialresourceespeciallyforbatteryoperated
The success of a collaborative bandwidth aggregation sys-
mobile devices. Hence, a device sharing its bandwidth needs
tem is tightly coupled with its ability to encourage users to
some compensation for its consumed energy. Noting that a
collaborate and share their available bandwidths. Incentive
selling device uses two of its interfaces (one connected to
systems have been widely studied in literature for a wide
the Internet, i, and the other connected to the buyer, j),
array of applicationswhere users share resources; approaches
calculatingtherequiredcompensationshouldtakeintoaccount
proposed generally fall into (1) reputation-based approaches
the following parameters: (1) the energy consumed to relay a
and (2) credit-based approaches.
unit of data for interfaces i and j would equal (e +e ), (2)
i j
In reputation based incentive approaches [26], [27], nodes
the battery capacity (E ), and (3) the remaining battery
Capacity
earn a reputation based on their collaboration with their
ratio(R ).Hencewecalculatetheenergycompensation
remaining
neighborswhothenspreadthisreputationaccordingly.Hence,
factor (EC ) as follows:
ij
the nodes carry the overhead of monitoring their neighbors
e +e
and spread their observations in the network to enable other EC =[γ+(1−R )] i j (2)
ij remaining
nodes to determine their reputation level and act accordingly. ECapacity
These approaches,however, are typically suitable for systems Where γ is a binary value reflecting whether the device is
with stable memberships that last long enough to earn high connected to a power source or not. If connected, γ = 0,
reputations.Theyarenotsuitablefordeployablecollaborative the user still needs to be compensated for the consumed
bandwidth aggregation systems, where nodes will randomly energy as part of the charging power is still used in the
meetatrandomlocationsatanypointin time(e.g.passengers bandwidth sharing. Therefore, the importance of your energy
at an airport or a train station). consumptiondegradestillitreaches0whenthebatteryisfully
Credit-based systems are more suitable for such networks charged.Ontheotherhand,ifthedeviceisrunningonbattery
since they usually rely on a trusted third party that maintains power, γ = 1 guarantees receiving appropriate compensation
credit for the communicating nodes [28], [29]. Like free even if the battery is fully charged.
markets, nodes in such systems collaborate with each other c. The Market Status: With potentially multiple devices
and pay for the services acquired from one another. requesting and offering bandwidth, the demand and supply
The Incentives Manager in our architecture is designed mini-market status amongst these devices is a crucial pricing
to enable bandwidth sharing amongst users by adopting a factor. A balanceis requiredin orderto maximizethe sharing
credit-basedsystem. Itis particularlyresponsiblefor handling node’sbenefit while providingpriority to users willing to pay
virtual currency exchange and bandwidth reservation. These more for important data they need to transmit. We therefore
functions are implemented using the following five modules: calculate the market dependent cost (MC) as follows:
the Tendering Price Determiner, the Tenders Collection and
Filtering Module, the Contracts Assessment Module, the Bal- MCn,i =max[MCn−1,i+
ance Handler, and the Reservation Manager. (1+κ)B −B (3)
i,Requested i,Offered
η,0
1) Tendering Price Determiner: This module runs at the
B
seller and is responsible for determining a price to offer (cid:18) i,Offered (cid:19) (cid:21)
Where i is the index of the interface connected to the
for sharing a device’s bandwidth on a given interface for a
fixed amount of time. It proposes an initial price covering Internet that we share its bandwidth, MCn−1,i is the market
status cost during the previous estimation session, B
the cost for using the shared bandwidth at interface j (which i,Requested
is connected to the Internet) while communicating with a
2Environmentalstatusisthecurrentmarketstatusincludingrequestsonthe
buyerthroughthesellerinterfaceibasedontheenvironmental sellerandreservations.
is the sum of the requested bandwidth for reservation during 4) Balance Handler: This module is responsible for han-
the last session from such interface (i.e. demand), B is dling the payment transactions with the virtual bank. It also
i,Offered
theofferedbandwidthfromthatinterface(i.e.supply),η isthe makessurethatbuyernodedoesnotmakerequeststhatexceed
costincreasefactordeterminedbytheuser,andκisarevenue its available credit.
multiplier.The reasonbehindusing the revenuemultiplier(κ) 5) Reservation Manager: This module is designed to keep
is to allow for increasing the price when the supply is equal track of the reserved bandwidth on each node to avoid using
to the demand and is set to a small value (0.1 in our case). aninterfaceaboveitscapacity.Itbuildsareservationtablethat
Theintuitionisthataslongasthedemandishigherthanthe contains the neighbor identifier, the reserved bandwidth, and
supply, we keep increasing the market status cost value. This time stamp for the most recent usage. A reservation time out
increase is a function of the cost variation factor (η) as well (RT ) is used to avoid indefinite reservation that may occur
Out
as the normalizeddifference between the demandand supply. due to mobility or the existence of a malicious node.
The Incentive Price: After discussing the various factors
that should be taken into consideration while calculating the G. Backward Compatibility Manager
requiredincentivespriceweputthemtogetherinthefollowing
In order to make OSCAR deployable in today’s Internet, it
equation:
shouldbebackwardscompatiblewithboththe currentlegacy
servers and applications. This is the responsibility of the
I =νEC +MC +c backward compatibility manager that consists of the Packet
n,i,j i,j n,i i
e +e Reordering Module, the NAT Module, and the Connection
i j
=ν[γ+(1−Rremaining)] +ci+ Redirection Module.
E
Capacity
+ 1) PacketReorderingModule: Thismoduleisactivatedfor
(1+κ)B −B
MCn−1,i+ i,Requested i,Offered η the packet-oriented mode to handle packet reordering issues
B
(cid:20) (cid:18) i,Offered (cid:19) (cid:21) introduced by transmitting data using multiple network inter-
(4)
faces to be compatible with legacy applications. It delays the
packetsaswellastheiracknowledgmentsbeforepassingthem
Where ν is an energy-cost conversion factor that maps the
to the upper layer. This is designed to avoid the performance
energycost (EC ) to monetarycost. The valueof ν is set by
i,j degradation introduced by using TCP over multiple paths. In
the virtual bank based on the current fair market prices.
ordertoavoidbeingover-protective,itmaintainsanestimation
At the end, this incentive price is proposed to the user and
of the TCP’s RTT in order to forward the out-of-order data
she can freelychangeit seeking moreor less profit.Although
to TCP before the timeout to prevent excessive drops in the
setting I >c can be used to avoid loss and loops, a user
n,i,j j congestion window.
has the ability to set the price even to zero to satisfy cases
2) Network Address Translation Module: When a connec-
when she shares bandwidth with other devices belonging to
tion go through a sharing neighbor to a legacy server, since
her or to family members. The IP of the interface with the
a legacy server can see only one IP address for the client,
farthest OSCAR device in the chain is always used in the
OSCAR needs to change the IP address in the packet to the
tenders to detect and avoid loops.
neighboring client IP. Otherwise, the reply from the legacy
Oncethiscostisdeterminedandsenttothebuyer,thebuyer
server will go directly to the client, rather than to the sharing
either accepts or rejects it.
neighbor,removing the benefit of sharing the bandwidth with
2) Tenders Collection and Filtering Module: This module neighbors.
runs at the bandwidth requestor and is responsible for col-
To address this, we implement a Network Address Trans-
lecting tenders from nearby OSCAR enabled devices. These
lation (NAT) module at each node which is activated on the
tenders contain the path available bandwidth and the price of
sharing(seller) node.This moduledoesthe NATingoperation
transmitting a unit of data in this path (as calculated by the
forthesentpacketsandreversesthisoperationforthereceived
Tendering Price Determiner at the seller). After successfully
packets from the legacy server.
collectingthetenders,themodulesortsthembasedoncostand
3) ConnectionRedirectionModule: Thismoduleisrespon-
forwardsthe least cost ones whose aggregatebandwidthsatu-
sibleforredirectingthenewlyrequestedconnectionstothethe
rates the requester bandwidth to the scheduler. This way, the
Resume Module in order to enable utilizing the connection
scheduler can take the available bandwidth throughneighbors
resumes while hiding this from legacy applications.
in its decision.
3) Contracts Assessment Module: When two neighbors
H. Scheduler
agree on the set of terms for bandwidth sharing (i.e. cost,
reserved bandwidth, and duration), both the buyer and seller This is the heart of the OSCAR system. It takes its
monitor the traffic to regulate and ensure these terms. Note input from all the previously mentioned modules in order
that this is enforcedby the trusted OSCAR components. This to schedule the connections and/or packets to the different
module is also responsible for renewing the contract before it available paths. We discuss the OSCAR scheduler in details
expires. in Section V.
I. Virtual Banks TABLEII
LISTOFSYMBOLSUSED
Thisisatrustedthirdpartythatmaintainsuseraccountsand
creditbalances.Itisalsoresponsibleforhandlingthepayment Symbol Description
process between the different users as well as OSCAR’s T Theoverall systemthroughput
L Thecurrent systemload
authentication. Li Thecurrent systemloadforstreami
Si Determines whether streamiisconnection-based (1)
V. OSCAR SCHEDULER orpacket-based (0)
Inthissection,weformulatetheoptimalOSCARscheduling bj Theeffective bandwidthofpathj
rj Thedatarateofpathj
problem. We begin by describing our system model followed aj Difference inpowerbetweenactive andidlestatesofpathj
by the optimal scheduling problem. E Theenergyconsumedinordertotransferthesystemload
Eavg Theaverage energyconsumedperunit
A. System Model datawhiletransferring thesystemload
Ej Theenergyconsumedforpathj totransferitsload
Table II summarizes the system parameters and notations. cj Thecostperunitdataofpathj
We assume a mobile device with m different paths to the C Thesystemloadtransferring cost
Cavg Theaverage costperunitdataoftransmitting thesystemload
Internet.Eachpathrepresentsaway toconnecttotheInternet Cj Pathj costoftransferring itsload
either by using the Interface’s direct connectivity or by using ∆j Pathj neededtimeforfinishingitsload
theinterfacetocommunicatewithoneofitsneighborssharing xij Forconnection-oriented streams,equals 1ifstreamiis
assignedtointerface j.Equals0otherwise.
its Internet connectivity. Each of those paths has its effective
wj Theratioofpackets assignedtointerface j
bandwidth b and cost per unit data c , which can be the n Numberofactive streamsincluding thenew request
j j
service providerusage costor the costpaid to the neighborin m Numberofdifferent paths
ordertousetheirconnectivity.Inaddition,eachpathusesone
mode(SectionV-B4).Finally,wepresentourproblemsolution
network interface and it has an energy consumption rate a ,
j (Section V-B5).
wherea equalsthedifferenceinpowerconsumptionbetween
j 1) Throughput Maximization Mode: In this mode, the
the active and idle states of the used interface. The data rate
scheduler’s goal is to maximize the system throughput under
of each path interface is denoted as r . The device runs a set
j certainenergyandcostconstraints.Inparticular,theuser puts
of connections that share these interfaces and varies in their
a limit on the average cost per unit data (C ) as well
characteristics. avg,Target
as a limit on the energyconsumptionper unit data (E )
Ourschedulingunitisaconnectionorapacket.Wereferto avg,Target
that should not be exceeded.
a standardnetworkconnectionas a stream to avoid confusion
Objective Function:
withtheschedulingunit.Schedulingdecisionsaretakenwhen
Theobjectiveofthescheduleratanydecisioninstanceisto
a new stream (number of streams active in the system is n,
maximizetheoverallsystemthroughput(T).Giventhesystem
including the new stream) is requested from an application.
load (L), the objective can be written as:
TheModeDetectionModule(SectionIV-E1)andtheResume
Module(Section)thendeterminewhethertheoperationmode L
MaximizeT = (5)
isconnection-based(Sn =1),orpacket-based(Sn =0)when maxj∆j
theotherendisOSCAR-enabledorsupportstheresumemode.
where ∆ is the time needed for path j to finish all its
In the former case, the scheduler’s goal is to determine to j
load (connection and packet based). Since L is constant, the
which path it should be assigned (sets x = 1 for only
nj
objective function is equivalent to:
one path j). In either case, the percentage of packets to be
assigned to each path, i.e. paths relative packet load (w ),
j Minimize max∆
should be re-calculated based on the current system load (L). j j
Our OSCAR scheduler has three modes of operation based
n n
on user preferences: (1) throughput maximization, (2) energy =Minimize max i=1(Li(1−Si)wj)+ i=1(LiSixij)
minimization, and (3) cost minimization. j (cid:18)P bj P (cid:19)
(6)
B. Optimal Scheduling Where the left summation represents the packet-oriented
In this section, we formulateour schedulerand its different mode load (each term, Li(1−Si)wj, is the numberof bytes
modes of operation. Generally, the decision variables are: (1) fromthepacket-orientedstreamiloadassignedtopathj)and
If S =1, which path to assign the new stream n to (variable the right summation is the connection-oriented mode load.
j
x ) and (2) the new values for w , ∀ :1≤j ≤m. Note that any stream i will be either connection-oriented
nj j j
WestartbypresentingtheformaldefinitionoftheThrough- (Si = 1) or packet-oriented (Si = 0) and thus will appear
put Maximization Mode (Section V-B1). Then, we present in only one of the two summations. Dividing the sum of two
the formal definition of the Energy Minimization Mode (Sec- loads by the available bandwidth on that path (bj) gives the
tion V-B2). After that, we present the formaldefinition of the time needed for path j to finish its load.
CostMinimizationMode(SectionV-B3)followedbythecom- Constraints:
mon constraints that must be satisfied regardless of operation The following constraints must be satisfied:
Target Cost: As we mentioned, the user puts a limit
(C ) on the average cost she is willing to pay per unit
avg,Target m n
a
data. Minimize E = j w (L (1−S ))+L S x
j i i n n nj
r
Cavg ≤Cavg,Target (7) j=1 j i=1 !
X X
(13)
Since Where: aj refers to the energy consumed per unit data.
C = mj=1cj[ ni=1(Li(1−Si)wj)+ ni=1LiSixij] wj ni=1(Lrij(1−Si))+LnSnxnj the load assigned to inter-
avg L facej exceptthepreviouslyassignedconnectionorientedload
P P P (8) (a P/r ) n−1L S x . We removed this part from the equa-
Substituting in (7): j j i=1 i i ij
tion since it is constant and will not affect the minimization
P
m n n process.
cj wj (Li(1−Si))+ LiSixij ≤LCavg,Target Constraints:
!
j=1 i=1 i=1 The following constraints must be satisfied:
X X X
(9)
Target Throughput: As we mentioned, in this mode, the
Where c is the averagecost per unitdata while using path
j client’s device has to achieve at least a certain level of
j. n (L (1−S )w )+ n L S x istheloadassigned
i=1 i i j i=1 i i ij throughput(T ). Therefore,
to path j as in (6). Target
P P
Target Energy: Similarly, the client’s device can tolerate L
T = ≥T (14)
up to a certain level of the average energyconsumed per unit max ∆ Target
j j
data.
where ∆ is the time needed for path j to finish all its load
E ≤E (10) j
avg avg,Target
(L)(connectionandpacketbased).Thethroughputistheratio
Since betweentheloadonthesystemandthetimeneededtotransfer
m aj [ n (L (1−S )w )+ n L S x ] this load.
Eavg = Pj=1 rj Pi=1 i Li j Pi=1 i i i(j11) →∀j, ∆j = ni=1(Li(1−Si)wj)+ ni=1(LiSixij) ≤ L
b T
Substituting in (10): P j P Target
Rearranging:
m n n
a
j
(Li(1−Si)wj)+ LiSixij ≤LEavg,Target n n−1
r Lb
Xj=W1hejr e Xia=j1is the average energXiy=c1onsumpti!on per unit d(1a2ta) →∀j, wjXi=1(Li(1−Si))+LnSnxnj ≤ TTargjet−Xi=1(Li(S1i5x)ij)
rj
while using path j. Note that the RHS is constant.
2) Energy Minimization Mode: In this mode, the sched- Target Cost: Similar to the throughputmaximization mode
uler’sgoalistominimizingtheoverallenergyconsumedunder (Section V-B1), the target cost constraint is represented by
certain throughputand cost constraints. In particular, the user Equation 9
puts a limit on the average cost per unit data to Cavg,Target and 3) Cost Minimization Mode: In this case, the user is
requests minimum throughput of Ttarget interested in achieving at least a certain level of throughput
Objective Function: T with limitations on the average energy consumed per
target
The objective of the scheduler at any decision instance is unit data E
avg,target
to minimize the overall system energy consumption (E). This Objective Function:
objective can be written as: The overall objective of the scheduler is to minimize the
overall system cost needed (C) to consume the system load:
Minimized E = E
j
Minimized C = C
j j
X
j
where, E is the energy consumption of path j. This energy X
j
consumptioncan be dividedinto two parts:(1)energyneeded where, C is the cost needed for path j. This cost can be
j
to finish the load of the connection-oriented streams and (2) dividedintotwoparts:(1)costneededtofinishtheloadofthe
energyneededtofinishtheloadofthepacket-orientedstreams. connection-oriented streams and (2) energy needed to finish
The former is equal to (a /r ) n L S x where a /r is the load of the packet-oriented streams. The former is equal
j j i=1 i i ij j j
the energy consumed per unit data while using path j and to c n L S x j, where c is the cost per unit data for
n−1L S x istheconnectionPorientedloadassignedtopath pathjj ani=d1 ini Li S x j thejamountof data assigned to this
j.Si=im1ilairlyi,tihjelaterisequalto(a /r ) n L (1−S )w . path iPn conneci=tio1n-ioriienitedmode.Similarly,the later is equal
SPince all the connection-orientedsjtreajms biu=t1theinewlyiarrivj- to c n PL (1−S )w . Since connection-orientedstreams
P j i=1 i i j
ingonecannotbere-assignedtootherinterfaces,theobjective cannot be re-assigned to other paths, the objective function
P
function becomes: becomes:
C. Discussion
m n In some cases, the selected constraints by the user may
Minimize C = c w (L (1−S ))+L S x lead to an infeasible solution. For example, it may be the
j j i i n n nj
j=1 i=1 ! case that there is no assignment strategy that achieves both
X X
(16)
the cost and energy consumption constraints specified by the
Constraints: user concurrently. To eliminate this possibility and make the
The following constraints must be satisfied: selectionprocessuserfriendly,wedesignaniterativeselection
TargetThroughput:Similartothethroughputmaximization policy where the user selects her top priority constraint first
mode (Section V-B2), the target cost constraint is represented (e.g.limitoncost).Basedonthisselection,OSCARcalculates
by Equation 15 the feasible range for the second constraint (e.g. energy
Target Energy: Similar to the throughput maximization consumption) and the user is asked to choose a value from
mode (Section V-B1), the target cost constraint is represented this range.
by Equation 12
VI. OSCAR COMMUNICATION PROTOCOLS
4) Common Constraints: Here, we present our set of con-
strains that must be satisfied regardless of the scheduling In order to implement OSCAR, we developed two sim-
mode. ple protocols. First, a collaboration protocol that handles
Integral Association: If the new stream is connection- bandwidth sharing amongst collaborating nodes. Second, an
oriented, it should be assigned to only one path: OSCARservercommunicationprotocolfordetectingOSCAR-
enabled servers and exchanging control information between
m the client and server. In this section we give an overview on
x +(1−S )=1 (17)
nj n our OSCAR communication protocols followed by a detailed
Xj=1 description for each protocol.
Note that when S = 0, x = 0,∀j, which is the case
n nj A. Protocols Overview
when the new stream is determined to be packet-oriented by
In order to enable the collaboration between nodes, the
the Mode Detection Module.
OSCARcollaborationprotocolaimsto:(1)efficientlydiscover
PacketLoadDistribution:Forpacket-orientedstreams,their
the neighbors, (2) authenticate both buyers and sellers, (3)
total load should be distributed over all interfaces:
enable cost agreement and (5) and guarantee the payment.
m On the other hand, the OSCAR Server communication
w =1 (18) protocolaimsto(1)discovertheexistenceofOSCAR-enabled
j
j=1 servers,(2)enabletheservertoknowthelistof(IP,port)pairs
X
that they can use to send their data to the clients, (3) and
VariableRanges:Thetrivialconstraintsfortherangeofthe exchange with the clients the packet scheduling weights. We
decision variables give the details of both protocols in the next two subsections.
1≥w ≥0,1≤j ≤m (19) B. OSCAR Collaboration Protocol
j
Figure 3 shows the state diagram of this protocol. The
xnj ∈{0,1},1≤j ≤m (20) communicating entities in this protocol are client devices
offering or requesting bandwidth (separated by vertical line)
5) Solution: Ingeneral,thisproblemisamixed0-1Integer
and the trusted bank server supporting our incentive system.
Programming problem, which is an NP-complete problem.
The figure also shows the states generally broken down to
However, it has a special structure that allows for an effi-
the followingthree phases (separatedby the horizontallines):
cient solution. In particular, we have two cases: if the new
discovery and authentication, tendering, and payment.
stream that triggeredthe schedulingdecisions is packet-based
(S =0) and if it is connection-based (S =1). C. Discovery and Authentication
n n
Solution for the packet-based case: In order to maintain energy-efficient neighbor discovery, a
In this case, xnj = 0∀j. The problem becomes a stan- requestingnode sends neighbordiscoveryrequests only when
dard linear programming problem. Hence, it can be solved it has active packet-oriented connections or new connection
efficiently to determine the different values of wj. requests. For this discovery, we adopt the eDiscovery [30]
Solution for the connection-based case: protocol and apply it to the available interfaces to determine
In this case, we need to determine the binary variables the best discovery duty cycle for each interface.
∀ x such that only one of them equals 1 and others are Once eDiscovery fires a neighbor discovery event, our
j nj
0. Our algorithm sets each one of them to 1 and then solves OSCARprotocolengagesandbroadcastsaDiscoveryRequest
the resulting linear programming problem to find ∀ w . The (DReq) packet.This discoverypackethas the header depicted
j j
value that achieve the best objective is then selected as the in Figure 4(a). It contains the OSCAR version running at
optimal decision. the requesting client, header length, the discovery request