DTIC ADA453374: ControlWare: A Middleware Architecture for Feedback Control of Software Performance

11 Pages·0.31 MB·English

by  Defense Technical Information Center

ControlWare: A Middleware Architecture for Feedback Control of Software
Performance
(cid:3)
RonghuaZhang,ChenyangLu,Tarek F.Abdelzaher,JohnA.Stankovic
DepartmentofComputerScience
UniversityofVirginia
Charlottesville,VA22903
e-mail: rz5b,chenyang,zaher,[email protected]
Abstract bation,theperformancevariableofchoicewillconvergeto
its desired value within a specified bounded time and that
Attainment of software performance assurances in open, its deviation from that value is always bounded. The suc-
largelyunpredictableenvironmentshasrecentlybecomean cessofcontroltheoryis,inlargepart,duetoitsrobustness
importantfocus for real-time research. Unlike closed em- in the face of modeling errors and external disturbances,
bedded systems, many contemporary distributed real-time which reduces the need for accurate system models — a
applications operate in environments where offered load muchwelcomepropertywhenaccuratemodelsaredifficult
and available resources suffer considerable random fluc- toconstruct. Recentresultshaveshownthatcontroltheory
tuations, thereby complicating the performance assurance can also be successfully appliedto the controlof software
problem. Feedbackcontroltheoryhasrecentlybeenidenti- performance[19].
fiedasapromisinganalyticfoundationforcontrollingper-
formance of such unpredictable, poorly modeled software The authorshave appliedcontroltheorysuccessfullyin
systems, the same way other engineering disciplines have severalexamplecasestudiesinvolvingcomputingapplica-
usedthistheoryforphysicalprocesscontrol. tions. These case studies includeperformanceisolation in
In this paper, we describe the design and implementa- webservers[5],webserverdelaycontrol[18],proxycache
tion of ControlWare, a middleware QoS-control architec- relative hit ratio control [21], network-layer active queue
ture based on control theory, motivated by the needs of managementfordelayandlossdifferentiation[10],andmi-
performance-assuredInternetservices. Itoffersanewtype croprocessor thermal management [29]. In this paper, we
of guarantees we call convergenceguarantees that lie be- leverage the underlying insights to develop a middleware
tweenhardandprobabilisticguarantees.Theefficacyofthe layer for QoS control that provides control-theoretic per-
architectureinachievingitsQoSgoalsunderrealisticload formanceguaranteesunderuncertainty. Themiddlewareis
conditionsisdemonstratedinthecontextofwebserverand specificallytargetedforInternetservices. Itallowstheuser
proxyQoSmanagement. toexpressQoSspecificationsoff-line,mapsthesespecifica-
tionsintoappropriatefeedbackcontrolloopsets,tunesloop
controllersanalyticallytoguaranteeconvergencetospecifi-
cations,andconnectsloopstotherightperformancesensors
1.Introduction
andactuatorsintheapplicationsuchthatthedesiredQoSis
achieved.Onemainnoveltyofthemiddlewareliesinisolat-
Toachievepredictablebehaviorindistributed,poorlymod- ingtheapplicationprogrammerfromcontrol-theoreticcon-
eled,uncertainenvironmentsoftoday’sopenperformance- cernswhileutilizingthistheorytoachievethedesiredQoS
assured applications, traditional approaches, such as re- guarantees.
source reservation [23] and a priori knowledge of worst
case execution conditions [27], are no longer applicable. Therestofthepaperisorganizedasfollows. Section2
Several recent research efforts have suggested the use of introduces the control theoretical approach in more detail
control theory [16, 30, 21, 18]. This theory offers a new andexplainshowQoSspecificationsaremappedintocon-
typeofguaranteethatliesbetweenhardandaverage(e.g., trolloops. Section3presentsthemiddlewarearchitecture.
probabilistic),whichwecallconvergenceguarantees[19]. Section 4 describes the resource management component.
Abasicconvergenceguaranteestatesthat,uponanypertur- An evaluation of our QoS control functionality using the
implemented middleware prototype in a web application
TheworkreportedinthispaperwassupportedinpartbytheNational
(cid:3) scenario is presented in Section 5. Section 6 presents re-
ScienceFoundation undergrants CCR-0093144 andCCR-0098269, and
DARPAgrantsnumberF33615-01-C-1905andN00014-01-1-0576. latedwork.ThepaperconcludeswithSection7.
Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS’02)
1063-6927/02 $17.00 © 2002 IEEE
Report Documentation Page Form Approved
OMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington
VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it

does not display a currently valid OMB control number.
1. REPORT DATE 3. DATES COVERED
2002 2. REPORT TYPE 00-00-2002 to 00-00-2002
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
ControlWare: A Middleware Architecture for Feedback Control of
5b. GRANT NUMBER
Software Performance
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
University of Virginia,Department of Computer Science,151 Engineer’s REPORT NUMBER
Way,Cahrlottesville,VA,22094-4740
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT
NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT
Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES
14. ABSTRACT
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF
ABSTRACT OF PAGES RESPONSIBLE PERSON

a. REPORT b. ABSTRACT c. THIS PAGE 10
unclassified unclassified unclassified
Standard Form 298 (Rev. 8-98)
Prescribed by ANSI Std Z39-18
2.Middleware-Based QoSControl
Control
QoS
Configuration
Contract
File
The application of feedback control theory to open dis-
tributed QoS-sensitive applications, such as mail servers,
webserversandproxycaches,encountersthreemainchal- Control Loop Controller
QoS Mapper Sys ID
lenges. First, from a control-theory perspective, there Composition Design
shouldexistageneralwaytoconvertQoSspecificationsofa
computingsystemsuchasanInternetserver,intofeedback
loops with known set points and feedback control param-
eters. This is achieved via multiple software tools and li- Controller Monitor GRM
Software QoS
brariesthatwedescribeinthissection. Second,fromasys-
Control Loops
temsperspective,aconvenientinterfaceneedsto befound ControlWare Library
between the service software and the middleware control
loopsthatmanageits performance. In oursystem, thisin- Figure2.DevelopmentMethodology
terfaceisimplementedbyanentitycalledaSoftBus,which
isdescribedin Section 3. Third,appropriatesoftwareper-
Controlloopcomposition:Theloopcomposerconfig-
formancesensorsandactuatorsmustbedesigned.Weelab-
(cid:15) uresQoSmonitors(alsocalledsensors),actuators,and
orateonthischallengeinSection4.Figure1showsanover-
controllersinthemannerdescribedbythetopologyde-
allpictureofthemiddlewarecomponents.
scriptionlanguage. Thesecomponentscancomefrom
Software Sensor/Actuator Inserted the library of ControlWare, and can also be supplied
Component API Call byusers.
Application Application
Standard
System identification: To design a controller that
Software
Sensor/Actuator (cid:15) can achieve the desired QoS guarantees, the mathe-
Interface
Socket maticmodelofthesystemmustbeknownbeforehand.
SoftBus Received info from performance sensors ControlWare provides a system identification service
Middleware for Performance Control AdaApcttoura) tcoor m(QmoaSnds thatautomaticallyderivesdifferenceequationmodels
based on system performance traces [7]. Our prior
Distributed
Machine Machine Machine Machine Platform publications [13,18,1]havealreadyshownthefeasi-
bilityandsuccessofsystemidentificationofsoftware
systems.
Figure1.OverallArchitecture
Controller configuration and tuning: Based on the
(cid:15) modelderivedbysystemidentification,ControlWare’s
2.1.ServiceDevelopmentwithControlWare controller design service can automatically tune the
controllerstoguaranteestabilityanddesiredtransient
To illustrate the main features of ControlWare, we first response to load variations [19]. The resultant con-
overview the development methodology of ControlWare- trollerparametersarewrittenintoaconfigurationfile.
based performance-assured software. An application de-
signer using our middleware for QoS provisioning would Theaboveprocessissomewhatsimilartothewaycon-
typicallyundergotheprocessshowninFigure2: trolengineersconfigureadistributedphysicalprocesscon-
trolsystem.Whatisnewhereisthatthecontrolledsystemis
QoS specification: The required QoS guarantees
asoftwareservice,andthecontrolgoalistoprovideconver-
(cid:15) shouldbespecified forthe system. ControlWarepro-
genceguaranteesonQoS.WithControlWare,softwareen-
videsasimpleContractDescriptionLanguage(CDL),
gineerscaneasilyaddperformanceassurancestotheirsys-
which is used to describe the desired QoS guarantee.
temswithouttheneedforacontrol-engineer’sbackground.
ApartialsyntaxofCDLispresentedinAppendixA.
Our middleware automates that part of the feedback loop
QoS to control-loop mapping: A tool called the QoS configurationprocess.
(cid:15) mapper interprets the CDL description offline and
mapstherequiredQoSguaranteestoasetoffeedback 2.2.QoSMapping
control loops and their set points. The QoS mapper
specifies the feedback control loops using a topology The cornerstone of a control theoretic paradigm for QoS
description language and stores it in a configuration guaranteesinsoftwaresystemsliesinourabilitytoconvert
file. common resource management and software performance
Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS’02)
1063-6927/02 $17.00 © 2002 IEEE
assurance problems into feedback control problems. Our Approximate
middleware contains a library of templates written in our System Model
topologydescriptionlanguage,eachformulatingaparticu- PerEforrrmorance RAellsoocuartcioen
Correction
lar typeofQoSguaranteesasa feedbackcontrolproblem. PAebrsfoolrumteance Actual
Set Point Actuator Software Performance
Thelibraryisextendibleinthatacontrolengineercantrans- Controller (Resource Allocator) System
forma newguaranteetype into a macrothat describesthe
correspondingloopinterconnectiontopologyandstorethat Performance
Measured Sensor
macrointhemiddleware’slibrary. Currentlythelibraryin- Performance
cludes template for absolute convergenceguarantees, per-
Figure4.AbsoluteGuarantees
formance isolation, statistical multiplexing, prioritization,
relativedifferentiatedserviceguarantees,and optimization
guarantees.Asanexample,wedescribetheimplementation
Theabsoluteconvergenceguaranteeloopistheelemen-
of the basic (absolute) convergenceguarantee, and its use
tarybuildingblockofourmiddlewarefromwhichallother
in formulatingrelativeguarantees, prioritization,and opti-
assurancesfollow,asdescribedbelow.
mizationasfeedbackcontrolproblems.
2.4.TheRelativeGuaranteeProblem
2.3.TheAbsoluteConvergenceGuarantee
The relative guarantee is to keep the ratio fixed between
the performance (such as delay, throughput, etc) of two
Since it is impossible to achieve absolute guarantees in a
traffic classes. In general, let be the number of classes
system where load and resources are not known a priori,
in the system, and be thenmeasured performance of
we define the absolute guarantee problem as one of con-
class . The differeHntiiation policy requires that the per-
vergenceto a specified performance. The statementof the
formainceof differentclasses be related bythe expression:
problemistoensurethataperformancemetric, ,(i)con-
vergeswithin a specified exponentiallydecayingRenvelope ,where isapro-
Hpo1rt:ioHna2li:ty:::co:nHstnan=toCrw1 e:iCgh2t:o:f::cl:aCssn.ThiskiCnidofguar-
to a fixed value, , and that (ii) the maximum de-
viation Rdesbireedboundedatalltimes, asshownin antee can be translated into controliloops, one for each
Figure3R.desired (cid:0)R class. In controlloop, thensensormeasuresthe relative
performanitche, . Thisvalue
iscomparedwRithi =theHsie=t(pHoi1n+t H2+to:::g+etHthen)performance
R Ci
error . TheresourceallocatioPnoCfjtheclassisthenaltered
Specified Maximum Deviation by ei . Itistrivialtoprovethat , for
Actual Performance, R anyfl(ienie)ar function . Hence, thePfee1d(cid:20)bia(cid:20)cnkfl(oeoip)s=ca0n op-
Rdes. Time erateindependentlyfwithoneloopperclass,whilethetotal
amount of allocated resource remains constant if the con-
Specified Decay Envelope trollerisalinearfunctionoferror.
Performance Resource
Error Allocation
Correction
Figure 3. The Absolute Guarantee Specifica- Ci Relative
Cj Actuator Software Performance
tion Controller (Resource Allocator) System
Performance
The problem requires that be both measurable and PMeerafosurmreadn Rceelative Sensor
controllable. MeasurablityrequRiresthatinthesteadystate,
Figure5.RelativeDifferentiatedService
the measured value of asymptotically approach its true
value. ControllabilityreRferstotherequirementthattheap-
plication must havesome adaption mechanism, that
affectsthevalueof . Forexample,if isCPUuAti(lRiza)tion, 2.5.ThePrioritizationProblem
canbeanadmRissioncontrolmecRhanism.
A(RTh)e absolute convergence guarantee is translated into Theprioritizationproblemis definedas onewhereall ser-
thecontrolloop shown in Figure 4. The loop samples the viceclientsarepartitionedinto classes,suchthatforevery
measured performance, compares it to the desired value class ,itisdesiredthatclientsonfthatclassdonotsufferany
, and uses the difference to induce changes in re- conteintionoversomesharedresource fromanyclientsof
Rsoduersciereadllocationviatheactuator . classes . r
A(R) i+1;:::;n
Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS’02)
1063-6927/02 $17.00 © 2002 IEEE
We implementthese semantics by a compositionof in- themaximumnetprofit,i.e.,maximize . Assum-
stances of the elementary block described in Section 2.3. inga concavecostfunction, , thekpwro(cid:0)fitgi(swm)aximized
First, we make the entire server capacity available to the when the marginalutility is egq(uwa)l to the marginalcost, or
highestpriorityclassusingthebasicconvergenceguarantee when . Theequationcanbesolvedfor which
dg(w)
loop of Figure 4 with a set point equal to total server ca- thenbecdowme=sthkecontrolsetpoint, . Inacompuwtingex-
pacity. Ifthatsetpointisnotreached(becausethereisnot ample, ,maybethedesiredserverRutilization,thedesired
enoughdemand),theunusedcapacityofeachclassismea- workloawd size, or other metrics depending in the problem
suredandtreatedasthesetpointfortheresourceallocation formulation.TheapproachisillustratedinFigure7.
tothelowerpriorityclass. Thisrequiresasensorarraysuch
that measuresthefractionofresource consumedby PerEforrrmorance RAellsoocuartcioen
Correction
clienSts(Rofi)class ,aswellasanactuatorarrayrwhere
Derived Actual
controls the resiource allocation of class . The arraAys(Rarie) MMaarrggiinnaall BCeonste f=it Set Point Controller (ResoAurcctue aAtollrocator) SSoyftswteamre Performance
implementedbyasetofperclassperformiancecountersand
admissioncontrollimits.
Performance
The feedback loop architecture for prioritization is de- Measured Sensor
Performance
scribedforatwoclassserverinFigure6. Onecontrolloop
isneededperclass. Applicationperformanceconvergesto Figure7.UtilityMaximization
that of a strictly prioritizedsystem. The approachmay be
usedtoimplementlogicalprioritiesinmiddlewarewhenthe As shown above, ControlWare can express the
controlledserveritselfdoesnotsupportprioritiesbydesign, most common guarantee types required in performance-
suchastheApache[12]webserver. Anexampleandeval- assurance software by casting them appropriatelyas feed-
uationofthisuseispresentedin[3]. backcontrolproblems.Oncethecontrolloopsaredesigned
from the QoS specification, the middleware uses textbook
techniques [28] to estimate system models and determine
Approximate
System Model appropriate feedback controller parameters for guaranteed
UCanpuasceidty AFidrsmt iCttleadss Clients convergence of the control loops to the specified perfor-
Correction
FAilrlsotc caltaiossn Actuator Software CRCoelasnsossuurmception mance.
Controller (Resource Allocator) System
3.SoftBus: The Distributed Interface
Performance
Measured Sensor
Consumption of First Class clients
To promote interoperability, the control engineering com-
Leftover
Capacity munity standardized open layered interface architectures
Unused Admitted
Capacity Second Class Clients
Correction such as the Fieldbus [9] greatly simplifying the intercon-
Class
Actuator Software CRoesnosuurmception nection of sensors, actuators, and controllers in a digital
Controller (Resource Allocator) System
controlsystem. Similarly,ControlWareimplementsaSoft-
Buswhosemainpurposeistoprovideacommoninterface
Performance
Measured Sensor forefficientinformationexchangebetweensoftwareperfor-
Consumption of Second Class clients
mance sensors, actuators and controllers across machines
Leftover
Capacity
andaddressspaces. Thesensors, actuatorsand controllers
need not know each other’s locations and need not worry
Figure6.Prioritization
aboutdistributedcommunication.Underneaththecommon
API,differentinformationexchangemechanismsaredevel-
opedfordifferentsituations.
2.6.TheUtility OptimizationProblem
3.1.InterfaceModules
Anothertypeofperformanceproblemsaddressableusinga
control-theoreticframework is that of utility optimization. We support two types of software sensors and actuators:
Following a microeconomic model [22], consider a com- passive and active. A passive sensor or actuator is just a
puting service which producesan amountof work . Let functioncallthatreturnssampledataoracceptsacommand
thebenefitperunitofworkbe . Hence,thetotalutwility when called by the controller. An active sensor or actua-
producedbytheserviceis k . LettheresourceconU- tor,incontrast,isaprocessorthreadwhichmayberunning
sumptionoftheservicebeUsom=eknwonlinearfunction, , in its own address space. It is usually awakened periodi-
whichrepresentsameasureofcost. Itisdesiredtoacgh(iwev)e callybytheoperatingsystemschedulertoperformsensing
Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS’02)
1063-6927/02 $17.00 © 2002 IEEE
oractuation.Forexample,anidleCPU-timesensormaybe 3.4.DataAgent
implementedasanactivesensorprocesswhichrunsatthe
lowestpriorityand computesthe percentageof time it has The data agent abstracts away remote communication be-
beenexecutingtoinferprocessorutilization. tweensensors,actuators,andcontrollers.Whenaninterface
Correspondingly,twointerfacemodulesareprovidedto moduleofsomecomponenthasdatatosendtoanother,the
facilitatethecommunicationbetweensensors/actuatorsand data agent first queries the registrar for information about
SoftBus. Each component is attached to the SoftBus via thetargetcomponent. Ifitisaremoteone,thelocalagent
anappropriateinterfacemodule.Internally,communication forwardstherequesttothedataagentonthedestinationma-
with local passive components is implemented as a direct chine. Ifthedestinationislocal,dataispassedtothatcom-
function call, while communicationwith local active ones ponent’sinterfacemoduleviasharedmemory.
isthroughsharedmemory. Section3.4describesthetrans- Figure 8 depicts the above components and their inter-
parentdistributedcommuncationbetweencomponents. actions. Thearchitectureallowseasyandflexibleconfigu-
ration of controlloopsin which thecontrolledsystem is a
softwareprocess.
3.2.Registrar
Machine A Machine B
Sensor Actuator Controller
Application
Configurability is achieved through the registration and
Middleware
dAnaePlrlIeyg,iistshtereaxtrpieoognritseotdrfabrcymonaatnirnotelanilntoistoypa, cccaaoclmlheepd.ontFheoenrtrsee.agcihsRtrleoagrci.asltrIcanottimeorn-- MSehmaroerdy ADgaetnat Registrar ServerDirectory Registrar ADgaetnat MSehmaroerdy
ponent, it recordsin the cache the component’stype(sen-
Figure8.TheSoftwareBus
sor/actuatoror controller), a callback functionpointer if it
is passive, or a shared memory address if it is active. For
remote components, it will record their location. Origi-
4.Resource Management
nally, onlylocalcomponentshaveentriesin theregistrar’s
cache. Whensomecomponent’sinformationisneededbut
An important challenge in applying a control-theoretic
cannotbefoundinthecache,theregistrarcontactsanex-
paradigmto softwareQoScontrolliesin findingappropri-
ternaldirectoryserverandcachesthereceivedinformation.
ate sensors and actuators for software services. Sensors
Whencachinginformationonremotecomponents,thereg-
typically amount to a modest instrumentation of applica-
istrar also creates a daemon to wait for invalidation mes-
tioncode.Forexample,asensormeasuringtherequestrate
sagesfromthedirectoryserver.Whenitreceivesamessage
onaparticularsitecanbeimplementedasasimplecounter
notifyingitofthederegistrationofsomecomponentsfrom
thatisresetperiodically. Asensormeasuringdelaycanbe
thedirectoryserver,theregistrarwillpurgethecorrespond-
implementedasamovingaverageofthedifferencebetween
ingentriesfromthecacheaccordingly.
twotimestamps.Oftenthemeasuredmetricisalreadyavail-
ableasavariablemaintainedbythecontrolledsoftwareser-
3.3.DirectoryServer vice(e.g.,somequeuelength)ortheoperatingsystem(such
asCPUutilization).Alloneneedstodotoimplementasen-
sorandpassthevaluetothemiddleware.
The directory server maintains the location and properties Themainapplicationinterfacechallenge,therefore,lies
of all control loop components. To maintain cache con- at the application/actuator boundary. To meet this chal-
sistency, the directory server keeps track of all machines lenge,ourmiddlewareincludesagenericresourcemanager
thatcacheits informationandnotifiesthemwhendatahas that serves as a multipurpose actuator. The manager ex-
changed.Whenallthecomponentsareononemachine,the portsauniformAPI to theapplicationandhas aback-end
directoryserveris nolongerneeded. Inthiscase, SoftBus interfacetothemachine’snativeresourceallocationmech-
optimizesitselfautomaticallybyshuttingdowntheunnec- anisms. Inthissection,wepresentthedesignofourmulti-
essarydaemons,andinhibitingcommunicationbetweenthe purposeactuatorandtheinterfacebetweenitandtheappli-
registrarsandthedirectoryserver.Inthepresentimplemen- cation.Notethat,custom-madeactuatorsthatarenotbased
tation,thenumberandidentitiesofthemachineswhichrun on our generic resource manager can still be interfaced to
SoftBusisstoredinastaticconfigurationfile. Itisreason- SoftBusasdescribedinSection3,sincethelatterisoblivi-
ablystraightforwardtoextendthisarchitecturetoallownew oustothetypeofactuatorused.
machinesto subscribeto the SoftBus dynamically using a Ourgenericresourcemanager(GRM)isdesignedforuse
groupmembershipservicesuchas[25,6,2]. withInternetserverssuchaswebservers,DNSservers,mail
Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS’02)
1063-6927/02 $17.00 © 2002 IEEE
servers,andproxycacheservers. Itunderstandsthenotion requests in all the queues. This policy influences the
of traffic classes, and exports the abstraction of resource orderoftherequestsinthelist. Systemdefaultpolicy
quotatorepresenttheamountoflogicalresourcesallocated isFIFO.
toaparticularclass. Theactionofthemanagerliesincon-
4. DequeuePolicy: Thispolicyinfluencesthedequeuing
trollingresourcequotaallocations.
of the request. Currently three choices are available:
Thestructureofthegenericresourcemanagerisshown
FIFO,priorityandproportional.FIFOmeansdequeue
inFigure9. Inthefigure,theClassifierandResourceAllo-
the request according to its position in the ordered
catorareprovidedbytheapplication.TheResourceAlloca-
list.Prioritymeansalwaysprocessingthehigh-priority
tordoesresourceallocation.TheQueueManagermaintains
queuebeforeprocessingthelow-priorityqueue.Ifpro-
onequeueforeachclass,governedbyacertainqueuingpol-
portionalpolicyischosen,theapplicationcanspecify
icy.TheQuotaManagermaintainsaresourcequotaforeach
thedequeueratioamongtheclasses. Forexample,by
class.
setting theratio to be , the queuefor theclass
willbedequeuedtwice2as: f1astasthequeueforclass 0.
1
c
Resource la Queue Resource 4.2.InteractionBetweenGRMandApplication
s
Request sifie Manager Allocator
r
Figure10summarizestheinteractionbetweenGRMandthe
application.Whensomeresourceisrequestedbytheappli-
Queue Quota
Policy Manager cation, the request is first classified by the Classifier. Af-
terthat, therequestispassed toGRMbycallinginsertRe-
Actuator quest. GRM controls resource allocation by checking the
request against two constraints: (i) the queue length, and
(ii)thequotaconstraint. Ifthequeueforthegivenclassis
Figure9.StructureofGRM
empty and the class has quota, the request is satisfied im-
mediatelyviathefunctioncallallocProctotheresourceal-
locator,andthequotaisupdatedaccordingly.Iftherequest
4.1.CustomizingtheGRM can’tbesatisfied immediately,itwillbebufferedintheits
queue. When some resourcebecomesavailable, the appli-
Tomakethismanagergeneralandflexible,wetrytoexpose cation calls resourceAvailable to notify GRM, which will
asmanytunable‘knobs’aspossiblesothattheapplication trytosatisfyasmanypendingrequestsaspossible.
can controlthebehaviorof themanageras needed. These It’s important to mention that quota is a purely logical
knobsareexposedtotheoutsideworldaspolicies.Someof concept. Unlike the traditional resource reservation sys-
thepoliciesare: tem, in our middleware the mapping of quota to physical
resourceconsumptionneednotbeknown.
1. SpacePolicy:Thispolicycontrolsthetotalspaceused Ineffect,theGRMisalogicalqueuing,admissioncon-
by the managed resource queues and the space allo- trol, and resource allocation policy interface with a back-
cation among the queues. The total space can be un- end that is capable of executing a primitive service func-
limited (limited only by available memory) or lim- tion such as assigning a request to a service process. The
ited. Theapplicationcanalsosetalimitonsome(all) GRMgeneralizestheexpressionofvariousresourcealloca-
queuesandlettheremainingqueuessharetheremain- tionpoliciesinacommonframeworkandmakesitpossible
ingspace. tocontrollogicalquotaallocationsinatrial-and-errorfash-
ionuntilperformanceconstrainsaremet.Thetrialanderror
2. Overflow Policy: This policy takes effect only when
isguaranteedtoconvergebecauseofthewaycontrollersare
some queues are sharing limited space and the space
designedwhichistheadvantageofusingacontrol-theoretic
gets used up. Two options are supported: reject and
approach.Mostimportantly,thephysicalmappingofquota
replace.Whenthepolicyisreject,currentrequestwill
toactualresourceconsumptionneednotbeknownforcor-
besimplyrejected. Ifthepolicyisreplace,thelastre-
rectoperation,whichseparatesthisapproachfromresource
questofthelowestpriorityqueuethatsharesthelim-
reservationsystems.
itedspacewillbeevictedfromthequeue(application
willbenotifiedviaacallbackfunction)andcurrentre-
5.Evaluation
questwilloccupyitsspace.
3. EnqueuePolicy: Apartfromthequeueforeachclass, TotestControlWare,weinstrumenttheApache[12]server
thequeuemanageralsomaintainsanorderedlistofthe and Squid [26] server to interface to the middleware. We
Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS’02)
1063-6927/02 $17.00 © 2002 IEEE
GRM Application Squid
Cache
insertReqest (.....){ icnlassesr t=R ecqlausessitf y(..(.r,eclqausess,.t.).) Web Miss Replacement dSeplatace uator
if (queue of this class is not empty){ Server Space Act
}i f} (ethlsiseabru ec{uplltlfdaoufeasrcnsrtPe ts rhqtoiisulcl o(rh.et.aa.q)s uu qesuasotgtea){ HTTHPT TRPe qRueessptonseRPreoqcueessst LCoaockhuep CacRclcaelhaascelssllao s Sc0s 1ps ance Monitor SoftBus Controller
buffer this request Resource Allocator
} Hit Ratio
} allocProc(...){
do resouce allocation
} Figure11.StructureofthemodifiedSquid
resourceAvailable (.....){
When some resource available
update quota usage resourceAvailable(...)
get requests from class The above proxy cache prototype was tested on a
that still has quota
allocProc(...) 100MbpsEthernetLANofnineLinuxPCs. Eachmachine
} hasa450MHzAMDK6-2processoranda256MBRAM.
Three client machines run Surge[8] to generate workload.
Surge is a web workload generation tool known for its
Figure10.Resourceallocationprocedure
realistic reproduction of real web traffic patterns such as
manifestationofaheavy-tailedrequestarrivalandfile-size
specifyrelativeservicedifferentiationastheQoSobjective. distributions, a Zipf requested file popularity distribution,
On the proxy cache, we require differentiationin terms of and proper temporal locality of accesses. Each client ma-
hitratioachievedtodifferentcontentclasses. Ontheserver, chine simulates 100 users. Three machines were used to
werequiredifferentiationin termsofservicedelay. While run Apache. Each client machine generates requests for
thesemanticsoftheperformancevariablebeingdifferenti- the content located at one of the Apache machines. In
ated are not interpreted by the middleware, our choice of our experiment, there are 3 content classes. We specified
variable is implicitly expressed in the choice of sensors. that the hit ratio of the three classes satisfy the condition
Hence, we instrument Squid to measure hit ratio and in- . Squidisconfiguredtouse8M
strument Apache to measure service delay. These sensors Hby0te:sHas1it:sHca2c=he.3F:ig2u:re112 shows the observedhit ratio
areinterfacedtoSoftBus. Thetwoserverschosenarequite differentiationduringtheexperiment.
differentintermsoftheresourcetypesmanagedtoachieve
1.0
performance differentiation. In Squid, we manage cache
size allocated to each content class. In Apache we man- 0.8
agethenumberof processesallocatedto serverequestsof Ratio 0.6 CCllaassss 01
each class. By applying our middleware to these servers, Hit 0.4 Class 2
we demonstrateits versatility and ability to satisfy diverse 0.2
performance guarantees. We describe our experiments in 0.0
0 500 1000 1500 2000 2500
moredetailnext. Time (second)
Figure12.HitRatioofthreeclasses
5.1.ProvidingHitRatioDifferentiationinSquid
As we can see from Figure 12, the instrumented squid
Figure 11 depicts the structure of the instrumented Squid.
serversuccessfullyprovidesthespecifiedhitratiodifferen-
Cachespaceissharedbyseveralclassesandeachclasshasa
tiation,illustratingthesuccessofthemiddlewareinprovid-
quotaofthespace. Generally,thespaceusedbysomeclass
ingperformanceguarantees.
will directly affect its hit ratio. The sensor, actuator and
controller provided by ControlWare constitute the control
5.2.ProvidingDelayDifferentiationinApache
loops. ControlWarecreatesoneforeachclass. Eachsensor
returnstherelativehitratioofclass ,i.e., .
HRi WeinterfacedtheApachewebservertoSoftBusasdepicted
S(i) i k=n
Eachactuator changesthe space allocated to itsPckla=s0sHbRyka in Figure 13. We implemented a request classifier, and a
valueproportionaltotheerror. Thesensorandactuatorare delay sensor. The generic resource manager described in
ofthe passivetype. Periodically,ControlWareinvokesthe Section 4 was used as the actuator. The GRM was inter-
controller, which reads data from the sensor via SoftBus, facedtoaresourceallocatorwhichpassedacceptedrequests
calculatestheresourcechangetobeapplied,andwritesthe (socketdescriptors)tobackgroundApacheprocesseswhen
resulttotheactuatorviaSoftBus. instructedbytheGRM.
Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS’02)
1063-6927/02 $17.00 © 2002 IEEE
GRM willreportonadetailedevaluationofothertypesofguaran-
Queue Quota Connection
Manager Manager Dispatch teesandotherInternetservicesinasubsequentpublication.
Resource
Classifier Allocator 5.3.PerformanceEvaluation
Connection Actuator Monitor
Request
TCP Listen Delay
Queue Controller SoftBus Server One concern of applying ControlWare is the overhead it
Process
introduces. We argue that the overhead is actually very
HTTP Request
small. First, when the application is not distributed, Con-
HTTP Response trolWarecanoptimizeitselfbyshuttingdownunnecessary
daemons, and stop communication between the registrars
Figure13.StructureofthemodifiedApache and directoryservers. Even in thedistributedcase, thedi-
rectoryserveronlyneedstobecontactedwhenthelocation
ofsomecomponentisunknown.Afterthat,thisinformation
Four client machines were used to run Surge and gen-
is cached locally. Since the control loop structure is very
eraterealisticserverworkload. Asbefore,eachclientsim-
stable,theoverheadofmaintainingthecacheconsistencyis
ulates100 users. We dividedthe client machinesinto two
almost neglectible. So the overhead is just the round trip
classeswithtwomachinesperclass. Inthefirsthalfofthe
time overthe networkfor fetchingdata fromremote com-
experiment, only one machine from class 0 generates re-
ponents.
quests. Thesecondoneisturnedonafter870seconds. We
To quantify the overhead, we design a small program
specifiedthatthe connectiondelay of class 0 andclass
basedonControlWare,andtestitonthetestbed. Thecon-
1 should satisfy Datiall times. Figure14
trol loop spans two machines. Sensor and actuator are lo-
showstheresultsDof0th:iDse1xp=er1im:en3t.
cated at one machine, and controller resides at the other.
Thedirectoryserverrunsonathirdmachine. Allthecom-
60
ponentsarereactive. Eachinvokationofthefeedbackcon-
Latency (second)2400 CCllaassss 01 tCfrrooelqnutcreoonslWtt,sau4rs.eu8,amtlhlsye. ainStvitnohckeeamtiinoanganoitftyupcdoiecnatolrfoalspelpoclooicpnadwti.oilnHlnebnoactseebd,etohsone
overheadofControlWarewillberelativelyevensmaller.
0
0 500 1000 1500
Time (second)
6.Related Work
Figure14.RelativeDelaybetweentwoclasses
Thecontroltheoreticalapproachhasbeensuccessfullyap-
pliedtoinseveralcomputersystemprojects.Atthenetwork
Fromthefigure,wecanseethatbefore870seconds,the layer, Hollotet. al. [16] appliedcontroltheoryto analyze
delayof class 1 is about3 times of thedelay of class 0 as the RED congestion control algorithm on IP routers. Re-
specified.Whenthesecondmachineofclass0isturnedon, centlyfeedbackcontrolisappliedtoactivequeuemanage-
the delay of class 0 is increased suddenly. The controller menttoprovidelossanddelaydifferentiation[10].
reacts by allocating more processes to class 0. At about In the area of CPU scheduling, Steere et. al. [30] de-
1000seconds,thedelayratioconvergetoaround3again. velopedafeedbackbasedCPUschedulerthatsynchronizes
Theevaluationclearlydemonstratesthatthemiddleware the progressof consumersand supplier. In [20], feedback
is capable of providing the specified performanceguaran- controlreal-timeschedulingalgorithmsweredevelopedto
teeswithonlyamodestinstrumentationcostandnocontrol- providedeadlinemissratioguaranteestoreal-timeapplica-
theoreticalexperiencerequiredfromthesoftwaredeveloper. tionswithunpredictableworkloads.
The middeware is versatile in that it is not tailored for a Recently Internet server software has become a focus
specific software service or a specific performancemetric. area of feedback control because the unpredictabilities of
Notonlydoesthemiddlewareallownewservicestobeeffi- theworkload. Examplesofsuch QoScontrolincludesde-
cientlyaugmentedwithQoSprovisioning,butalsoitmakes layandbandwidthcontrolin Web servers[4, 18], hitratio
it easy to retrofit delivery of QoS assurances into services differentiationin web caches [21] and queuemanagement
thatwerenotdesignedwiththispurposeinmind. Thispa- ine-mailservers[24].
perprovidesaproofofconceptoftheutilityofControlWare The above projects demonstrated that feedback control
as an embodimentof a generalcontrol-theoreticparadigm provides a powerful theoretical foundation to provide ro-
for QoS guaranteesin software systems. Relative guaran- bust QoS guarantees in a wide range of software systems.
teeswereusedasanexampleinourevaluation.Theauthors However,their feedbackcontrolloopsareimplementedas
Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS’02)
1063-6927/02 $17.00 © 2002 IEEE
individualcasesfromscratch.Significanteffortsareneeded toQoScontrol,oursisthefirstcomprehensivemiddelware
todevelopandtunethefeedbackcontrolloopineverycase. servicethatincorporatestheseprinciplesunderaclearwell
Nomiddlewarehasbeendevelopedintheaboveprojectsto defined set of APIs which substantially reducesthe devel-
providegeneralsupportforcompositionandtuningofsoft- opmenteffortof performanceassured applicationsand In-
warefeedbackcontrolloops. ternetservices.
The SoftBus architecture in ControlWare is related to Futureworkof theauthorswill focusonunderstanding
distributed middleware such as CORBA [15] and DCOM thelimitationsofacontrol-theoreticapproachandderiving
[11]. Similar to the location-transparent method invoca- new guarantee semantics. A possible disadvantage of us-
tion in CORBA and DCOM, SoftBus supports plug-and- ingfeedbackonlyasameanstocorrectperformanceisthe
play control loop components (i.e., monitors, controllers, need for a performance error to occur first before a feed-
andactuators).However,unlikeCORBAandDCOM,Soft- back controller can respond. In the future, we shall focus
Bus provides direct support for feedback control by sup- on mechanisms that combine prediction with feedback to
porting active and passive interaction mechanisms among improveconvergencetospecificationsinahighlydynamic
monitors, controllers, and actuators, as well as generalre- unpredictablesystem. Weshallalsoextendthemiddleware
sourcemanagersforserversystems. Furthermore,Control- to allowfully dynamiconline re-configurationduringnor-
Ware provides system identification and controller tuning mal system operation, and investigate other types of per-
services that are not supported by other common middle- formance guarantees that might be achievable in a feed-
wareservices. back control context. For example, it may be interesting
The SWiFT project [14] at OGI and the Agilos project tocastadaptiveguaranteesonserviceavailability,security,
[17] at UIUC share some similar goals with ControlWare. andfault-toleranceasfeedbackcontrolproblems.
SWiFTisa toolkitforconstructingfeedbackcontrolloops
Acknowledgments
from libraries. It also supports the visualization and sim-
ulationof softwarecontrolloops. The Agilosprojectcon- Theauthorswouldlike to thankSangSon, GangTao, and
structs a middleware to support QoS control and adapta- YingLuforprovidingusefulcomments.
tion.ControlWareisdifferentfromSWiFTandAgilosinits
uniqueSoftBusarchitecturethatenablesflexibleplug-and-
References
playcontrolcomponentsina locationindependentfashion
(e.g.,componentsofasamecontrolloopcanbefromdiffer-
[1] T.Abdelzaher. Anautomoatedprofilingsubsystemforqos-
entaddressspacesandevenremotenodes). Incomparison,
awareservices. InIEEEReal-TimeTechnologyandAppli-
Agilos has a fixed two level control structure and a fixed
cationsSymposium,Washington,D.C.,June2000.
setofmonitors(e.g.,CPUandbandwidthmonitors). More [2] T. Abdelzaher, A. Shaikh, F. Jahanian, and K. Shin. RT-
importantly, ControlWare is the first middleware that pro- CAST:Lightweightmulticastforreal-timeprocessgroups.
videsend-to-endsupportofthewholedevelopmentprocess In IEEE Real-Time Technology and Applications Sympo-
ofcreatingQoS controlin softwaresystems. Thisprocess sium,Boston,Massachusetts,June1996.
includesdefiningQoScontracts,mappingcontractstofeed- [3] T.F.Abdelzaher and N.Bhatti. Webserver QoSmanage-
ment by adaptive content delivery. In International Work-
backcontrolloops,systemidentification,controllertuning,
shoponQualityofService,London,UK,June1999.
andimplementation.Forexample,neitherSWiFTorAgilos
[4] T.F.AbdelzaherandN.T.Bhatti. Webcontentadaptation
supportsthemappingfromQoScontractstofeedbackcon-
to improve server overload behavior. WWW8 / Computer
trolloopsorsystemidentification.Theyalsodonotprovide
Networks,31(11-16):1563–1577,1999.
thegenericresourcemanagerasinControlWare. [5] T. F. Abdelzaher, K. G. Shin, and N. Bhatti. Perfor-
mance guarantees for web server end-systems: A control-
7.Conclusions theoretical approach. IEEE Transactions on Parallel and
DistributedSystems,2001. Accepted.
[6] Y.Amir,L.Moser,P.Melliar-Smith,D.Agarwal,andP.Cia-
In this paper, wedescribed anew middlewarearchitecture rfella.TheTotemsingle-ringorderingandmembershippro-
forQoSguaranteesindistributedenvironmentssuchasthe tocol. ACMTransactionsonComputerSystems,13(4):311–
Internet. Thearchitectureimplementsa newparadigmfor 342,November1995.
QoS control, which is especially suitable for systems op- [7] K.J.AstromandB.Wittenmark. AdaptiveControl, chap-
ter2. AddisonWesley,2ndedition,1995.
eratingin highlyuncertainenvironmentsor whenaccurate
[8] P.BarfordandM.Crovella. Generatingrepresentativeweb
system load and resource models are not available. Our
workloads for network and server performance evaluation.
preliminaryevaluationof the ability of this middlewareto
InMeasurementandModelingofComputerSystems,pages
provideadvertisedguaranteesinthecontextofselecteddif-
151–160,1998.
ferentapplicationsillustratesthepromiseofthisapproach. [9] A. Chatha. Fieldbus: the foundation for field control sys-
Whileprioreffortshavebeenmadetoapplycontroltheory tems. ontrolEngineering,41(6):47–50,May1994.
Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS’02)
1063-6927/02 $17.00 © 2002 IEEE