WiSeDB: A Learning-based Workload Management Advisor for Cloud Databases

0.64 MB·

by  Ryan Marcus

WiSeDB: A Learning-based Workload Management
Advisor for Cloud Databases
Ryan Marcus Olga Papaemmanouil
BrandeisUniversity BrandeisUniversity
[email protected] [email protected]
6 ABSTRACT Expanding on our vision [21], we argue that cloud-based
1 databasescouldbenefitfromaworkloadmanagementadvisorser-
Workloadmanagementforclouddatabasesdealswiththetasksof
0 vicethatremovestheburdenoftheabovechallengesfromapplica-
resourceprovisioning,queryplacement,andqueryschedulingina
2 tiondevelopers.Applicationsshouldbeabletospecifytheirwork-
mannerthatmeetstheapplication’sperformancegoalswhilemin-
loadcharacteristicsandperformanceobjectives,andsuchaservice
r imizingthecostofusingcloudresources. Existingsolutionshave
p shouldreturnasetoflow-costandperformance-efficientstrategies
approachedthesethreechallengesinisolationwhileaimingtoop-
A forexecutingtheirworkloadsonacloudinfrastructure.
timize a single performance metric. In this paper, we introduce
Wehaveidentifieda numberofdesigngoalsforsuch anadvi-
WiSeDB,alearning-basedframeworkforgeneratingholisticwork-
0 sor service. First, given an incoming query workload and a per-
loadmanagementsolutionscustomizedtoapplication-definedper-
3 formance goal, the service should provide holistic solutions for
formancegoalsandworkloadcharacteristics. Ourapproachrelies
onsupervisedlearningtotraincost-effectivedecisiontreemodels executing a given workload on a cloud database. Each solution
]
should indicate: (a) the cloud resources to be provisioned (e.g.,
B forguidingqueryplacement, scheduling, andresourceprovision-
number/typeofVMs),(b)thedistributionofresourcesamongthe
D ingdecisions.Applicationscanusethesemodelsforbothbatchand
workloadqueries(e.g.,whichVMwillexecuteagivenquery),and
onlineschedulingofincomingworkloads. Auniquefeatureofour
. (c)theexecutionorderofthesequeries.Werefertothesesolutions
s system is that it can adapt its offline model to stricter/looser per-
c formancegoalswithminimalre-training.Thisallowsustopresent collectivelyasworkloadschedules.
[ Second, to support diverse applications (e.g., scientific, finan-
totheapplicationalternativeworkloadmanagementstrategiesthat

3 addressthetypicalperformancevs.costtrade-offofcloudservices. cial, business-intelligence, etc), we need a customizable service
thatsupportsequallydiverseapplication-definedperformancecri-
v Experimentalresultsshowthatourapproachhasverylowtraining
teria. Weenvisionacustomizableservicethatgeneratesworkload
1 overheadwhileofferinglowcoststrategiesforavarietyofperfor-
schedulestunedtoperformancegoalsandworkloadcharacteristics
2 mancemetricsandworkloadcharacteristics.
specifiedbytheapplication. Supportedmetricsshouldcapturethe
2
performanceofindividualqueries(e.g., querylatency)aswellas
8 1. INTRODUCTION
theperformanceofbatchworkloads(e.g.,maxquerylatencyofan
0
Cloudcomputinghastransformedthewaydata-centricapplica- incomingqueryworkload).
.
1 tions are deployed by reducing data processing services to com- Third,sincecloudprovidersofferresourcesforsomecost(i.e.,
0 moditiesthatcanbeacquiredandpaidforon-demand.Despitethe price/hour for renting a VM), optimizing schedules for this cost
6 increasedadoptionofclouddatabases,challengesrelatedtowork- is vital for cloud-based applications. Hence, any workload man-
1 loadmanagementstillexist,includingprovisioningcloudresources agement advisor should be cost-aware. Cost functions are avail-
v: (e.g.,virtualmachines(VMs)),assigningincomingqueriestopro- ablethroughcontractsbetweentheserviceprovidersandtheircus-
i visionedVMs,andqueryschedulingwithinaVMinordertomeet tomersintheformofservicelevelagreements(SLAs). Thesecost
X performance goals. These tasks strongly depend on application- functions define the price for renting cloud resources, the perfor-
r specificworkloadcharacteristicsandperformancegoals,andthey mancegoals,andthepenaltytobepaidiftheagreed-uponperfor-
a aretypicallyaddressedbyad-hocsolutionsattheapplicationlevel. manceisnotmet.Aworkloadmanagementserviceshouldconsider
A number of efforts in cloud databases attempt to tackle these allthesecostfactorswhileassistingapplicationsinexploringper-
challenges(e.g., [8,9,16,18,19,20,27,29,30]). However, these formance/costtrade-offs. Sincedifferentworkloadschedulesoffer
techniquessufferfromtwomainlimitations.First,theydonotpro- differentperformancevs. costtrade-offsfordifferentperformance
videholisticsolutionsbutinsteadaddressonlyindividualaspects metrics,thesystemshouldbeabletodiscoverthe“best”strategy
oftheproblem,suchasqueryadmission[27,30],queryplacement forexecutingagivenworkloadunderanapplication-specificper-
to VMs [9,18,19], query scheduling within a VM [8,9], or VM formancegoal.Lowcostworkloadschedulesshouldbediscovered
provisioning[16,18,20,29]. Sincethesesolutionsaredeveloped independentlyoftheperformancemetric.
independentlyofeachother,theirintegrationintoaunifiedframe- This paper introduces WiSeDB ([W]orkload management
workrequiressubstantialeffortandinvestmentto“getitright”for [Se]rvice for cloud [DB]s), a workload management advisor
eachspecificcase. Second,whileabroadrangeoflatency-related for cloud databases designed to satisfy the above requirements.
performancemetricsarecoveredbythesesystems(e.g.,queryre- WiSeDBofferscustomizedsolutionstotheworkloadmanagement
sponse time [8,9,19,29], average query latency [19]), each of- problem by recommending cost-effective strategies for executing
fers solutions tuned only for a specific metric. Adapting them to incomingworkloadsforagivenapplication. Thesestrategiesare
supportawiderrangeofapplication-specificmetrics(e.g.,maxla- expressed as decision-tree models and WiSeDB utilizes a super-
tency,percentilemetrics)isnotastraightforwardtask.
1
visedlearningframeworkto“learn”modelscustomizedtotheap- 1. Query Templates
plication’s performance goals and workload characteristics. For n Performance Goal Model Generator
o
an incoming workload, WiSeDB can parse the model to identify ati Original Original
thenumber/typeofVMstoprovision,theassignmentofqueriesto plic Model Training Set
VMs,andtheexecutionorderwithineachVM,inordertoexecute Ap 2. Alternative Strategies & Cost Strategy Recommendation
theworkloadandmeettheperformanceobjectivewithlow-cost. nt
Eachmodeliscost-aware: itistrainedonasetofperformance me 3. Runtime Workload & Strategy
e Schedule Generator
and cost-related features collected from minimum cost schedules g
na 4. Workload Schedule
oinfgsaasmwpleellwaosraknloyapdesn.aTltiheisspcaoisdtdaucceotuonftasilfuorrerteosomuerceettphreovpiesrifoonr-- a Ma 5. Schedule Execution Virtual Machines... ProvIaidaeSr
mancegoals. Furthermore,ourproposedfeaturesareindependent at
D
from the application’s performance goal and workload specifica-
tion,whichallowsWiSeDBtolearneffectivemodelsforarangeof
metrics (e.g., average/max latency, percentile-based metrics). Fi- Figure1:TheWiSeDBsystemmodel
nally, eachmodelistrainedofflineonce, andcanbeusedatrun-
timetogenerateschedulesforanyworkloadmatchingthemodel’s
SELECT SUM (l_extendedprice * l_discount)
workloadspecifications. Givenanincomingbatchworkloadanda
FROM lineitem
decisionmodel,WiSeDBparsesthemodelandreturnsalowcost
WHERE l_shipdate >= date’[DATE]’
scheduleforexecutingtheworkloadoncloudresources.
AND l_shipdate < date’[DATE]’
WiSeDB leverages a trained decision model in two additional
+ interval’1’year
ways. First, thetrainingsetofthemodelisre-usedtogeneratea
AND l_discount between [DISC] - 0.01
setofalternativemodelsforthesameworkloadspecification,but
and [DISC] + 0.01
stricterormorerelaxedperformancecriteria. Severalmodelsare
AND l_quantity < [QUANT];
presentedtotheuser,alongwiththecostestimateforeachasfunc-
tionoftheworkloadsize. Thisallowsapplicationstoexplorethe
Figure2:TPC-HQueryTemplate#6
performancevs.costtrade-offfortheirspecificworkloads.Second,
eachmodelcanbeadjusted(withsmalloverhead)duringruntime
tosupportonlineschedulingofqueriesarrivingoneatatime. discussesrelatedworkandweconcludeinSection9.
Thecontributionsofthisworkcanbesummarizedasfollows:
1. WeintroduceWiSeDB,aworkloadmanagementadvisorfor
clouddatabases.Wediscussitsdesignandrichfunctionality, 2. SYSTEMMODEL
rangingfromrecommendationsofalternativeworkloadexe-
Our system is designed for data management applications de-
cutionstrategiesthatenableexplorationofperformancevs.
ployedonanInfrastructure-as-a-Service(IaaS)cloud(e.g.,[1,2]).
cost trade-offs to resource provisioning and query schedul-
Theseproviderstypicallyoffervirtualmachines(VMs)ofdifferent
ingforbothbatchandonlineprocessing. Allrecommenda-
types(i.e.,resourceconfigurations)foracertainfeeperrentingpe-
tionsaretunedforapplication-definedworkloadcharacteris-
riod. WeassumethisdeploymentisrealizedbyrentingVMswith
ticsand(querylatency-related)performancegoals.
preloadeddatabaseengines(e.g.,PostgreSQL[3])andthatqueries
2. Weproposeanovellearningapproachtotheworkloadman-
canbeexecutedlocallyonanyoftherentedVMs.Thispropertyis
agement problem. WiSeDB learns its custom strategies by offeredbyfullyreplicateddatabases.1
collectingfeaturesfromoptimal(minimumcost)schedules
Workload Specification Applications begin their interaction
ofsampleworkloads. Werelyonagraph-searchtechnique withWiSeDBbyprovidingaworkloadspecificationindicatingthe
to identify these schedules and generate a training set in a
querytemplates(e.g.,TPC-Htemplates[4])thatwillcomposetheir
timely fashion. Furthermore, we have identified a set of
workloads. Inanalyticalapplications,incomingqueriesaregener-
performanceandcost-relatedfeaturesthatcanbeeasilyex-
allyinstances ofasmall numberoftemplates, andqueriesof the
tractedfromtheseoptimalsolutionsandallowustolearnef-
same template have similar performance properties (i.e., latency)
fective(low-cost)strategiesforexecutinganyworkloadfor
because they access and join the same tables [25]. In practice,
variousperformancemetrics.
WiSeDB’s approach is agnostic to the tables accessed or joined
3. We propose an adaptive modeling technique that generates
andcaresonlyaboutthelatencyofeachtemplate,i.e.,querieswith
alternativeworkloadexecutionstrategies, allowingapplica-
identicallatencycanbetreatedasinstancesofthesametemplate.
tionstoexploreperformancevs.costtrade-offs.
Query Templates We assume that a user’s workload is com-
4. Weleveragethetrainedmodelstoschedulebatchworkloads
posed of queries, each of which are derived from a finite set of
aswellastosupportonlineschedulingbyefficientlygener- query templates, much like TPC-H templates [4]. A query tem-
atinglow-costmodelsuponarrivalofanewquery.
plate, generally expressed in SQL, is a query with a set of miss-
5. WediscussexperimentsthatdemonstrateWiSeDB’sability
ingvalues. Userscaninstantiateanewqueryofagiventemplate
tolearnlow-coststrategiesforanumberofperformancemet-
byprovidingthemissingvalues. Figure2showsanexampleofa
ricswithverysmalltrainingoverhead.Thesestrategiesoffer
query template from the TPC-H benchmarking suite. A new in-
effectivesolutionsforbothbatchandonlineschedulingin-
stanceofthisquerytemplatecouldbecreatedbygivingvaluesfor
dependentlyofworkloadandperformancespecifications. [DATE],[DISC],and[QUANT].Conversely,anyquerymatch-
WefirstdiscussWiSeDB’ssystemmodelinSection2andthen
ingthesemanticsofsometemplateforacertainsetofvaluescan
define our optimization problem in Section 3. We introduce our
besaidtobeaninstanceofthattemplate.
modeling framework in Section 4 and the adaptive modeling ap-
proachinSection5. Section6discussesWiSeDB’sruntimefunc-
1Partialreplication/datapartitioningmodelscanalsosupportedbyspecify-
tionality. Section 7 includes our experimental results. Section 8
ingwhichdatapartitionscanserveagivenquery.
2
PerformanceGoalsApplicationsalsospecifytheirperformance t=0 t=1m t=2m t=3m
goals for their workloads as functions of query latency. Perfor- VM1 $"" $##
mance goals can be defined either at the query template level or
workloadlevel. Currently,wesupportthefollowingfourtypesof Scenario 1 VM2 $'"
metrics. (1) Per Query Deadline: users can specify an upper la- VM3 $("
tencyboundforeachquerytemplate(i.e.,queriesofthesametem- violation period (%&&)
platehavethesamedeadline).(2)MaxLatencyDeadline:theuser VM1 $## $""
expresses an upper bound on the worst query response time in a Scenario 2
queryworkload. (3)AverageDeadline: setsanupperlimitonthe VM2 $'" $("
averagequerylatencyofaworkload.(4)PercentileDeadline:spec- violation period (%)&)
ifiesthatatleastx%oftheworkload’squeriesmustbecompleted !" deadline !# deadline
withintseconds.Thesemetricscoverarangeofperformancegoals
Figure3:TwodifferentschedulesforQ={q1,q2,q2,q2}
typicallyusedfordatabaseapplications(e.g.,[8,19,27,30]).How- 1 2 3 4
ever, WiSeDB is the first system to support them within a single
workloadmanagementframework. eachscheduleS indicates(1)thenumberandtypeofVMstobe
Performance goals are expressed as part of a Service-Level- provisioned, (2) the assignment of each query qji ∈ Q to these
Agreement (SLA) between the IaaS provider and the application VMsand(3)thequeryexecutionorderoneachVM,vmi ∈S. A
j
thatstates(a)theworkloadspecification,(b)theperformancegoal, completescheduleassignseachqueryinQtooneVM.
and(c)apenaltyfunctionthatdefinesthepenaltytobepaidtothe Wedenotethelatencyofaqueryqjx(oftemplateTx)whenexe-
applicationifthatgoalisnotmet.Oursystemisagnostictothede- cutedonaVMoftypei,vmi,asl(qx,i). Latencyestimatescan
k j
tailsofthepenaltyfunction,incorporatingitintoitscost-modelas beprovidedbyeithertheapplication(e.g.,byexecutingrepresenta-
a“blackbox”functionthatmapsperformancetoapenaltyamount. tivequeriesa-prioriontheavailableVMtypes)orbyusingexisting
The architecture of the WiSeDB Advisor is shown in Fig- predictionmodels(e.g.,[10,11]).QueriesassignedtoaVMcanbe
ure1. Workloadandperformancespecificationsaresubmittedto executedimmediatelyorcanbeplacedinaVM’sprocessingqueue
WiSeDB, which trains a decision model, a.k.a. strategy (Model ifnomoreconcurrentqueriescanbeprocessed.2
Generator).Thetrainingsetforthismodelisalsoleveragedtogen- Figure 3 shows two possible schedules for a workload of four
eratealternativedecisionmodels(strategies)forstricterandmore queriesdrawnoutoftwotemplates. Thefirstscenariousesthree
relaxed performance goals (Strategy Recommendation). These VMs while the second executes the queries on two VMs. Based
strategiesarepresentedtotheuseralongwithacostfunctionthat onournotation,thefirstscheduleisexpressedasS = {vm =
1 1
estimatesthemonetarycostofeachstrategybasedonthefrequency [q2,q1],vm = [q2],vm = [q2]},whereasthesecondscenario
2 1 2 3 3 4
ofeachquerytemplateinagivenworkload. isexpressedasS = {vm = [q1,q2],vm = [q2,q2]}. Here,
2 1 1 2 2 3 4
Given an incoming workload at runtime, the application esti- p(R,S ) (cid:54)= 0, since the violation period for the schedule repre-
2
matestheexpectedcostandperformanceofexecutingthesework- sentedinthesecondscenarioisnotzero.
loadsusingourproposedstrategiesandchoosestheonethatbet- Cost Model To calculate the monetary cost of processing a
terbalancesperformanceneedsandbudgetconstraints(Execution workload,weassumeeachVMoftypeihasafixedstart-upcost
Strategy). WiSeDB identifies (a) the type/number of VMs to be fi,aswellasarunningcostfi perunitoftime(i.e. thepricefor
s r
provisioned,(b)theassignmentofqueriestotheseVMsand(c)the rentingtheVMforthattimeunit).Wealsoassumeapenaltyfunc-
executionorderofthequerieswithineachVM,inordertotoex- tionp(R,S)thatestimatesthepenaltyforagivenscheduleS and
ecute the incoming workload based on the chosen strategy. This performancegoalR. Withoutlossofgenerality(andsimilarlyto
step is executed by the Schedule Generator which can generate themodelusedbyIaaSproviders[1]),weassumepenaltiesarecal-
theseworkloadschedulesforbothbatchworkloadsaswellassin- culatedbasedontheviolationperiod,i.e.,afixedamountpertime
glequeriesastheyarrive(onlinescheduling).Applicationsexecute periodofviolationwithinascheduleS.
their workloads according to WiSeDB’s recommendations. They Theviolationperiodisthedurationoftimethattheperformance
rent VMs as needed and add queries to the processing queue of goalRwasnotmet. Ifthegoalistocompletequeriesofagiven
VMsaccordingtotheproposedschedule. VMsarereleasedupon template within a certain deadline (per query deadline goal), the
completionoftheworkload’sexecution. Inthecaseofonlinepro- violationperiodforeachqueryismeasuredfromthetimepointit
cessing,theapplicationaddsthenewquerytotheprocessingqueue misseditsdeadlineuntilitscompletion.Figure3showsanexample
ofanexistingVMortoanewVM. ofthiscase. LetusassumethatqueriesoftemplateT haveanex-
1
ecutiontimeof2minutesandqueriesofT executewith1minute.
2
3. PROBLEMDEFINITION Thefigureshowstheviolationperiodsforthesecondscenario(for
q2,q2)assumingthatthedeadlineoftemplateT is3minuteand
Here, we formally define our system’s optimization goal and 2 4 1
forT is1minutes(thefirstscenariodoesnothaveanyviolations).
discuss the problem’s complexity. Applications provide a set of 2
For the maximum latency metric, where no query can exceed
query templates T = {T ,T ,...} as the workload specifica-
1 2
themaximumlatency,theviolationperiodiscomputedinthesame
tionandaperformancegoalR. Giventhesetofquerytemplates
way. Foranaveragelatencyperformancegoal,thedurationofthe
T, WiSeDB generates decision models for scheduling workloads
violation period is the difference between the desired average la-
withqueriesinstancesdrawnfromT. Letusassumeaworkload
Q={qx,qy,...}whereeachqueryqj ∈Qisaninstanceofthe tencyandtheactualaveragelatencyofeachquery.Forapercentile
1 2 i performance goal that requires x% of queries to be complete in
templateT ∈ T. GivenaworkloadQandoneofthegenerated
j
y minutes, the violation period is the amount of time in which
decisionmodels,WiSeDBidentifiesascheduleSforexecutingQ.
We represent a VM of type i, vmi, as a queue vmi = 100−x%ofthequerieshadlatenciesexceedingy minutes. We
[qx,qy,...] of queries to process in that order and we represent
1 2
ascheduleS = {vmi1,vmj2,...}oftheworkloadQasalistof 2Most DBMSs put an upper limit on the number of concurrent queries,
VMs such that each VM contains only queries from Q. Hence, referredtoasmultiprogramminglevel.
3
Symbol Meaning Feature Generation Model Generation
R performancegoal (per sample workload) Training Data
S workloadschedule Graph Collection
Performance Construction
fTqsjiii qjst-uatehrrt-yquutpeemrcyops(ltoaoftefteiVmMplatytepei)i QuTe =r yG{ ToT1ea,m lT 2pR,l…at}es GWeSnoaermkralpotlaieodn OptGimenael rSacthioOendputilme al LeMaorndienlg
fi costofrentingVMtypeiperunittime Schedules Workload
r Feature Management
l(qj,i) latencyofquerytemplatejonVMtypei Extraction Model
p(R,S) penaltyofSunderR
Figure4:Generationofthedecisionmodel
cost(R,S) totalcostofSunderR
vmi jthVM(oftypei) findthescheduleS = {q3,q3,q2],[q2,q1],[q1]}. However,a
j FFI 5 6 3 4 1 2
vmi VMoftypei betterstrategyisonethatattemptstoplaceaninstanceofT1,then
vs partialscheduleatvertexv aninstanceofT2,thenaninstanceofT3,thencreateanewVM,re-
v unassignedqueriesatvertexv sultinginthescheduleS(cid:48) = {[q1,q2,q3],[q1,q2,q3]},whichhas
u 1 3 5 2 4 6
alowercostbecauseitprovisionsonelessVM.
Table1:Notationtable
WiSeDBdepartsfromthe“one-strategy-fits-all”approachused
by standard approximation heuristics. Instead, it offers effective
denotethemonetarycostofperformanceviolationsforagivenper- schedulingstrategiesforcustomperformancegoalsandworkload
formancegoalRandascheduleSasp(R,S). specifications by learning heuristics tailored to the application’s
Problem Definition Given a workload Q = {q1x,q2y,...}, needs.WenextdescribehowWiSeDBidentifiessuchstrategies.
where qj is of template T , and a performance goal R, our goal
i j
istofindacompletescheduleSthatminimizesthetotalmonetary 4. DECISIONMODELGENERATION
cost(provisioning,processing,andpenaltypayoutscosts)ofexe-
WiSeDBreliesonsupervisedlearningtoaddressourworkload
cutingtheworkload.Wedefinethistotalcost,cost(R,S),as:
managementproblem.Next,wedescribethisprocessindetail.
 
4.1 ApproachOverview
cost(R,S)= (cid:88) fi+ (cid:88) fi×l(qm,i)+p(R,S)
 s r k  Givenanapplication-definedworkloadspecification(i.e.,query
vmi∈S qm∈vmi templates and a performance goal), WiSeDB generates a set of
j k j
(1) workloadexecutionstrategiesthatcanbeusedtoexecuteincoming
Problem Complexity Under certain conditions, our optimization queryworkloadswithlowcostandwithintheapplication’sperfor-
problembecomesthebinpackingproblem,wherewetryto“pack” mancegoal. Formally,ourgoalistoidentifystrategiesthatmini-
each query into one of the available “VM-bins”. For this reduc- mizethetotalcostasdefinedinEquation1. Towardsthisend,our
tion, we need to assume that (1) the number of query templates frameworkgeneratessamplesofoptimalschedules(i.e.,thatmin-
is unbounded, (2) infinite penalty, p(R,S) = ∞, if the perfor- imizethetotalcost)andreliesondecisiontreeclassifierstolearn
mancegoalisviolated,and(3)thestart-upcostfiisuniformacross “good”strategiesfromtheseoptimalsolutions.
s
allVMtypes. Undertheseconditions,theproblemisNP-Hard. Our framework is depicted in Figure 4. Initially, we create a
However, these assumptions are not valid in our system. Limit- large number of random sample workloads, each consisting of a
ing the number of query templates relaxes the problem to one of smallnumberofqueriesdrawnfromthequerytemplatedefinitions.
polynomialcomplexity,butstillnotcomputationallyfeasible[28]. Ournextstepistoidentifytheoptimalscheduleforeachofthese
Twocommongreedyapproximationstothisoptimizationprob- sampleworkloads.Todothisefficiently,werepresenttheproblem
lem are the first-fit decreasing (FFD) [22] and first-fit-increasing ofschedulingworkloadsasagraphnavigationproblem. Onthis
(FFI)algorithms,whichsortqueriesindecreasingorincreasingor- graph,edgesrepresentqueryassignmentorresourceprovisioning
der of latency respectively and place each query on the first VM decisionsandtheweightofeachedgeisequaltothecostofthat
where the query “fits” (incurs no penalty). If the query will not decision. Hence,eachpaththroughthegraphrepresentsdecisions
fitonanyVM,anewVMiscreated. Existingcloudmanagement thatcomposesomescheduleforthegivenworkload. Findingthe
systems have used FFD (e.g., [7]) for provisioning resources and optimal schedule for thatworkload isthus reduced to finding the
schedulingqueries. However, itisoftennotclearwhichofthese shortestpathonthisgraph.
greedyapproachesisthebestforaspecificworkloadandperfor- Next, foreachdecisionwithinanoptimalschedule, weextract
mancegoal. Forexample,whenappliedtotheworkloadandper- asetoffeaturesthatcharacterizethisdecision. Wethengenerate
formance goals shown in Figure 3, FFD schedules all queries on atrainingsetwhichincludesallcollectedfeaturesfromalltheop-
their own VM, which offers the same performance as scenario 1 timal schedules across all sample workloads. Finally, we train a
but uses an additional VM (and hence has higher cost). A bet- decisiontreemodelonthistrainingset.Thelearningprocessisex-
terapproximationwouldbeFFI,whichproducestheschedulethat ecutedofflineandthegeneratedmodelscanbeusedduringruntime
isdepictedinscenario1,schedulingthefourqueriesacrossthree onincomingworkloads.Next,wedescribethesestepsindetail.
VMswithoutviolatingtheperformancegoal.
4.2 WorkloadSampling
Furthermore, there might be scenarios when none of these ap-
proximationsofferthebestsolution. Forexample,considerwork- WiSeDB first generates sample workloads based on the
loads consisting of three templates, T ,T ,T with latencies of application-providedquerytemplatesT.WecreateNrandomsam-
1 2 3
four, three, and two minutes respectively. Assume we have two pleworkloads,eachcontainingmqueries.Here,N andmmustbe
queriesofeachtemplate,q1,q1 oftemplateT ,q2,q2 oftemplate sufficientlylargesothatqueryinteractionpatternsemergeandthe
1 2 1 3 4
T ,andq3,q3oftemplateT andwewishtokeepthetotalexecu- decisiontreemodelcanbeproperlytrained.However,mmustalso
2 5 6 3
tiontimeoftheworkloadbelownineminutes. FFDwillfindthe besufficientlysmallsothatforeachsampleworkloadwecaniden-
schedule S = {[q1,q1],[q2,q2,q3],[q3]}, while FFI would tifytheoptimalscheduleinatimelymanner.
FFD 1 2 3 4 5 6
4
Su:{!"", !##,!$#} Au:{!"", V!M##,1!: $#[]} Hwoenrkcleo,avdsQre.prEeascehntvspaolssosibhlaes(aposetetnotifaullnyapssairgtinael)dsqcuheerdiueslefsrofomrtQhe,
A
v ,thatmuststillbeplacedontoVMs.
u
Bu:{!##,!$#} The start vertex, A ∈ V, represents the initial state where all
VM1: [!""] B C queriesareunassigned. Therefore, Au includesallthequeriesin
the given workload and A is empty. If a vertex g ∈ V has no
DVVuMM:{!12::## [[,]!!""$#]} D VM1: [E!u"":{,!!$###}] E … CVuM:{!1:"" [,!!##$#]} … sucnhAaesndsiuegldnegegedsiqnisuEearcireoespm,rpewlseeetnestassycsohntehedauotlfvet.ewrtoexpogssiisblaegaoctailonvse:rtex and its
1. Astart-upedge(u,v,i)representsrentinganewVMoftype
… … i,vmi.Itconnectsavertexutovwherevhasanadditional
F FVVuMM:{!12::$# [[}!]"",!##] G GVuM:{1}: [!"", !##,!$#] eqmueprtiyejsV, sMo,uiu.e.=, vsvu=. Tuhse∪wevimghijt.ofItadsoteasrt-nuopteadsgsiegnisatnhye
costtoprovisionanewVMoftypei:w(u,v)=fi.
Figure5: Asubgraphofaschedulinggraphfortwoquerytemplatesand 2. A placement edge (u,v,qx) represents placingsan unas-
Q={q11,q22,q32}.Gisonegoalvertex signed query qx ∈ u intyo the queue of a rented VM in
y u
v . Itfollowsthatv = u −qx. BecauseWiSeDBisag-
s u u y
Inordertoensurethatoursamplingcoversthespaceofpossi- nostictothespecificsofanyparticularquery,theplacement
ble workloads, we rely on uniform direct sampling of the query ofaninstanceofquerytemplateT isequivalenttoplacing
x
templates. If the sampling is not uniform, the decision tree may anyotherinstanceofT .Therefore,weincludeonlyasingle
x
notbeabletolearnhowtwoquerytemplatesinteract,orthedeci- placementedgeperquerytemplateevenifthetemplateap-
siontreemayhaveverylittleinformationaboutcertaintemplates. pearsmorethanonceinu . Thecostofanedgethatplaces
u
Furthermore,wegeneratealargenumberofsamples(e.g.,inour query qx into a VM of type i is the execution time of the
y
experimentsN =3000)inordertoensurethatourworkloadsam- query multiplied by the cost per unit time of the VM, plus
pleswillalsoincludeworkloadsthatareimbalancedwithrespectto anyadditionallyincurredpenalties:
thenumberofuniquetemplatestheyinclude.ThisallowsWiSeDB (cid:104) (cid:105)
tohandleskewedworkloads. w(u,v)= l(qyx,i)×fri +[p(R,vs)−p(R,us)] (2)
Eventhoughourtrainingwillbebasedonuniformsamplingof
Figure5showspartofaschedulinggraphforaworkloadQ =
thequerytemplates, WiSeDBcanstillhandleskewedworkloads.
{q1,q2,q2}. ArepresentsthestartvertexandGrepresentsagoal
This is because uniform direct sampling generates some sample 1 2 3
vertex.3 Ateachedge,aqueryisassignedtoanexistingVM(AB,
workloadsthatarebalancedandsomesampleworkloadsthatare
AC ,BE, EG), or a new VM is created (BD, EF). The path
notbalanced.Theunbalancedsampleworkloadsteachthedecision
ABEGrepresentsassigningthethreequeries,q1,q2,q2,tobeex-
treeclassifierhowtohandleodd,skewedworkloads,andthebal- 1 2 3
ecutedinthatorderonvm .
ancedworkloadsteachthedecisiontreehowtohandlethe“usual” 1
TheweightofapathfromthestartvertexAtoagoalvertexg
mixes.WedemonstratethisexperimentallyinSection7.5.
willbeequaltothecostofthecompletescheduleofthegoalver-
tex,cost(R,g ),foragivenperformancegoalR. Sinceallcom-
4.3 OptimalScheduleGeneration s
plete schedules are represented by some goal state, searching for
Givenasetofsampleworkloads,WiSeDBlearnsamodelbased aminimumcostpathfromthestartvertexAtoanygoalvertexg
on the optimal schedules for these workloads. To produce these willprovideanoptimalschedulefortheworkloadQ.Wethenfind
optimalsolutions, werepresentschedulesaspathsonaweighted theoptimalpaththroughthegraph,notingtheschedulingdecision
graph, and we find the minimum cost path on this graph. This madeateverystep,i.e.,whichedgewasselectedateachvertex.
graph-basedapproachprovidesanumberofadvantages.First,each Graph Reduction To improve the runtime of the search algo-
“best” path represents not only an optimal schedule, but also the rithm,wereducethegraphinanumberofways. First,weinclude
steps taken to reach that optimal schedule. The information rel- astart-upedgeonlyifthelastVMprovisionedhassomequeries
evant to each optimal decision is captured by each vertex’s state. assignedtoit,i.e.,weallowrentinganewVMonlyifthemostre-
Second,agraphrepresentationlendsitselftoanaturalwayofelim- centlyprovisionedVMisnotempty.Thiseliminatespathsthatpro-
inatingredundancyinthesearchspaceviacareful, strategicpath visionVMsthatareneverused. Second,queriesareassignedonly
pruning.Finally,thewell-studiednatureofshortest-pathproblems tothemostrecentlyprovisionedVM,i.e.,eachvertexhasoutgoing
enablestheapplicationofdeeply-understoodalgorithmswithdesir- placementedgesthatassignaqueryonlytothemostrecentlyadded
ableproperties.Next,wedescribeourgraphconstructionindetail, VM.Thisreducesthenumberofredundantpathsinthegraph,since
andhighlighttheseadvantages. eachcombinationofVMtypesandqueryorderingsisaccessibleby
Graph Construction Given a sample workload Q = onlyasingleinsteadofmanypaths. Thisreductioncanbeapplied
{q1x,q2y,...}, we construct a directed, weighted graph G(V,E) withoutlossofoptimality, e.g. withouteliminatinganygoalver-
whereverticesrepresentintermediatestepsinschedulegeneration, tices.
i.e., partialschedulesandasetofremainingqueriestobesched-
uled.Edgesrepresentactions,suchasrentinganewVMorassign- LEMMA 4.1. Given a scheduling graph G and a reduced
ing a query to a VM. The cost (weight) of each edge will be the scheduling graph Gr all goal vertices with no empty VMs in G
cost of performing a particular action (e.g., the cost of starting a arereachableinGr.
newVM).Werefertothisasaschedulinggraph. PROOF. ConsideranarbitrarygoalvertexwithnoemptyVMs
Formally,eachvertexv ∈V hasascheduleforsomequeriesof g∈G.Foranyvertexv,lethead(vs)bethemostrecentlycreated
theworkload,vs ={vmi1,vmk2,...},whichincludestheVMsto VMinvs(theheadofthequeue).
berentedforthesequeries. EachVMjoftypei,vmij,isaqueue 3ThedottedvertexrepresentstheprovisioningofthefirstVMwhichisalwaysthe
ofqueriesthatwillbeprocessedonthatVM,vmij =[qkx,qmy,...]. firstdecisioninanyschedule.
5
Letusassumehead(g )=vmi andqxisthelastquerysched- 4.4 FeatureExtraction
s j y
uled in vmij. Then there is a placement edge ep = (v,g,qyx) After we have generated the optimal schedules for each of the
connecting some vertex v with query qyx in its set of unassigned sampled workloads, we generate the training set for our decision
queries, i.e., qyx ∈ vu, tog. Further, weknowthatep isanedge tree classifier. The training set consists of (decision,features)
of Gr because ep is an assignment of a query to the most re- pairsindicatingthedecisionsthatwasmadebyA*whilecalculat-
cently created VM. If head(vs) is non-empty, and qkm is the last ing optimal schedules and performance/workload related features
queryonhead(vs),thenthereisanothervertexuconnectedtov atthetimeofthedecision.Eachdecisionrepresentsanedgeinthe
via a placement edge ep = (u,v,qkm) in Gr by the same argu- searchgraph,andisthereforeadecisiontoeither(a)createanew
ment. Ifhead(vs) = vmij isempty,thenthereisastart-upedge VMoftypei,or(b)assignaqueryoftemplateTj tothemostre-
es =(y,v,i)connectingsomevertexy,withthesamesetofunas- centlycreatedVM.Thus,thedomainofpossibledecisionsisequal
signedqueriesasv,i.e.,vu = yu,tov. Weknowthates mustbe tothesumofthenumberofquerytemplatesandthenumberofVM
an edge of Gr because v can have at most one empty VM. This types.
processcanberepeateduntilthestartvertexisreached.Therefore, Wemapeachdecision(edge)(u,v)intheoptimalpathtoaset
thereisapathfromthestartvertexinGr toanygoalvertexwith offeaturesofitsoriginvertexusincethereisacorrespondencebe-
noemptyVMsg∈G. tweenavertexandtheoptimaldecisionmadeatthatvertex.Specif-
ically,foragivenvertexu,theedgeselectedintheoptimalpathis
SearchHeuristicWiSeDBsearchesfortheminimumcostpath independent of u’s parents or children but depends on the unas-
(i.e.,optimalschedule)fromthestartvertextoagoalvertexusing signed queries u and the schedule so far, u . However, the do-
u s
theA*searchalgorithm[14]whichoffersanumberofadvantages. mainsofv andv arefartoolargetoenumerateanddonotlend
u s
First,itiscomplete,meaningthatitalwaysfindsanoptimalsolu- themselvestomachinelearningalgorithms(v andv areneither
u s
tionifoneexists. Second,A*cantakeadvantageofanadmissible numericorcategorical). Therefore,weextractfeaturesfromeach
heuristic,h(v),tofindtheoptimalpathfaster.Anadmissibleh(v) oftheverticesinalltheoptimalpathswecollectedforalloftheN
providesanestimateofthecostfromavertexvtotheoptimalgoal sampleworkloads.
vertex that is always less or equal to the actual cost, i.e., it must FeatureSelectionOneofthemainchallengesofourframework
neveroverestimatethecost.Foranygivenadmissibleheuristic,A* istoidentifyasetofefficiently-extractedfeaturesthatcanleadtoa
isoptimallyefficient,i.e.,noothercompletesearchalgorithmcould highlyeffectivedecisionmodel. Here,thespaceofcandidatefea-
searchfewerverticeswiththesameheuristic. tures is extremely large, as one could extract features relating to
The heuristic function is problem specific: in our system, the queries(e.g.,latency,tables,join,predicates,etc.),theunderlying
costofapathtov isthecostofthescheduleinvs, cost(R,vs), VM resources (CPU, disk, etc.), or combinations thereof. Since
thusitiscalculateddifferentlyfordifferentperformancegoalsR. even enumerating all possible features would be computationally
Hencesatisfyingtheadmissiblityrequirementdependsonthese- difficult,findingtheoptimalsetoffeaturesisnotfeasible. There-
mantics of the performance metric. Here, we distinguish mono- fore, we focus on features that (1) are fast to extract, (2) capture
tonically increasing performance metrics from those that are not. performanceandcostfactorsthatmayaffecttheschedulingdeci-
Aperformancegoalismonotonicallyincreasingifandonlyifthe sions,and(3)haveappearedinexistingworkloadschedulingand
penalty incurred by a schedule us never decreases when adding resourceprovisioningheuristics[7,8,19].
queries. Formally, at any assignment edge connecting u to v, Surprisingly, many “common sense” features did not prove to
p(R,vs) ≥ p(R,us). Maximum query latency is monotoni- be effective. For example, we extracted the number of available
cally increasing performance metric, since adding an additional queriesofacertaintemplateXstillunassigned,unassigned-X.
queryonthequeueofthelastprovisionedVMwillneverdecrease Sinceeachtrainingsampleissmall,thetrainingsetonlycontained
the penalty. Average latency is not monotonically increasing, as smallvaluesofunassigned-X,sothedecisiontreedidnotlearn
adding a short query may decrease the average latency and thus howtohandletheverylargevaluesofunassigned-Xencoun-
thepenalty. Formonotonicallyincreasingperformancegoals, we teredatruntime. Wefindthateffectivefeaturesmustbeunrelated
defineaheuristicfunctionh(v)thatcalculatesthecheapestpossi- tothesizeoftheworkload,sincethelargeworkloadsizesencoun-
bleruntimeforthequeriesthatarestillunassignedatv. Inother teredatruntimewillnotberepresentedinthetrainingset. Addi-
words,h(v)sumsupthecostofthecheapestwaytoprocesseach tionally,weextractedthenumberofassignedqueriesofacertain
remainingquerybyassumingVMscouldbecreatedforfree4: templateX,assigned-X.Withtightperformancegoals,thisfea-
h(v)= (cid:88) min(cid:104)fi×l(qx,i)(cid:105) (3) tureiswell-representedinthetrainingdatabecausemachines“fill
r y up”afterasmallnumberofqueries. However,thisfeatureshares
i
qyx∈vu informationwithwait-time,asthetwoarehighlyrelated. Be-
causeofthisredundancy,addingassigned-Xtothemodeldid
LEMMA 4.2. Formonotonicallyincreasingperformancegoals, not improve performance. Therefore, effective features must be
thesearchheuristicdefinedinEquation3isadmissible. reasonablyindependentfromoneanothertoavoidrepresentingre-
dundantdata.
PROOF. Regardlessofperformancegoals,queryclasses,orVM Wehaveexperimentallystudiednumerousfeatureswhichhelped
performance,onealwayshastopaythecostofrentingaVMforthe us form a set of requirements for the final feature set. First, our
durationofthequeries.Moreformally,everypathfromanarbitrary selected features should be agnostic to the specifics of the query
vertexvtoagoalvertexmustincludeanassignmentedge,withcost templatesandperformancegoals. Thiswillallowourframework
givenbyEquation2,foreachqueryinvuandforsomeVMtypei. tobecustomizableandenableittolearngoodstrategiesindepen-
Whentheperformancegoalismonotonicallyincreasing,theterm dentlyofthequerytemplatesandperformancemetricexpressedby
p(R,vs)−p(R,us)isnevernegative,soh(v)isneverlargerthan theapplication.
theactualcosttothegoalvertex. Second, effective features must be unrelated to the size of the
workload, since the large workload sizes encountered at runtime
4
Forperformancegoalsthatarenotmonotonicallyincreasing,wedonotuseaheuris-
will not be represented in the training sample workloads, whose
tic,whichisequivalenttousingthenullheuristic,h(v)=0.
6
sizeisrestrictedinordertoobtaintheoptimalschedulesinatimely (1) wait-time Initial u:{!"",!$$, !#$}
manner(e.g. trackingtheexactnumberofunassignedinstancesof >= 2 < 2 m={VM1: []}
asmpaalrltidcuurlianrgtetrmaipnliantgeawnidllvneortyblaergueseafturl,unastimtheis).valuewillbevery (2) new-VM Y(3) have-T2N Step 1 um:={{!V""M, !1:#$ }[!$$]}
Third, featuresmustbeindependentfromoneanothertoavoid (4) cost-T2 (5) assign-T1 Step 2 u={!#$}
extractingredundantinformation. Forexample,thewaittimeina < 100 >= 100 m={VM1: [!$$,!""]}
ganivdenwVeMobisserrveeladtetdhatot itnhcelunduimngbeornolfyqouneerieosfathsseisgenemdettoricthsaitnVoMur, (6) assign-T2 Y(7) have-T1 N Step 3 um=={{!V#$M}1: [!$$,!""], VM2: []}
featurelistwassufficienttogiveuslow-costschedules.
Fromaninformation-theoreticpointofview,theminimumnum- (8) assign-T1 (9) new-VM Step 4 m= {VM1: [!$$,!""], VM2: [!#$]
ber of bits needed for an optimal feature set is log (|T|+|V|), Figure6:Anexampledecisionmodel
2
where|T|+|V|issumofthenumberofquerytemplatesandthe
5. have-X:whetherornotaqueryoftemplateXisstillunas-
numberofVMtypes,becausethisistheminimumnumberofbits
signed. This feature helps our model understand how the
neededtodifferentiateeachedgeinthegraph.However,extracting
templatesoftheunassignedqueriesaffectsthedecisionson
suchanoptimalsetoffeaturesisnotlikelytobecomputationally-
theoptimalpath. Ifthereisnoinstanceofsomequerytem-
efficient because of the complexity of the problem. Since we
plateT unassigned,thenthemodelplacesoneoftheremain-
require feature extraction to be fast, we do not attempt to find j
ingtemplates. IfaninstanceofT exists, themodelmight
a feature set with such low entropy. Computationally-efficient j
prefertoschedulethatasopposedtoanyothertemplate.
approaches must therefore make use of higher-entropy, easy-to-
WiSeDBlearnsmodelsthatcombinethesefeaturesintocustom-
extractfeaturesasguidestowardsgoodapproximatesolutions.
tailoredheuristicsforspecificworkloadsandperformancegoals.
Based on these observations, we extract the following features
We note that while these features are not enough to uniquely
foreachvertexvalongtheoptimalpath:
identifyavertexandthuslearntheexactconditionsthatleadtothe
1. wait-time: theamountoftimethataquerywouldhave
optimalschedule,theycanshedlightontheworkload/performance
towaitbeforebeingprocessedifitwereplacedonthemost
conditionsrelatedtotheoptimaldecision. Furthermore,although
recentlycreatedVM.Formally,wait-timeisequaltothe
we cannot claim that these features will always allow WiSeDB
executiontimeofallthequeriesalreadyassignedtothelast
tolearneffectiveheuristics,ourexperimentalresultsindicatethat
VM. Thisfeaturecanhelpourmodeldecidewhichqueries
thesefeaturesallowWiSeDBtolearnareasonablecross-sectionof
shouldbeplacedonthelastaddedVMbasedontheirdead-
theschedulingalgorithmspace,andthattheyareexpressiveenough
line. For example, if a machine’s wait time has exceeded
to generate scheduling strategies capable of efficiently handling
the deadline of a query template, it is likely that no more
commonly used performance goals. One can invent an adversar-
queriesofthistemplateshouldbeassignedtothatVM.Al-
ial performance goal (for example, penalizing all VMs without a
ternatively,ifamachine’swaittimeisveryhigh,itislikely
primenumberofassignedqueries)thatwillfalloutsidethescope
thatonlyshortqueriesshouldbeassigned.
ofthisfeatureset.
2. proportion-of-X: the proportion of queries on the
most recently created VM that are of query template X. 4.5 WorkloadManagementModel
In other words, proportion-of-X is the ratio be-
Givenatrainingset,WiSeDBusesadecisiontreelearnertogen-
tween the number queries of template X assigned to
erateaworkloadmanagementmodel. Figure6showsanexample
the VM and the total number of queries assigned to
model defined for two query templates T and T . Each feature
the VM. For example, if the VM currently has four 1 2
node(orangenode)ofthedecisiontreerepresentseitherabinary
queries assigned to it, with one of those queries be-
splitonanumericorbooleanfeature. Thedecisionnodes(white
ing of template T and three being of template T , then
1 2 nodes)representthesuggestedactions.
we extract the features proportion-of-T1=0.25 and
The decision tree maps a vertex v to an action a by extracting
proportion-of-T2=0.75. We only need to consider
featuresfromvanddescendingthroughthetree.Sinceeachaction
themostrecentlycreatedVMbecausetheassignmentedges
a is represented by a single out-edge of v, one can navigate the
inthereducedgraphonlyassignqueriestothemostrecent
decision tree by traversing the edge corresponding to the action
VM. Since each sample workload contains only a limited
selected.
numberofqueries,keepingtrackoftheexactnumberofin-
TherightsideofFigure6showshowthedecisiontreeisused
stancesofeachquerytemplatewouldnotscaletolargework-
to come up with the schedule for a workload Q = {q1,q2,q2}.
loads.Therefore,wetracktheproportioninstead. 1 2 3
EachqueryinT hasalatencyoftwominutesandthegoalisto
3. supports-X:whetherornotthemostrecentlycreatedVM 1
executeitwithinthreeminutes. InstancesofT havealatencyof
is capable of processing a query of class X. This feature is 2
oneminuteandthegoalisforeachinstancetobecompletedwithin
neededwhennotallVMtypesarecapableofhandlingevery
oneminute. Forsimplicity,weassumeVMsofasingletypeand
typeofquery.
thatqueriesareexecutedinisolation.
4. cost-of-X:thecostincurred(includinganypenalties)by
Given this workload, the tree is parsed as follows. In the first
placing a query of template X on the most recently created
node (1), we check the wait time, which is zero since all queries
VM,orinfinityifthemostrecentlycreatedVMdoesnotsup-
areunassigned,henceweproceedtonode(3). Theworkloadhas
portqueriesofclassX.cost-of-Xisequaltotheweight
queriesoftemplateT andthereforeweproceedtonode(4). Here
oftheoutgoingassignmentedgefortemplateX.Thisallows 2
wecalculatethecostofplacinganinstanceofT .Letusassumethe
ourmodeltocheckthecostofplacinganinstanceofacer- 2
costisislessthan100(nopenaltyisincurred),whichleadstonode
tainquerytemplateand,basedonitsvalue,decidewhether
(6)whichassignsaninstanceofT tothefirstVM.Sincewehave
toassignanotherquerytothelastrentedVMorcreateanew 2
morequeriesintheworkloadwenextre-parsethedecisiontree.In
VM.
node(1)wecheckthewaittimeonthemostrecentVMwhichis
now1minute(theruntimeofqueriesofT )sowemovetonode
2
7
t=0 t=1m t=2m t=2.5m t=3m hasbeenmovedupbyhalfaminute,causingSchedule1toincura
VM1 !## !"" penaltyitdidnotpreviouslyincur.
Inordertogenerateanewmodelα(cid:48),WiSeDBre-usesthetrain-
Scenario 1 VM2 !$# violation period ingdataoftheexistingmodelαasfollows.Foreachsamplework-
VM3 !%# loadusedforα,itmodifiesitscorrespondingschedulinggraphG
violation period by updating the weights to reflect the new performance goal R(cid:48).
VM1 !"" !## Specifically,start-upedgesmaintainthesamecost(thecosttostart
Scenario 2 violation period a new VM), while the cost of placement edges increase due to a
VM2 !$# !%# stricterperformancegoalsandhencehigherpenaltyfees.Formally,
thenewcostofanassignmentedge(u,v,qx)thatplacesanunas-
C2deadline C1deadline Old C1deadline signedqueryqx ∈u intothequeueofareyntedVMinv is:
y u s
Figure7:TwodifferentscheduleswithshiftedSLA
w(u,v)+(cid:2)p(R(cid:48),v )−p(R,v )(cid:3)−(cid:2)p(R(cid:48),u )−p(R,u )(cid:3)
s s s s
Toidentifytheminimumcostpathontheupdatedgraph,weuse
(3). SincewehaveonemorequeryofT2 unassigned,wemoveto anewheuristic. Formetricsthataremonotonicallyincreasing,the
(4).NowthecostofassigningaqueryofT2ismorethan100since newheuristich(cid:48)isexpressedintermsoftheoriginalonehas:
the new query would need to wait for q2 to complete (and thus
1 h(cid:48)(v)=max[h(v),cost(R,g)−cost(R,v)]
incurapenalty).Hence,wemovetonode(7)andwecheckifthere
areanyunassignedinstancesofT .Sincethereare(q1),weassign
1 1 Forperformancegoalsthatarenotmonotonicallyincreasing,like
q11 to the last VM. We re-parse the tree in the same way and by average latency, we simply drop the h(v) term, giving h(cid:48)(v) =
followingnodes(1)→(2), thenagainas(1)→(3)→(4)→(7)→(9),
cost(R,g)−cost(R,v).
soweassigntheremainingqueryq32ontoanewVM. We use h(cid:48) to find the optimal schedule for each of α’s sample
Eachmodelrepresentsaworkloadschedulingstrategy. Givena
workloads(i.e.,wesearchfortheminimumcostpathsoneachsam-
batch of queries, the model in Figure 6 will place an instance of
ple’s scheduling graph G with the updated weights). These new
T2,thenaninstanceofT1,andthencreateanewVM.Thisprocess pathswillserveastrainingsamplesforα(cid:48). Intuitively,h(cid:48)(v)gives
will repeat itself until queries of T or T are depleted from the
1 2 thecostofgettingfromvertexv totheoptimalgoalvertexunder
incomingbatch. Whenallinstancesfromoneofthetemplatesare
the old performance goal R. Since the new performance goal is
assigned,singleinstancesoftheremainingtemplatewillbeplaced strictlytighterthanR,h(cid:48)(v)cannotoverestimatethecost,making
onnewVMsuntilnoneremain.Thisstrategyisequivalenttofirst-
itanadmissibleheuristic.Next,weprovethisformally.
fit increasing, which sorts queries in decreasing order of latency
andplaceseachqueryonthefirstVMwherethequery“fits”(i.e., LEMMA 5.1. h(cid:48)(v) is an admissible heuristic, i.e, it does not
incursnopenalty).
overestimatethecostoftheschedulev atthevertexv.
s
PROOF. Basedonourcostformula(Equation1),foranysched-
5. ADAPTIVEMODELING ulesgeneratedbythemodelα,thenewperformancegoalR(cid:48)will
onlyaffectthepenalty(i.e.,sincestricterperformancegoalsonthe
Itisoftendesirabletoallowuserstoexploreperformance/cost
sameschedulecanleadonlytomoreviolations).Therefore:
trade-offswithinthespaceofpossibleperformancegoals[24].This
canbeachievedbygeneratingdifferentmodelsforthesamework- ∀s(p(R(cid:48),s)≥p(R,s)) (4)
loadwithstricter/looserperformancegoalsandthushigher/lower
costs. However, WiSeDB tunes its decision model for a specific Tomapthisintoourschedulinggraph,letusconsiderthesetofver-
goal. Changes in this goal will trigger WiSeDB to re-train the ticesalonganoptimalpathtoagoalvertexgforonesamplework-
model,sincechangesinperformancegoalleadtodifferentoptimal loaddiscoveredusedforthetrainingofα.Foranyparticularvertex
schedulesandhencedifferenttrainingsets.Therefore,generatinga v,weknowfromEquations 1and4thatcost(R(cid:48),v)≥cost(R,v).
setofalternativedecisionmodelsfornumerousperformancecon- Inotherwords, thecostofthatvertex’spartialschedulewillcost
straintscouldimposesignificanttrainingoverhead. moreifwehaveastricterperformancegoal. Sincethisholdsfor
To address this challenge, WiSeDB employs a technique that allverticesontheoptimalpath,italsoholdsfortheoptimalgoal
adaptsanexistingmodeltrainedforagivenworkloadandperfor- vertexgunderα.Hence,iftheperformancegoalbecomesstricter,
mancegoaltoanewmodelforthesameworkloadandstricterper- thecostofthefinalschedulecanonlyincrease.
formancegoals. Ouradaptivemodelingrequireslittlere-training ThisisillustratedinFigure7: thecostofScenario1increased
sinceitleveragesthefactthattwodecisionmodelswillsharesig- whentheperformancegoalwasshifted.
nificantpartsoftheirtrainingsetsiftheyweretrainedforthesame Furthermore, for any vertex v along the optimal path to g, we
query templates T and similar performance goals. Our approach knowthattheminimumpossiblecosttogofromvtogisexactly
reliesontheadaptiveA*algorithm[17],whichreusesinformation cost(R,g)−cost(R,v).Iftheedgeweightcanonlyincreasedue
fromonegraphtocreateanewsearchheuristic, h(cid:48)(v), tosearch toastricterperformancegoal,thenthecosttogofromvtogunder
anothergraphwithidenticalstructurebutincreasededgeweights. R must be less than or equal to the cost to go form v to g un-
Graph Reuse Let us assume a model α trained for templates derR(cid:48). Formally,sincecost(R(cid:48),v) ≥ cost(R,v),itfollowsthat
T and goal R and a desired new model α(cid:48) trained for the same cost(R(cid:48),g)−cost(R(cid:48),v)≥cost(R,g)−cost(R,v).
templatesT butadifferentperformancegoalR(cid:48). Withoutlossof Forexample,inthegraphshowninFigure5,cost(R,B) = 6
generality,weonlyconsidercaseswheretheperformancegoalR(cid:48) andcost(R,G)=26.ThecosttogetfromBtoGisthus20.IfR(cid:48)
isstricterthanR,sinceonecanstartwithasubstantiallylooseper- resultedintheassignmentEGhavingahighercostof100(because
formancegoalthantheonerequestedandrestrictitincrementally. q2missesitsdeadline),thenitholdsthatthecostofgoingfromB
3
For example, Figure 7 shows a modified version of the SLA toGunderR(cid:48)(110)isgreaterthanthecostofgoingfromBtoG
shown in Figure 3. In Figure 7, the deadline for query class C underR:cost(R(cid:48),G)−cost(R(cid:48),G)≥cost(R,G)−cost(R,B).
2
8
Whiletheoptimalgoalvertexforasetofqueriesmaybediffer- 0m 0.5m 1m 1.5m 2m 3m 4m
entunderRthanunderR(cid:48),thecostoftheoptimalvertexg(cid:48)under At t1 VM1 !""
R(cid:48)isnolessthanthecostoftheoptimalvertexgunderR,because
gswasoptimalunderR.Intuitively,iftherewassomevertexγand At t2 VM1 !"" !#"
completescheduleγ ofaworkloadunderR(cid:48)withalowercostthan
s
uonpdtiemraRls,cwhehdicuhlecgosn,trtahdeinctγssthwaotuglsdiaslsoopthiamvael.aTlohweererfcooresttthheancogsst At t3 VVMM21 !"" !#" !$"
fromanyvertexvunderR(cid:48)totheunknownoptimalgoalvertexg(cid:48)
t1 t2 t3 t4
isnolessthanthecosttogetfromvtogunderR:
Figure8:Onlineschedulingexample
cost(R(cid:48),g(cid:48))−cost(R(cid:48),v)≥cost(R,g)−cost(R,v)
Hencecost(R,g)−cost(R,v)isadmissiblesinceitneveroveres- processwasgiveninSection4.5. Thescheduleindicatesthenum-
timatestheactualcosttogetfromavertexvtotheunknowngoal berand typesofVMs needed, theassignment ofqueriesto VMs
vertexg(cid:48)undertheperformanceR(cid:48). andthequeryexecutionorderineachVM.Theapplicationisthen
responsible for executing these scheduling decisions on the IaaS
Sinceh(cid:48)(v)isadmissible,answersproducedbyusingitareguar- cloudinfrastructure.Intheeventthattheworkloadincludesaquery
anteed by the A* algorithm to be correct [14]. The improved that cannot be exactly matched to one of the templates given in
heuristic is able to find the optimal solution much faster, as we theworkloadspecification,WiSeDBwilltreattheunknownquery
willdemonstrateexperimentallyinSection7.Thisapproachsaves asaninstanceofthequerytemplatewiththeclosestpredictedla-
WiSeDBfromsearchingtheentireschedulinggraphforeachnew tency. Whilequeriesofunseentemplatesareobviouslynotideal,
model,whichisthestepwiththedominatingoverhead. thissolutionensuresthatqueriesofunseentemplateswillatleast
be placed appropriately based on their latency, since two queries
6. RUNTIMEFUNCTIONALITY withidenticallatencyareidenticaltoWiSeDB.
Using our adaptive modeling approach, WiSeDB recommends
6.3 OnlineQueryScheduling
totheapplicationasetofdecisionmodels(a.k.a. workloadman-
agementstrategies)forschedulingincomingqueries. Duringrun- WiSeDBcanalsohandlenon-preemptiveonlinequeryschedul-
time,theuserselectsamodelwithadesirableperformancevs.cost ing. We will describe how WiSeDB handles the general case of
trade-off.Givenabatchqueryworkload,WiSeDBusestheselected onlineschedulingandthenwewilldiscusstwooptimizations.
modeltogeneratethescheduleforthatworkload.Thesamemodel GeneralOnlineSchedulingLetusassumeadecisionmodelα
canalsobeusedforonlineschedulingwherequeriesarriveoneata trainedforaperformancegoalRandtemplatesT.Wealsoassume
time. Thesystemremainsinthismodeuntiltheusereitherwishes that a user submits queries, {qx,qy,...,} at times {t ,t ,...}
1 2 1 2
toswitchstrategyorhasnomorequeriestoprocess. respectively. Onlineschedulingcanbeviewedasaseriesofsuc-
cessivebatchschedulingtaskswhereeachbatchincludesasingle
6.1 StrategyRecommendation
additionalquery.Thefirstqueryisscheduledattimet asifitwas
1
Whilegeneratingalternativedecisionmodels,ourgoalistoiden- a batch. When query qx arrives at t , we create a new batch B
i i i
tify a small set of k models that represent significantly different containingthequeriesthathavenotstartedexecutingbyt anduse
i
performance vs. cost trade-offs. To achieve this we first create themodeltore-schedulethemalongwithqueryqx.
i
asequenceofperformancegoalsinincreasingorderofstrictness, SchedulingB asabatchusingthesamedecisionmodelmight
i
R = R ,R ,...,R ,inwhichtheapplication-definedgoalRis not offer low-cost solutions. This is because this batch includes
1 2 n
themedian. ForeachgoalR ∈R,wetrainadecisionmodel(by queriesthathavebeensittinginthequeue,andhencetheirexpected
i
shifting the original model as described in Section 5) and calcu- latency is now higher than what was assumed when training the
late the average cost of each query template over a large random model. Formally, atanygiventimet , anyqueryqx thathasyet
i y
sample workload Q. Then, we compute the pairwise differences torunwillhaveawaittimeof(t −t )andthusafinallatencyof
i y
between the average cost of queries per template of each perfor- l(cid:48)(qx,k)=l(qx,k)+(t −t ),wherekisthetypeoftheVMon
y y i y
mance goal R ∈ R using Earth Mover’s Distance [6]. We find whichqxwasassignedandt isthearrivaltimeofqx. Toaddress
i y y y
thesmallestsuchdistancebetweenanypairsofperformancegoals, this, WiSeDB treats qx as if it were of a “new” template which
y
EMD(R ,R ), and we remove R from the sequence. We has the same structure at T but its expected latency is l(cid:48)(qx,k).
i i+1 i+1 x y
repeatthisprocessuntilonlykperformancegoalsremain. Then, it trains a new decision model for the augmented template
A similar technique was also used in [24] where it was shown setthatincludesthis“new”template.Thisnewmodelwillprovide
that it produces a desirable distribution of performance goals (or schedulesthataccountfortheelapsedwaittimeofqx.
y
“service tiers”) that represent performance/cost trade-offs. How- Figure8showsanexample. Here,amodelαistrainedforone
ever,unlikein[24],wedonotneedtoexplicitlyexecuteanywork- templateT . InstancesofT havealatencyof2m. Weassume
1 1
loadsinordertocomputetheexpectedcostofaparticularstrategy. queriesarriveintheorder: {q1,q1,q1}. WiSeDBfirstschedules
1 2 3
Instead,foreachstrategy,WiSeDBprovidesacostestimationfunc- queryq1,whicharrivedatt ,asabatchB ={q1}. Att ,q1ar-
1 1 1 1 2 2
tion that takes as a parameter the number of instances per query rivesandWiSeDBcreatesabatchB ={q1}foritandgeneratesa
2 2
template. Users can easily experiment with different parameters scheduleusingmodelαwhichassignsittothefirstVM,rightafter
forincomingworkloadsandcanestimatetheexpectedcostofexe- thefirstquery. Attimet ,whenonemorequeryq1arrives,q1has
3 3 2
cutingtheseworkloadsusingeachoftheproposedstrategies. notyetstartedexecuting,sowecreateanewbatchB ={q1,q1}.
3 2 3
However, even though q1 and q1 are of the same template, they
6.2 BatchQueryScheduling 2 3
havedifferentexecutiontimes:q1hasalatencyof1minute,butq1
3 2
Givenoneoftherecommendedstrategiesandabatchofqueries hasalatencyof1+(t −t )minutessinceithasbeenwaitingfor
3 2
to be executed on the cloud, WiSeDB uses the strategy and pro- (t −t )minutes.Usingmodelα,whichhasbeentrainedtohandle
3 2
duces a schedule for this workload. A detailed example of this queriesofwithanexecutiontimeofoneminute,mightnotproduce
9
adesirable(low-cost)scheduleforB . So,wetrainanewmodel, Nextwepresentourexperimentalresults,whichfocusonevalu-
3
α(cid:48),whoseworkloadspecificationincludesan“extra”templatefor atingWiSeDB’seffectivenesstolearnlow-costworkloadschedules
instancesofT withlatencyof1+(t −t )minutes.α(cid:48)placesq1 foravarietyofperformancegoals,aswellasitsruntimeefficiency.
1 3 2 3
afterq1onthefirstVMandq1ontoanewVM.
1 2
7.1 ExperimentalSetup
6.3.1 RetrainingOptimizations
WeimplementedWiSeDBusingJava8andinstalleditonanIn-
Asubstantiallyfastqueryarrivalratecouldrapidlycreate“new
telXeonE5-2430serverwith32GBofRAM.Theservicegenerates
templates”asdescribedabove,causingahighfrequencyofretrain-
schedulingstrategiesforadatabaseapplicationdeployedon6In-
ing.Sincetrainingcanbeexpensive,weoffertwooptimizationsto
telXeonE5-2430serversthatcanhostupto24VMswith8GBof
preventoraccelerateretraining.
RAMeach.ThisprivatecloudemulatesAmazonAWS[1]byusing
Model Reuse Upon arrival of a new query, WiSeDB creates a
querylatenciesandVMstart-uptimesmeasuredont2.medium
newdecisionmodelwhichtakeintoaccountthewaittimesofthe
EC2instances. Bydefault, ourexperimentsassumeasingletype
queriessubmittedsofarbutnotalreadyrunning. WiSeDBstrives
ofVMunlessotherwisespecified.
toreusemodels, aimingtoreducethefrequencyofretraining. In
Our database application stores a 10GB configuration of the
theaboveexample,letusassumeanewqueryq1 arrivesatt ,at
4 4 TPC-H [4] benchmark on Postgres [3]. Query workloads consist
whichpointweneedtogenerateascheduleforB ={q1,q1}(q1
4 3 4 1 ofTPC-Htemplates1-10. Thesetemplateshavearesponsetime
andq1arerunning). If(t −t )=(t −t ),thenthepreviously
2 3 2 4 3 rangingfrom2to6minutes,withanaveragelatencyof4minutes.
generatedmodelα(cid:48)canbereusedtoscheduleB ,sincethe“new”
4 We set the query processing cost to be the price of that instance
templaterequiredtoscheduleq1isthesameasthe“new”template
3 (f =$0.052perhour),andwemeasureditsstart-upcostexperi-
previouslyusedtoscheduleq1. r
2 mentallybasedonhowlongittookfortheVMtobecomeavailable
Next,letusconsiderthegeneralcase. Wedefineω(i)tobethe
viaconnectionsafteritentereda“running”state(f =$0.0008).
differenceintimebetweenthearrivalofoldestquerythathasnot s
We trained our models on N = 3000 sample workloads with
startedprocessingattimet andthearrivalofthenewestqueryat
i asizeofm = 18queriesperworkload,withqueriesexecutedin
timet .Formally,
i isolation. Higher values for N and m added significant training
ω(i)=t −t where overheadwithoutimprovingtheeffectivenessofourmodelswhile
i j
j =min{j |j ≤i∧∃qx(qx ∈B ∧qx ∈B )} lower ones resulted in poor decision models. We generated our
y y j y i modelsusingtheJ48[5]decisiontreealgorithmandusedthemto
Clearly, the absolute time of a workload’s submission is irrel- generateschedulesforincomingworkloads.Ourexperimentsvary
evant to the model: only the difference between the current time the size of these workloads as well as the distribution of queries
and the time a workload was submitted matters. Formally, if acrossthetemplates.Eachexperimentreportstheaveragecostover
ω(i) = ω(j), then B and B can be scheduled using the same 5workloadsofagivensizeandquerydistribution.
i j
model. In practice, B and B can be scheduled using the same PerformanceMetrics. Ourexperimentsarebasedonfourper-
i j
modelifthedifferencebetweenω(i)andω(j)islessthantheer- formancegoals: (1)Maxrequiresthemaximumquerylatencyin
rorinthequerylatencypredictionmodel. Bykeepingamapping theworkloadtobelessthanxminutes.Bydefault,wesetx=15,
ofω(x)todecisionmodels5,WiSeDBcansignificantlyreducethe which is 2.5 times the latency of the longest query in our work-
amountoftrainingthatoccursduringonlinescheduling. load. Thereisachargeof1centpersecondforanyquerywhose
Linear Shifting For some performance metrics, scheduling latencyexceedsxminutes. (2)PerQueryrequiresthateachquery
queries that have been waiting in the queue for a certain amount ofagiventemplatenotexceeditsexpectedlatencybyafactorof
oftimeisthesameasschedulingwithastricterdeadline.Thiscan x. Bydefault,wesetx = 3sothattheaverageofthesedeadlines
significantlyreducethetrainingoverheadasthenewmodelscould isapproximately15minutes,whichis2.5timesthelatencyofthe
reusetheschedulinggraphofthepreviousonesasdescribedinSec- longestquery. Thereisachargeof1centpersecondinwhichany
tion5. Consideraperformancegoalthatputsadeadlineoneach queryexceedsxtimesitspredictedlatency. (3)Averagerequires
querytoachievealatencyofthreeminutes.Ifaqueryisscheduled thattheaveragelatencyofaworkloadisxminutes.Wesetx=10
one minute after submission, that query can be scheduled as if it sothatthisdeadlineis2.5timestheaveragequerytemplatelatency.
werescheduledimmediatelywithadeadlineoftwominutes. Thereisachargeofanumbercentsequaltothedifferencebetween
Thisoptimizationcanbeappliedonlytolinearlyshiftableper- theaveragelatencyofthescheduledqueriesandx.(4)Percentre-
formance metrics. In general, we say that a metric R is linearly quiresthaty%ofthequeriesintheworkloadfinishwithinxmin-
shiftableifthepenaltyincurredunderRforaschedulewhichstarts utes. By default, we set y = 90 and x = 10. If y% of queries
queries after a delay of n seconds is the same as the penalty in- finishwithinxminutes,thereisnopenalty. Otherwise,apenalty
curredunderR(cid:48)byaschedulethatstartsqueriesimmediately,and ofonecentpersecondischargedfortimeexceedingxminutes.
whereR(cid:48) isatighteningofRbysomeknownfunctionofn. For
themetricofmaximumquerylatencyinaworkload,thisfunction 7.2 EffectivenessResults
of n is the identity: the penalty incurred under some goal R af-
Todemonstratetheeffectivenessandversatilityofourapproach,
teradelayofnsecondsisequaltothepenaltyincurredunderR(cid:48),
wecompareschedulesgeneratedbyWiSeDBwithoptimalsched-
whereR(cid:48)isRtightenedbynseconds. Forlinearshiftableperfor-
ulesandknownheuristicsthathavebeenshowntoofferlow-cost
mancemetrics,traininganewmodelwhennewqueriesarrivecan
solutionsforspecificperformancemetrics.
bereducedtoshiftingadecisionmodel,asdescribedinSection5.
OptimalitySinceourschedulingproblemiscomputationallyex-
pensive,calculatinganoptimalscheduleforlargeworkloadsisnot
7. EXPERIMENTALRESULTS feasible.However,forsmallerworkloadsizes,wecanexhaustively
search for an optimal solution. Figure 9 shows the final cost for
5 scheduling workloads of 30 queries uniformly distributed across
Experimentally,wefoundthatthismappingcanbestoredusingafewMBsinceeach
decisiontreeisrelativelysmall. 10querytemplates. WecompareschedulesgeneratedbyWiSeDB
10