Explorative Benchmarking of JavaScript Memory Management

12 Pages·2014·0.67 MB·English

ACDC-JS: Explorative Benchmarking of
JavaScript Memory Management
MartinAigner+ ThomasHu¨tter+ ChristophM.Kirsch+
AlexanderMiller+ HannesPayer∗ MarioPreishuber+
+UniversityofSalzburg ∗Google,Inc.
fi[email protected] [email protected]
Abstract are tuned to perform well on such benchmarks. Furthermore, the
We present ACDC-JS, an open-source1 JavaScript memory man- benchmarksuitesonlycoverasmallsetofpossibleworkloads.
One important issue in JavaScript performance is memory
agementbenchmarkingtool.ACDC-JSincorporatesaheapmodel
managementwhichaffectsallimportantperformancedimensions
based on real web applications and may be configured to expose
namely throughput and latency as well as memory consumption.
virtually any relevant performance characteristics of JavaScript
However, the most common JavaScript benchmarking suites Oc-
memorymanagementsystems.ACDC-JSisbasedonACDC[11],
tane (Google) [5], Kraken (Mozilla) [4], and SunSpider (We-
abenchmarkingtoolforC/C++thatmodelsperiodicallocationand
bKit)[9]focusmainlyonthroughputandonlytworecentbench-
deallocationbehavior(AC)aswellaspersistentmemory(DC).We
marks in Octane (SplayLatency and MandreelLatency) provide
identify important characteristics of JavaScript mutator behavior
information on garbage collection latency and compiler latency,
and propose a configurable heap model based on typical distri-
respectively. No memory consumption is reported by any bench-
butions of these characteristics as foundation for ACDC-JS. We
markingsuitebecausethereisnoportablewaytomeasurememory
describe heap analyses of 13 real web applications extending ex-
consumptionfromwithinJavaScriptrunninginabrowser.
istingworkonJavaScriptbehavioranalysis[13].Ourexperimental
WeproposeACDC-JS,anopensourceJavaScriptmemoryman-
results show that ACDC-JS enables performance benchmarking
agement benchmarking tool based on ACDC which is a tool for
anddebuggingofstate-of-the-artJavaScriptvirtualmachinessuch
benchmarking memory allocators for C/C++ [11]. ACDC-JS im-
asV8andSpiderMonkeybyexposingkeyaspectsoftheirmemory
plementsanconfigurableallocationmodelthatallocatesobjectsof
managementperformance.
differentsizesperiodically(AC)aswellaspermanently(DC).Ad-
Categories and Subject Descriptors D3.4 [Programming Lan- ditionally, ACDC-JS may be configured to create heap structures
guages]:MemoryManagement toemulatedatastructuresfoundinrealapplications.ACDC-JSre-
ports throughput and latency of memory allocation and access as
GeneralTerms Performance,Measurement
well as memory consumption. ACDC-JS may thus expose trade-
Keywords benchmarking;automaticheapmanagement;program offsofimplementationdetailsofJavaScriptgarbagecollectors.
behavior Inthisworkweextendrecentstudiesontheallocationbehavior
ofrealJavaScriptapplications[13]andusethisinformationtoim-
plementaconfigurablemutatormodelforbenchmarkingJavaScript
1. Introduction
memory management systems.Although we believe that it is not
JavaScript performance is an important issue for supporting the possible to have the realistic benchmark, we do believe that hav-
next generation of web applications. Browser vendors compare ing a configurable workload generator based on properties found
their products mainly by presenting JavaScript benchmarking re- in real web applications is useful to cover a large set of possible
sults. As a consequence, development of JavaScript virtual ma- andmeaningfulworkloads.Wealsoverifyexperimentallythatthe
chines is driven by industry-standard benchmarking suites. How- allocationbehaviorofACDC-JSiscomparabletotheaverageallo-
ever,recentstudiessuggestthatsuchbenchmarksdonotreflectthe cationbehavioroftherealJavaScriptapplications.
behavior of real web applications [13]. Speedups based on such Our experimental evaluation of the state-of-the-art JavaScript
benchmarks do therefore not necessarily translate into speedups virtual machines V8 and SpiderMonkey shows that ACDC-JS is
of real web applications [16] since JavaScript implementations capableofexposingdifferencesinallrelevantperformancedimen-
sionsaswellasexposingtrade-offsofdifferentgarbagecollection
1https://github.com/chromium/ACDC4GC policiesimplementedindifferentconfigurationsoftheVMs.The
evaluationfollowsa4-stepprocessforperformancedebuggingthat
beginswithselectinganexpectedperformancecharacteristicofa
givenmemorymanagementfeature,e.g.,highallocationthrough-
Permissiontomakedigitalorhardcopiesofallorpartofthisworkforpersonalor
putwithgenerationalGCsonworkloadswithmostlyshort-living
classroomuseisgrantedwithoutfeeprovidedthatcopiesarenotmadeordistributed
forprofitorcommercialadvantageandthatcopiesbearthisnoticeandthefullcitation objects.Wethendesignaworkloadthatexposesthischaracteristic
onthefirstpage.CopyrightsforcomponentsofthisworkownedbyothersthanACM and,ideally,stillresemblesthememoryusageofrealapplications.
mustbehonored.Abstractingwithcreditispermitted.Tocopyotherwise,orrepublish, ThethirdstepistoconfigureACDC-JSsuchthatitemulatesthis
topostonserversortoredistributetolists,requirespriorspecificpermissionand/ora
workload. The final step is to run ACDC-JS in this configuration
[email protected].
andcomparetheresultwithourexpectations.
DLS’14, October20–24,2014,Portland,OR,USA.
Copyright(cid:13)c 2014ACM978-1-4503-3211-8/14/10...$15.00.
http://dx.doi.org/10.1145/http://dx.doi.org/10.1145/2661088.2661089
67
Intotal,wepresentthreeexperiments:(1)throughputofgener- modelthatpreservesrepresentativenessofbenchmarksbutcanalso
ationalgarbagecollection,(2)latencyofincrementalmarking,and beconfiguredtoexposecornercasesintheperformanceofmem-
(3)performancerobustnessagainstvaryingobjectsizes.Thefirst orymanagementsystems.However,ACDC-Cfocusesonexplicit
experiment confirms the performance advantage of generational allocation and deallocation in C/C++. Our model requires differ-
garbagecollectiononworkloadswithmostlyshort-livingobjects. entparametersandadifferentimplementationtorepresentmemory
This experiment also reveals the impact of incremental marking managementworkloadsofJavaScriptapplications.Section4gives
onthroughput,latency,andmemoryconsumptioninthepresence ashortoverviewofACDC-Canddescribesouradaptationstowards
of long-living objects. The second experiment follows up on the ACDC-JS.
firstwithanin-depthanalysisofthefullthroughput,latency,and AlotofworkhasbeenpublishedonbenchmarkingJavavirtual
memory consumption trade-off with incremental marking, and in machinesincludingtheirmemorymanagementperformance.The
turnrevealsthecostofbarriersforgenerationalgarbagecollection. DaCapo benchmark suite [12] seems to be the gold standard for
Thethirdexperimentchecksperformancerobustnessagainstvary- Javaperformanceanalysistoday.However,theDaCaposuiteisa
ingobjectsizesandrevealsapotentialperformancebuginV8that selectionofrealapplicationscoveringalargerangeofworkloads
produceslargevariationsinlatencyforneighboringsizeclasses. and thus very different from ACDC’s approach. We have ported
Thispapermakesthefollowingcontributions:(1)Ananalysis ACDC-CtoJavaaswellbutareleaseincorporatingourexperiences
ofpopularandJavaScript-intensiverealwebapplicationstoobtain withACDCforJavaisfuturework.Tothebestofourknowledge
distributionsofobjectsizeandheapstructureproperties(confirm- thereisalsonobenchmarkingsuiteforothermanagedlanguages
ingexistingstudiesonthedistributionofobjecttypesandlifetimes) thatissimilartoourapproach.
and (2) a JavaScript implementation of the ACDC benchmarking
tool that is enhanced to incorporate (1) and capable of exposing 3. JavaScriptHeapAnalysis
performance differences and anomalies in various configurations
Modelingaconfigurableallocationbehaviorrequiresunderstand-
ofV8andSpiderMonkeybyexploringalargerangeofworkloads.
ingthepropertiesoftheallocationbehaviorofreal-worldapplica-
tions.RecentworkonJavaScriptapplicationbehavior[13,14,16]
2. RelatedWork
provides empirical data of real web applications and shows that
Given the importance of JavaScript for programming web appli- state-of-the-art benchmarks do not reflect their allocation behav-
cations, there are surprisingly few papers on JavaScript perfor- ior. Unfortunately, not all aspects that we would like to model
manceanalysis.DevelopersofJavaScriptimplementationsrelyon in ACDC-JS are covered, namely distribution of object sizes and
benchmarksuiteslikeOctane(Google)[5],Kraken(Mozilla)[4], structural information of typical JavaScript heaps. In this section
and SunSpider (WebKit) [9] to evaluate the performance impact wedescribeouranalysisofreal-worldJavaScriptapplicationsand
of certain implementation details. However, behavioral analysis presenttheresultsthatweincorporateinourallocationmodelof
and comparison to the behavior of real-world applications sug- ACDC-JS.
gestthatthebenchmarksuitesarenotrepresentativeofrealwork- OurheapanalysisisbasedonsamplingtheJavaScriptheapin
loads[13,14,16]. regular intervals. The applications and user interactions we have
Recentliterature[16]proposesarecord-and-replayapproachto analyzed are listed in Table 1. For comparability, the selection is
create macro benchmarks automatically based on real web appli- similartopreviousworkonJavaScriptapplicationbehavior[13].
cations.Whilethisapproachimprovestherepresentativenessofa We utilize the heap snapshot facility of the Chrome Dev-
benchmarkitdoesnotallowtocreateworkloadsforexposingcer- Tools[3]whichispartoftheV8JavaScriptvirtualmachine.Note
tain performance characteristics of a JavaScript implementation. thatwemodifiedV8tocreateaheapsnapshotautomaticallyata
ACDC-JS, on the other hand, aims at preserving the representa- samplerateof4KB,i.e.,asnapshotiscreatedwhenever4KBof
tivenessofrealworldapplications(macrobenchmarks)whilealso newobjectsareallocatedbytheapplication.Takingaheapsnap-
providingtheabilitytotesttheperformanceofspecificimplemen- shot triggers a full GC cycle so the resulting snapshot is a graph
tation details (micro benchmarks). However, since ACDC-JS can representation of all reachable objects on the JavaScript heap. It
neveremulateallpossiblerealapplicationswebelievethatreplay- containsinformationaboutthesizeoftheobjects,theirtype,and
ingtracesofrealapplicationscanbeameaningfulextensiontothe alsothestructureoftheheapthroughreferencesbetweentheob-
syntheticnatureofACDC-JS. jects.ThewebapplicationsareexecutedintheChromiumbrowser
PreviousworkonrealJavaScriptapplicationbehavior[13,14] running our modified version of V8 and the user interactions are
performedanextensiveanalysisintermsoftheexecutionoffunc- automated by the Selenium browser automation framework [8].
tionsandcode,heapallocationofobjectsanddata,andtheperfor- Note that performing our analysis based on V8 might have side
manceofeventhandling.Inourworkwefocusonheapallocation effectsnotvisibleinotherVMs.However,webelievethatforour
behaviorandextendpreviouswork[13]withananalysisofobject purposetheabstractionoftheheapsnapshotsisportableenoughto
sizedistributionandheapgraphpropertiestoprovideamorede- createaconfigurablemutatormodel.
tailedallocationmodelthatcoversallrelevantaspectsofmemory For our model, we are interested in the distribution of object
managementperformanceinJavaScriptimplementations. types,sizesandlifetimes,thenumberofoutgoingedges,andthe
Another approach to extend the understanding of JavaScript distance from the GC roots to the objects. For each property, we
behavior focuses on the use of dynamic language features, e.g., nowdescribethewayinwhichwehaveretrievedthedistribution
changingtheprototypechain,addingordeletingproperties,orthe fromthesnapshotsanddiscusstheresults.
useofeval,ratherthantheutilizationofdynamicmemory[15,17]. Theobjecttypedistributionisahistogramofthetypesofallob-
AlthoughACDC-JSdoesnotaimatmodelingdynamicfeaturesof jectsallocatedineachworkload.Theobjecttypeisprovidedinthe
JavaScript we rely on the conclusion that they are rarely used in heapsnapshots.Noadditionalprocessingofthesnapshotdatawas
realapplicationsandthereforedonotemploysuchfeaturesinour necessary. Figure 1 illustrates the object type distribution for all
implementation with the additional benefit of avoiding their side workloads.Notethattheheapsnapshotsalsocontaininformation
effectsonACDC-JS’sexecution. specifictotheV8virtualmachineandtheDevToolsheapsnapshot-
Theapproachofmodel-basedperformanceanalysisisbasedon tingmechanism,namelyhiddenclassesthatdescribeobjectprop-
ACDC[11],heredenotedACDC-C,whereempiricaldataonprop- ertiesinV8,andso-calledsyntheticnodesthatrepresentGCroots.
ertiesofrealapplicationsallowstheimplementationofaworkload We disregard such information here. The type distribution in our
68
1
SCcTeEencsihSNtnopeePN.nnNEco.comcomionsmotm.icstom URRRESocnueeesieaaareRordddnepcisssseneuttt,aaatl&rerrrtetttsrapppataedaaacncgggtdfihieeeornrsnnnneotaeeelaodwwwrgtssssiyi,,,c,tssslerewww.eoaiiitttdfcccThhhfiortttpsoootEccNaaaurAtttreeiSocggCplooeeArr.yyRSt,ocrliiecsk. mount of objects larger than size x 000...468
Hhoottmmaaiill.com Sdeiglenteini,t,cahnedcksiignnbooxu,t.sendemail,readanemail, relative a 0.2
Gmail Signin,checkinbox,sendemail,readanemail,
0
www.gmail.com deleteit,andsignout. 0 32 64 96 128 160 192 224 256
size in byte
BingSearch SearchforNewYorkandlookat array object closure number
bing.com resultingimagesandnews. string code regexp
GoogleSearch SearchforNewYorkandlookat Figure2:Sizedistributionofrealwebapplications.
google.com resultingimagesandnews.
Facebook Loginandpostamessage.
facebook.com
analysisisdominatedbyarrays,strings,anduser-definedobjects.
Google+ Loginandpostamessage. TheanalysisofobjecttypedistributionintheworkofLivshitset
plus.google.com
al.[13]appearstorecordlessarraysthanwedoinouranalysis.The
BingMaps SearchfordirectionsfromAustinto discrepancyiscausedbygrowingarrays.Wheneveranarraygrows
maps.bing.com Houstonbycarandwalk. inawaythatrequirescopyingtheoriginalcontent,wetreatthenew
arrayasalogicallynewheapobject.This,ofcourse,increasesthe
GoogleMaps SearchfordirectionsfromAustinto
maps.google.com Houstonbycarandwalk. numberofallocatedarraysandshortenstheiraveragelifetime,but
webelievethatfromthepointofviewofthememorymanagement
amazon SearchbookQuantitativeComputer
systemacopiedormovedarrayisinfactanewobjectbecausethe
amazon.com Architecture,addtoshoppingcart,lookatcart.
oldarraycanbereclaimedbytheGC.Thispolicyisalsoreflected
eBay Searchforthebook intheresultsonthelifetimeofarraysbelow.
www.ebay.co.uk QuantitativeComputerArchitecture. Objects in the heap snapshots have a unique identifier. More-
over,theheapsnapshotsalsocontainthesizeofeachobject.Hence
Table1:Userinteractionsperformedfortheheapanalysisofreal weareabletocountthenumberofobjectsofacertainsizeofall
webapplications.TheselectionisbasedontheworkofLivshitset snapshotsofallworkloads.TheresultsareillustratedinFigure2.
al.[13]. Onthexaxiswehavethesizeinbytesandontheyaxistherelative
amountofobjectsthatarelargerthanthesizeonthexaxis.Since
thesizeheavilydependsontheobjecttype,wegivethesizedistri-
butionforeachtypeseparately.Theresultsaresimilartothesize
distributioninallocationintensiveCprograms[18]inthesensethat
small objects occur very frequently whereas the amount of large
objectsissmall.Thefixedsizesfornumbers,regularexpressions
1 array (regexp),andclosuresareV8specificandwillnotbeconsideredin
string
object ourmodel.
code
closure Forouranalysisofobjectlifetime,wecountthenumberofsub-
0.8 regexp
relative amount of object types 00..46 number sAljsatieeenlmsclrqaotaupiactleseagihcnfdtieeoovttbdntesisymsnnbetayteooqhptbueesdbjeshieessnowcntctirtaseneiipb,dswmsuethahthepiloeeaoeltornnnaecsdtcaaihaencmtneneuatdparolalmflbuicronjonyocegmtuaciolntrtefatttwdhthooeieifubstohrmysofntaobee4abjmscpeKjesecrorhaBctrttotyaih.stileWnniisrfedaeiewtedmthinpelmayptnrnieltefiaiisnilfieeanlgdoennrcstarauaaoltsyhtrtceevdsecd.iieuvsoaBarebdifsdys--..
0.2 TheresultsareillustratedinFigure3.Thexaxisgivesobjectlife-
timeinKBandtheyaxisgivestherelativeamountofobjectswith
alifetimelongerthanxforeachobjecttype.Comparedtothework
0
google plusfacebookbing searcheBay google searacmhazongoogle maphsotmailespn bing mapsthe economaisvterage breyaLsoivnsdhiistscuestsaeld.[a1b3o]vaer:rawyesltirveeatsghroorwteirnignaorurrayansaalsysnieswfoorbthjeecstsamree-
sultinginahigheroccurrenceofarrayswithshorterlifetime.
workload We are also interested in the structure of a typical JavaScript
heap. Therefore we analyze the number of outgoing edges of the
Figure 1: Histogram of the object type distribution for all work-
heapnodes,calledout-degree.Sinceheapsnapshotsaregraphrep-
loads,confirmstheworkofLivshitsetal.[13].
resentations we directly derive this property from the snapshots.
Figure 4 shows the out-degree distribution of the analyzed appli-
cationsforeachobjecttype.Onthexaxiswehavetheout-degree
69
1 1
relative amount of objects living longer than x 0000....2468 relative amount of objects with an in-degree greater than x 0000....2468
0 0
0 128 256 384 512 640 768 896 1024 1152 1280 0 5 10 15 20
object lifetime in allocated KB in-degree
array object closure number array object closure number
string code regexp string code regexp
Figure3:Lifetimedistributionofrealwebapplications,confirms Figure5:In-degreedistributionofrealwebapplications.
theworkofLivshitsetal.[13].
1
mount of objects with an out-degree greater than x 0000 ....12468 relative amount of objects with a root distance greater than x 0000 ....02468
relative a 0 0 5 satrrrinagy 10 ocbojedmcetin 1im5um rootc rdeloigss teu2axr0nepce n2u5mber 30 35
0 5 10 15 20
out-degree Figure6:Minimumrootdistancedistributionofrealwebapplica-
array object closure number
string code regexp tions.
Figure4:Out-degreedistributionofrealwebapplications.
4. OverviewofACDC-CandACDC-JS
Our approach to benchmarking JavaScript implementations is
based on ACDC-C [11], a multi-core scalable mutator emulation
andontheyaxistherelativeamountofobjectswithanout-degree ofallocationintensiveC/C++programsforbenchmarkingexplicit
greaterthanx.Mostarrayshavealowout-degreewhichsuggests memorymanagementsystems.Inthissectionwewillgiveabrief
that JavaScript arrays usually contain primitive types rather than overview of ACDC-C and describe our modifications and exten-
referencestootherJavaScriptobjects. sionsthatenableadetailedanalysisofthememorymanagementin
Thein-degreeofthegraphnodesisalsodirectlyderivedfrom JavaScriptvirtualmachines.
theheapsnapshotsandpresentedinthesamedimensionsastheout- ACDC-Cimplementsanallocation,objectsharing,access,and
degree.TheresultsillustratedinFigure5suggestthatmostobjects deallocation model based on the behavior of real-world applica-
arereferencedbyonlyafewotherobjects. tions.Moreover,ACDC-Callowstoconfiguremodelsforexplor-
Anotherstructuralheappropertywehaveanalyzedistheshort- ingcornercasesofmutatorbehavior.Sizeandlivenessofallocated
est distance from the GC roots to an object. The heap snapshots objectsaredeterminedbasedonrandomdistributionsthatreflecta
containspecialnodestorepresenttheGCroots.Startingfromthese behaviorwheremoresmallandshort-livingobjectsthanlargeand
nodesweperformedadepthfirstsearchtoallnodesofeachwork- long-living objects are allocated. For that, ACDC-C implements
load to obtain the minimal root distance. The results for the root a logical notion of time that advances whenever a configurable
distance distribution are illustrated in Figure 6. On the x axis we amountofmemorycalledthetimequantumhasbeenallocated.
havetheminimumrootdistanceandontheyaxiswepresentthe ACDC-Cdistinguishesobjectlivenessandlifetime.Objectlive-
relativeamountofobjectwithaminimumrootdistancegreaterthan nessisthetimebetweenallocationandlastaccesswhereasobject
xforeachobjecttype.Thissuggeststhat80%oftheobjectscanbe lifetimeisthetimebetweenallocationanddeallocation.Thetime
tracedfromtherootsinlessthan10steps. between last access and deallocation is called deallocation delay
andemulatesdeficienciesinidentifyingdeadobjects.
70
Root
ward reference String OOPbbrjjoeepccettrties RPaeyfeloreandces
for
Array 1 2 ... n
forward reference
Root
References
OPbrjoepcetrties ...
Payload
Object
Figure7:Alist-basedlifetime-size-class.Listnodessupportdiffer- ...
entobjecttypesandforwardreferencesemulaterootdistance.
Figure8:Aheap-basedlifetime-size-class.Forwardreferencesem-
ulaterootdistance.
ACDC-Ccollectsdetailedper-threadinformationonallocation,
access,anddeallocationperformance.Foreachallocationanddeal-
location it counts the number of CPU cycles spent in the malloc classes.Similartotree-basedlifetime-size-classesinACDC-C,the
andfreefunctionsoftheallocator.Aftereachnewlyallocatedtime accessorderofelementsinheap-basedlifetime-size-classesisdif-
quantumitalsocountstheCPUcyclesspentinaccessingthelive ferent from the allocation order because shortcuts and cycles are
objects.Thisallowstonotonlycomparetheallocationanddeal- randomlyadded.Asaconsequence,wepreservetheconfigurabil-
locationperformanceofallocatorsandtheirscalabilitytomultiple ityofallocation-orderaccessversusout-of-orderaccessbutarealso
threadsbutalsoindicatesthequalityofthememorylayoutcreated able to emulate complex heap structures that may occur in real-
bytheallocators. world applications. Figure 7 and 8 illustrate the structure of the
Theexperiments[11]suggestthatACDC-Cisindeedcapable lifetime-size-classesimplementedinACDC-JS.
ofexposingdifferencesinallrelevantperformancedimensions. Finally,JavaScriptdoesnotofferaportablewaytogatherper-
WhilebothC/C++andJavaScriptaregeneralpurposeprogram- formancemetricsatthesamelevelofgranularitythanC/C++.We
minglanguages,theydifferinmanyaspectsandusuallyservedif- discussthelimitationsindetailinSection5.1.
ferentpurposes.Theremainderofthissectiondiscussestheirsim-
ilarities and differences as well as the resulting modifications of 5. ImplementationDetailsofACDC-JS
ACDC-CthatleadtotheallocationmodelofACDC-JS.
Wereusethemodelfortypicalobjectsizeandlivenessdistributions
Our analysis of real-world JavaScript programs and related
fromACDC-C[11]sincetheJavaScriptheapanalysissuggeststhat
work [13] suggests that the distribution of object size and live-
thesizeandlivenessdistributionsofrealapplicationsaresimilar,in
nessinJavaScriptprogramsisverysimilartoC/C++programs.We
thisregard,tothoseofC/C++programs.However,weextendthe
thereforereusethesizeandlivenessmodelofACDC-C.
modelwiththetypedistributiondiscussedinSection3.Thetypes
JavaScript has no language support for concurrency and even
dominatingtypicalJavaScriptheapsarearrays,strings,andobjects.
recent asynchronous programming models like web workers [10]
WeadapttheallocationmodelofACDC-Caccordinglytoaccount
do not support a shared state but implement a message passing
fortherelativeamountofoccurrencesofthesetypes.
paradigm.Asaconsequence,wedisregardallmulti-threadingand
For emulating realistic memory allocation at runtime ACDC-
objectsharingconceptsofACDC-C.
JS first selects an object size from a uniformly distributed, dis-
MemoryallocatorsforC/C++arenotawareoftheobjecttype
creteinterval[2r,2r+1]whererisselectedfromauniformlydis-
anallocatedchunkofmemoryissupposedtorepresentintheap-
tributed,discreteinterval[log (min.size),log (max.size)).Next,
plication.ThisisnotthecaseinJavaScriptimplementationswhere, 2 2
ACDC-JSselectsanobjectlivenessfromauniformlydistributed,
e.g.numberobjectsmightbeallocateddifferentlythanstringob-
discreteinterval[min.liveness,max.liveness]andaddstheconfig-
jects. ACDC-JS therefore needs to support allocation of different
ureddeallocationdelaytoobtainanobjectlifetime.Then,ACDC-
datatypes.Ouranalysisofreal-worldJavaScriptapplicationscom-
JS calculates the number of objects to be allocated based on the
binedwithpreviouswork[13]allowsustomodelsuchanalloca-
previously selected size and liveness with the following formula
tionbehavior.
definedin[11]:
ForACDC-JSwehavealsoextendedthestructureoftheheap
allocated by ACDC-C since our heap structure analysis suggests
numberofobjects=(log (max.size)−log (selectedsize)+1)2
thattheoriginalACDC-Cmodelisnotsufficientanymore.ACDC- 2 2
∗(max.liveness−selectedliveness+1)2
Cmodelsaverysimplerelationbetweenallocationandaccessor-
dering by gathering objects of the same size and liveness in so- Finally,ACDC-JSalsoemulatestypedistribution.Inparticular,
called lifetime-size-classes which are either lists (accesses hap- ACDC-JSrandomlyselectsanobjecttypetobeeitherastring,an
pen in allocation order) or binary trees (allocation in pre-order, array,orauser-definedobject.Theaveragetypedistributionofreal
left to right, and accesses in pre-order, right-to-left). ACDC-JS applicationsillustratedinFigure1showsaboutthesamenumber
also supports list-based lifetime-size-classes but extends the tree- of occurrences of strings and user-defined objects but about 1.5
basedlifetime-size-classestosupportmorethantwochildnodesto timesmorearrays.ACDC-JSapproximatesthistypedistributionby
emulateobjectscontainingmultiplereferences,shortcutsfromthe multiplyingthenumberofobjectswiththatfactoriftherandomly
lifetime-size-classroottocertainelementstocontrolthedistance selectedtypeisanarray.
between the objects and the GC roots, and back references from Notethatwedonotaccountfortypeswhenselectingsizeand
certainelementstoelementsathigherlevelstoemulatecyclesin livenesstokeeptheallocationmodelsimpleandbecausethesize
theobjectgraph.Wecallsuchstructuresheap-basedlifetime-size- and lifetime distributions show a similar trend for either arrays,
71
Parameter Value InJavaScriptthereisnowaytoretrieveinformationaboutmem-
timequantum 1024KB oryconsumptionfromwithinthemutator.Thereforewesamplethe
benchmarkduration 100 residentsetsizereportedbytheoperatingsystematasamplerate
min.size 8B of1millisecond.Wealsoreportthemaximumresidentsetsizeover
max.size 64B anexecutionofACDC-JS.
min.liveness 1
max.liveness 10 5.2 AccurateTimeMeasurementinJavaScript
min.rootdistance 1
The most portable time measurement facility in JavaScript is the
max.rootdistance 20
Date.now()method[2]whichissupportedbyallmajorJavaScript
min.numberofreferences 0
implementations.However,itonlyprovidesmillisecondresolution.
max.numberofreferences 5
Asaconsequence,modernbrowserssupporthigh-resolutiontime
list-basedratio determinedbytypedistribution
measurement through the performance.now() method [7], speci-
accessliveobjects TRUE
fying microsecond accuracy. Unfortunately, the stand-alone Spi-
read-onlyaccess FALSE
derMonkey shell does not provide the performance object. In or-
der to report time with the highest resolution possible, we em-
Table2:Defaultsettings
ploythePerfMeasurementobject[6]availableforSpiderMonkey
onLinux.PerfMeasurementreportsCPUcycleswhichwescaleto
microsecond-accuratetimevaluesusingtheclockspeedofourex-
perimentalenvironment.Wehavecheckedthetimingresultsofa
strings,andobjects.AmoredetailedallocationmodelofACDC-JS
simpleJavaScriptloopwithoutGCinterferencetoverifythatper-
accountingfordifferentlivenessbasedontypeinformationremains
formance.now()inV8andPerfMeasurementinSpiderMonkeypro-
futurework.
videcomparableresults.
Strings and arrays are implemented as standard JavaScript
types. User-defined objects types are implemented with a com-
mon object representation illustrated in Figure 7. Payload repre- 6. ExperimentalEvaluation
sentstherandomobjectsizeandReferencesmodelstheout-degree
All experiments ran on a desktop machine with an Intel i5-2400
(discussedbelow).
4-core 3.1 GHz processor, 32 KB L1 and 256 KB L2 data cache
ACDC-JSimplementslifetime-size-classesdifferentfromACDC-
percore,6MBsharedL3cache,8GBofmainmemory,andLinux
Cwherethelistandtreereferencesarestoredintheobject’spay-
kernel version 3.2.0. We repeated each experiment 10 times. For
load.SinceJavaScriptdoesnotallowpointerarithmetic,list-based
each metric we report the arithmetic mean and sample standard
lifetime-size-classes require separate list nodes as illustrated in
deviationoftherepetitions.
Figure7.
Thegoalofourexperimentsistodemonstratethecapabilities
Heap-basedlifetime-size-classes,illustratedinFigure8,differ
ofACDC-JSandnottoprovideaqualitativecomparisonofJava-
fromtree-basedlifetime-size-classesinACDC-Ctosupportmore
thantwochildobjects.ThesizeofReferencesischosenrandomly Scriptvirtualmachinesalthoughthedifferencesbetweenthesys-
temsundertestmightbeinterestingforVMdevelopers.Toexpose
betweentheparametersmin.numberofreferencesandmax.num-
theperformanceimpactofdifferentimplementationsandconfigu-
berofreferencesuponallocationtoapproximatetheout-degreedis-
rationswerunACDC-JSdirectlyonV8andSpiderMonkey.Both
tributionwithacommonobjectmodel.Heap-basedlifetime-size-
virtualmachinescaneasilybeconfiguredtoenableordisablecer-
classes also emulate cycles by randomly selecting a cycle length
tainGCfeatures.WealsorunACDC-JSinChromeandFirefoxto
between the parameters min. cycle length and max. cycle length.
studytheperformanceimpactoftheactualbrowsersaroundV8and
Heaplevelswithadistanceoftherandomlyselectedcyclelength
SpiderMonkey.
areconnectedthroughabackreferencetoemulatecyclesinaJava-
The experimental analysis follows a 4-step process. First, we
Scriptheap.
pickanexpectedperformancecharacteristicofaparticularmemory
Anotherextensiontobothtypesoflifetime-size-classesisem-
management feature. Second, we define a workload that exposes
ulatingtherootdistancedistributionbyrandomlyselectingaroot
thisverycharacteristicandstillcomplieswithtypicalJSmemory
distancebetweentheparametersmin.rootdistanceandmax.root
usage found in our heap analysis in Section 3. The third step is
distance.Therootdistancedistributionisapproximatedbyforward
to configure ACDC-JS to emulate the workload. The fourth and
referencestonodeshavingarootdistanceofamultipleoftheran-
finalstepistomeasuretheperformancequantitiesinactualrunson
domlyselectedrootdistance.
theJavaScriptVMsandcomparetheresultswithourexpectations.
ThedefaultparametersofACDC-JSarelistedinTable2.For
Contradictoryresultsindicatepotentialperformancebugs.
allexperimentsdescribedinSection6weusethesesettingsunless
WedemonstrateACDC-JS’scapabilitiesonthreeperformance
statedotherwise.
characteristics: (1) memory management throughput for increas-
ingobjectliveness,(2)memorymanagementlatencyforincreasing
5.1 MetricsandProbes
heapsize,and(3)memorymanagementrobustnessforincreasing
Alongwithtotalscriptexecutiontime,ACDC-JSalsoreportstem- objectsizes.Thethroughputexperimentrevealsspeedupcharacter-
poral metrics in average allocation time and average memory ac- isticsandpromotionthresholdsofgenerationalGCsandalsoillus-
cess time. The data is collected from within ACDC-JS. We keep tratesthethroughput,latency,andmemoryconsumptiontrade-off
track of the time it takes to allocate each time quantum and—in with incremental marking. The latency experiment follows up on
caseACDC-JSisconfiguredtoaccessmemory—thetimerequired thattrade-offandprovidesanin-depthanalysiswhichalsoreveals
totraverse(andoptionallyalsowrite)alllivelifetime-size-classes inturnthecostofwritebarrierswithgenerationalGCs.Therobust-
after a time quantum has been allocated. For both the allocation nessexperimentcheckswhetherthroughput,latency,andmemory
timeandaccesstimesamples,wereportthearithmeticmeanand consumptionarecontinuousinobjectsize.Thisexperimentreveals
thesamplestandarddeviation.Theallocationandaccesstimejitter apotentialperformancebuginV8.Finally,wealsoprovideanex-
isalsoreportedasthecoefficientofvariation,i.e.,standarddevia- perimentthatdoesnotintendtorevealperformancedifferencesin
tiondividedbyarithmeticmean. VMsbutinsteadverifiesthatACDC-JS’sallocationbehaviorisin-
72
deed comparable to the object size and lifetime distributions ob- Parameter Value
tainedfromthe13realwebapplicationsinSection3. max.size 8B
We compiled and executed the JavaScript virtual machines in min.liveness increasingfrom1to20
the following configurations and labeled the graphs accordingly: max.liveness min.liveness
V8 can be configured through start-up flags so all V8 results are timequantum 64KB
obtained from the same build using the default compilation set- benchmarkduration 200
tings.V8representsthedefaultconfigurationofV8,i.e.,agener- accessliveobjects FALSE
ationalGCwithincrementalmarkingintheoldgenerationanda
semi-spacecollectorinthenewgeneration.V8NrunsV8without Table3:Non-defaultACDC-JSsettingsforthethroughputexperi-
incrementalmarking(flag:–noincremental-marking)andV8Eruns ment.
V8inamodethateagerlycompactstheoldgenerationaftereach
GCrun(flag–always-compactand–noincremental-marking).Note
thatV8EisratheradebuggingconfigurationofV8thanaproduc-
25
tionsetting.Weanywayincludeitinourevaluationtoincreasethe
setofdifferentGCpolicies.AllV8configurationsareexecutedin
tdWSttd–hhoeieceirsernrs.aMiSadSpbFdbp8Mtloaeleinedsfsb-Nat-hkgeeeuerecYnxrMllyitculeanhntccovcb?mtosrneu”-enrkrSaimsofile[rpid1okyogeis]tdsnuni,renrtse3oaaarhgqM.nlta2u)ifaor4iroapenran.nneo2dm,dsktp8hieeu–..cSeewyleoM.an,mowcraaorGipkbvnmtihlielerfipoennto-iuugaclrtateibcrmlaeiglnetumeemisctmnoerpaaeenmeacrmgtarhaaeasteiltienmninotmceettneariasatanal2lrlrltg.iym)kizso.S-aanrsNMtuurwaiksonoleeinntGnrede.iegpnCpAtag(hrctlfl(eaolflJsat“laSaegAlvtegnp:hrcasti–ees---: average allocation time in milliseconds (lower is better) 112 5050
non-default features of SpiderMonkey might also be still experi-
mental but since we are not interested in a qualitative compari-
son of JavaScript VMs, we include these settings to demonstrate 0
0 5 10 15 20
ACDC-JS’scapabilitiesonvariousGCpolicies.Furthermore,SM max liveness
apparently employs a policy where incremental marking is per- V8 V8E SMN Firefox
V8N SM SMG Chrome
formed only through the event loop effectively shifting the prob-
lemofpausetimestowhereitcannotbetriggeredbyACDC-JS. Figure9:Averageallocationtimeforanincreasingobjectliveness.
AllSpiderMonkeyconfigurationsareexecutedinthejsshellver-
sionJavaScript-C27.0.
For the configurations of Chrome and Firefox we executed nocrankshaft) where we observed different absolute performance
ACDC-JS in the browser embedded in a simple HTML page in- characteristics but the same relative differences in V8, V8N, and
cludedintheACDC-JSframework.Weexpectsimilarresultsfor V8E.Additionally,wehaverunallexperimentsinV8’spredictable
Chrome compared to V8 and for Firefox compared to Spider- mode(d8flag–predictable)andcomparedtheresultstoV8,V8N,
Monkey. We include the results to also visualize possible perfor- and V8E running in default mode. Here we did not observe any
mance impacts of embedding the JavaScript virtual machines in notabledifferencesintheresultsinanyperformancedimension.
thebrowsers.Notethatsomeresultsmightdifferfromthebarema-
chineresultssinceweranthelateststablebrowserversionswhich 6.1 CapabilitiesofACDC-JS:Throughput
areshippedwithdifferentJavaScriptengineversions.Anotherrea-
In this experiment we demonstrate ACDC-JS’s capability to pro-
son for possible differences in performance is that browsers may
vide detailed allocation throughput information for generational
call the garbage collector explicitly and that collection policies
garbagecollection.Generationalgarbagecollectionexploitsappli-
might be different in the shell than when running in a browser.
cationpredispositionforallocatingshort-livingobjects(younggen-
Furthermore,thememoryconsumptionofthebrowserscontainthe
eration)attheexpenseofdelayingthereclamationoflong-living
whole browser footprint and not only the JavaScript virtual ma-
objects (old generation). We expect higher allocation throughput
chine.WepresentresultsforChromeversion32.0.1700.17shipped
forshort-livingobjectswhichisequivalenttolowerallocationtime
with V8 version 3.22.24.17 and for Firefox version 27.0 shipped
sinceACDC-JSrunningontheJavaScriptVMsrepresentsaclosed
withSpiderMonkeyversion24.2.
system.
JavaScriptVMsmayintroducenon-determinismthroughadap-
We select a workload that gradually increases the liveness of
tivetechniqueslikeconcurrentcodeoptimizations.ForallVMcon-
the objects from one time quantum (64 KB) to 20 time quanta
figurationsweneverthelessdonotdisablefeaturesthatmightadd
(1280KB).ThesesettingsreflectourresultsfromSection3where
non-determinismtotheexecutionofACDC-JSsinceweareinter-
50%ofallobjectsliveshorterthan64KBandvirtuallyallobjects
ested in VM behavior in production environments. However, we
liveshorterthan1280KB.ACDC-JS’sbenchmarkdurationisset
address non-determinism statistically by replicating experiments
to 200, i.e., 200 time quanta will be allocated before ACDC-JS
10 times and reporting the sample standard deviations. We have
terminates. This ensures that even for a liveness of 20 (where
observedthatahighernumberofreplicationshasonlynegligible
ACDC-JS reaches a steady state after 20 time quanta) ACDC-
impactonthesamplestandarddeviation.Wethereforebelievethat
JS executes in a steady state for 90% of the benchmark duration
foroursetup10replicationsyieldameaningfulestimateoftheac-
providing data unbiased by a warm-up phase. The object size is
tual deviation. Furthermore, we have run all experiments with a
fixed to 8 bytes to isolate the impact of object liveness. Table 3
V8configurationthatdisablestheoptimizingcompiler(d8flag–
gives an overview of the non-default ACDC-JS settings for this
experiment.
2SpiderMonkey./configure flags: –enable-optimize –disable-debug – Theallocationperformanceforincreasingobjectlivenessispre-
enable-threadsafe–with-system-nspr sented in Figure 9 where the y axis shows the average allocation
73
500 100
450 90
400 80
B
maximum resident set size in M (lower is better) 122335050500000 average allocation time jitter (lower is better) 3456700000
100 20
50 10
0 0
0 5 10 15 20 0 5 10 15 20
max liveness max liveness
V8 V8E SMN Firefox V8 V8E SMN Firefox
V8N SM SMG Chrome V8N SM SMG Chrome
Figure 10: Maximum resident set size for an increasing object Figure11:Averageallocationtimejitter(coefficientofvariation)
liveness. foranincreasingobjectliveness.
Parameter Value
timepertimequantuminmilliseconds.Notethatlowerallocation
timequantum 512KB
time implies higher allocation throughput in our setup. The gen-
deallocationdelay increasingfrom10to100
erational garbage collection implemented in V8, V8N, V8E, and
benchmarkduration 200
Chrome provides a significant speedup for allocating short-living
accessliveobjects TRUE
objects.ThissuggeststhatACDC-JSisindeedcapableofcreating
a workload that exposes the performance impact of generational
Table4:Non-defaultACDC-JSsettingsforthelatencyexperiment.
garbagecollection.TheSpiderMonkeyconfigurations(SM,SMN,
andFirefox)donotenablegenerationalgarbagecollectionandthus
shownearlyconstantallocationperformanceforallobjectliveness
realapplicationsbenefitfromfasterallocationofsmallobjectswith
settings. Although SMG enables generational garbage collection
ahighprobability.Wealsoconcludethatforlong-livingobjectsthe
we do not observe better allocation performance for short-living
tracingpolicyoftheoldgenerationaffectsallocationperformance.
objects.Weaccountthistothefactthatatthemomentofwriting,
Wewillisolatethisimpactinthenextsection.
thegenerationgarbagecollectionfeatureofSpiderMonkeyisstill
experimental.
6.2 CapabilitiesofACDC-JS:Latency
Theallocationtimeresultsforthisexperimentillustrateanother
GCfeaturewhichisorthogonaltogenerationalgarbagecollection, Animportantperformancecharacteristicmostlyignoredbystate-
namelyincrementalmarkingoftheoldgeneration.V8Nperforms of-the-artJavaScriptbenchmarkingsuitesismemorymanagement
slightly better for long-living objects than the default V8 setting, latency.Incrementalmarkingisanapproachtoreducelatency,e.g.,
exceptforonedatapoint.SincebothV8andV8Nimplementthe pause times caused by the GC, at the expense of higher memory
samegenerationalGCpolicy,theperformancedifferencemaybe consumptionduetodelayedcollection.Inthisexperimentwecon-
explainedbytheoverheadofincrementalmarkingtheoldgenera- figureACDC-JStoexposetheperformanceimpactandtrade-offs
tion.WewilldiscussthisfeatureindetailinSection6.2. introducedbyincrementalmarking.
Figure 10 shows on the y axis the maximum resident set size Inordertoisolatetheimpactofincrementalmarkingwecon-
in MB. For all VMs longer object liveness increases the size of figure ACDC-JS to increase the load exclusively on the marking
theheapsincethetimequantumiskeptconstant.SM,SMN,and phase. We achieve this by emulating a reachable memory leak
SMG show constant memory consumption for object liveness up (acrossdatapoints).Inparticular,weincreasethesizeoftheob-
tofivewhichsuggeststhatforthisrangeSpiderMonkeyallocates jectgraphthatmustbetracedbytheGCwhilekeepingtheimpact
morememoryfromtheoperatingsystemthanitactuallyneeds.The ofgenerationalcollectionconstant(eachobjectispromotedtothe
differences in the heap size of different VMs also grow with the oldgenerationonlyonce).ACDC-JSimplementsaso-calleddeal-
livenessoftheobjectswhereweobservethatincrementalmarking locationdelaytoemulatesuchamutatorbehavior.Adeallocation
of the old generation in V8 causes additional memory overhead delay of x delays the collection of objects by x time quanta. The
comparedtoV8N. non-defaultACDC-JSparametersforthisexperimentarelistedin
Figure11showsontheyaxistheaverageallocationtimejitter Table4.
(coefficient of variation) for this experiment. While SM, SMN, Figure12showsontheyaxistheaverageallocationtimeper
and SMG only show a slightly increasing allocation time jitter, timequantumforthisexperiment.Thegrowingamountofreach-
for V8N and V8E the allocation time jitter nearly doubles. This ablegarbageonlyincreasestheallocationtimeslightlybecausethe
suggests that moving objects from the new generation to the old amount of newly allocated objects is constant per time quantum.
generation adds to the latency of tracing the old generation as Nevertheless,theabsolutevaluesofthebestperformingconfigura-
well.V8,implementingincrementalmarkingoftheoldgeneration, tionSMNareonlyhalfoftheslowestconfigurationV8E.However,
maintainsconstantlatencyforlong-livingobjects. thefocusofourevaluationisonexposingtheperformanceimpact
WedemonstrateinthisexperimentthatACDC-JSiscapableof ofcertainVMimplementationdetailsandnotonaqualitativecom-
exposing the performance impact of generational garbage collec- parison of different VMs although such a comparison is possible
tion.Togetherwiththeresultsofourheapanalysisweconcludethat withACDC-JSaswellandmightbeinterestingtoVMdevelopers.
74
25 1200
1000
average allocation time in milliseconds (lower is better) 112 5050 maximum resident set size in MB (lower is better) 468000000
200
0 0
0 20 40 60 80 100 0 20 40 60 80 100
deallocation delay deallocation delay
V8 V8E SMN Firefox V8 V8E SMN Firefox
V8N SM SMG Chrome V8N SM SMG Chrome
Figure12:Averageallocationtimeforanincreasingdeallocation Figure14:Maximumresidentsetsizeforanincreasingdealloca-
delay. tiondelay.
350 40
average allocation time jitter (lower is better) 11223 505050000000 average memory access time in milliseconds (lower is better) 112233 5050505
0 0
0 20 40 60 80 100 0 20 40 60 80 100
deallocation delay deallocation delay
V8 V8E SMN Firefox V8 V8E SMN Firefox
V8N SM SMG Chrome V8N SM SMG Chrome
Figure13:Averageallocationtimejitter(coefficientofvariation) Figure15:Averagememoryaccesstimeforanincreasingdeallo-
foranincreasingdeallocationdelay. cationdelay.
Thekeytrade-offwithincrementalmarkingisallocationlatency uredACDC-JStoalsoaccesstheliveobjectsaftereachtimequan-
versusmemoryconsumption.Theaverageallocationtimejitter(co- tum.Thisspecificallyexcludestheobjectsofthereachablememory
efficientofvariation)reflectingthevariationofallocationlatencyis leak.Asaconsequence,theamountofliveobjectsisconstantfor
presentedontheyaxisofFigure13.Heretheincrementalmarking all deallocation delay settings while the ratio of live objects ver-
policyofV8andChromeshowlow,constantallocationtimejitter. susallheapobjectsdecreaseswithanincreasingdeallocationde-
Forallotherconfigurationstheallocationtimejitterincreaseswith lay.Thechangeinratiomaycausetheinterestingcharacteristicof
anincreasingdeallocationdelaycausedbylongerGCpausetimes fastermemoryaccessforlargerheaps.Alsonotetheabsolutedif-
duetotracing.TheincrementalmarkingimplementedinSMshows ferenceinmemoryaccessperformanceforSM,SMN,andFirefox
noimpactbecauseitsintegrationpolicyrequireseventloopactions compared to the other configurations. These configurations allow
totriggerincrementalmarking. muchfastermemoryaccessbecausetheydonotimplementgener-
Themaximumresidentsetsizeforthisexperimentisshownon ationalgarbagecollectionandthereforedonotrequirereadorwrite
theyaxisofFigure14.Thegrowingamountofreachablegarbage barrierstodeterminethegenerationofanobject.
causes higher memory consumption for all configurations. How- Figure 16 shows on the y axis the average memory access
ever,V8NconsumeslessmemorythanV8becauseV8Nreclaims time jitter (coefficient of variation). Here we observe increasing
garbage earlier than with incremental marking in V8. This illus- allocation time jitter at comparable levels for all configurations.
tratesthetrade-offofallocationlatencyversusmemoryconsump- Thissuggeststhatthegrowingaccesstimejitterisnotcausedby
tionwithincrementalmarking.SM,SMN,andSMGshowconstant VM implementation details but rather by architectural properties
memoryconsumptionforadeallocationdelaylargerthan60.This ofourexperimentalsetup,e.g.,thecachearchitecture.
suggests that for smaller deallocation delays they allocate more This experiment demonstrates ACDC-JS’s capabilities of ex-
memoryfromtheoperatingsystemthanactuallyrequired. posingtheperformanceimpactofincrementalmarking.Thebenefit
Anotherinterestingeffectcanbeobservedforthememoryac- ofincrementalmarkingdependsontherequirementsoftheappli-
cess time illustrated on the y axis of Figure 15. We have config- cation.Non-incrementalmarkingyieldslowermemoryfootprints
75
140 250
120
average memory access time jitter (lower is better) 1 46800000 average allocation time in milliseconds (lower is better) 112 50500000
20
0 0
0 20 40 60 80 100 0 50 100 150 200 250
deallocation delay max size
V8 V8E SMN Firefox V8 V8E SMN Firefox
V8N SM SMG Chrome V8N SM SMG Chrome
Figure16:Averageaccesstimejitter(coefficientofvariation)for Figure17:Averageallocationtimeforanincreasingobjectsize.
anincreasingdeallocationdelay.
50
Parameter Value
45
min.size increasingfrom8to256B
max.size min.size 40
Table5:mtbaiNcemacnoxeecn.sh-qsldmiuvleaiaefvnranektueuolsdmtbsuAjreaCcttiDsoCn-JS16F5sA40etLKtiSBnEg*smfoirnt.hseizreobustnessexperi- average allocation time jitter (lower is better) 1223350505
ment.
10
5
(significantonembeddeddevices)whileincrementalmarkingen- 0
ableslowpausetimesforsoftreal-timeapplications.Inadditionto 0 50 100 150 200 250
max size
theimpactofincrementalmarking,theaccesstimeresultsinthis V8 V8E SMN Firefox
experimentalsoexposethecostsofgenerationgarbagecollection. V8N SM SMG Chrome
Fromthatweconcludethatitisimportanttoexploreboth,through-
Figure18:Averageallocationtimejitter(coefficientofvariation)
putandlatencyworkloadsinallrelevantperformancedimensions
foranincreasingobjectsize.
tocoverallrelevanttrade-offsinJavaScriptVMs.
6.3 CapabilitiesofACDC-JS:Robustness
unexpectedbehavior.ThisdemonstratesACDC-JS’scapabilitiesof
Thisexperimentisintendedtoexposetheimpactofobjectsizeon exposingperformanceanomaliesforcertainworkloadsbyexplor-
allocationperformance.Althoughourheapanalysissuggeststhat ingalargesetofworkloadconfigurations.Furthermore,ACDC-JS
smallobjectsoccurmorefrequentlyitisimportantforVMstobe canalsobeusedtodebugunexpectedperformanceresults.Inthe
robust against a large variety of object sizes. With robustness we following,wewillgiveanexampleofdiggingdeeperintothedata
meanthatcontinuousvariationofaworkloadcharacteristicyields collectedbyACDC-JS.
continuousresponseofallperformancemetrics. The maximum resident set size for this experiment is shown
WeconfigureACDC-JStoallocateaconstantnumberofshort- inFigure19.Again,whilemostsettingsforthemin.sizeyielda
living objects of an increasing size starting at 8 bytes up to 256 continuous change in response, this workload triggers outliers in
bytes to reflect the results of our heap analysis. The number of memoryconsumptionforthesameworkloadparametersasforthe
allocatedobjectsiscontrolledtobeconstantbyincreasingthetime allocationtimejitter.Thissuggestsarelationbetweenthediscon-
quantum linearly to the object size parameters. The non-default tinuousbehavioroftheallocationtimejitterandthememorycon-
ACDC-JSsettingsarelistedinTable5. sumption.However,sincethemaximumresidentsetsizeisanag-
Figure17showstheaverageallocationtimeforthisexperiment. gregation of memory consumption over time, we have to inspect
All VMs show similar behavior, although at different absolute thetracesoftheresidentsetsize(createdbyACDC-JSonthefly)
values. The allocation time tends to increase with an increasing forobjectsizesthatyieldoutliersinmemoryconsumption.
objectsizesuggestingrobustbehaviorforallobjectsizes. Figure20showsatraceoftheresidentsetconsumedbyV8E
The allocation time jitter (coefficient of variation) illustrated for a min. size of 200 bytes and 232 bytes, respectively, because
inFigure18,ontheotherhand,showssignificantfluctuationsfor thesesettingsshowlargedifferencesintheallocationtimejitterof
someVMs.SM,SMN,andSMGshowcontinuousbehaviorofal- V8E in Figure 18 as well as in the maximum resident set size in
locationtimejitterwhereSMandSMNevenshowanimprovement Figure19.Thexaxisgivesthetimestampoftheresidentsetsize
forlargerobjectsizes.RunningthissettingofACDC-JSonV8and sampletakenduringtheexecutionofACDC-JS.Asaconsequence,
V8Ecausessignificantoutliersforsomeobjectsizeswhichisan thelineforamin.sizeof200bytesisshorterbecausetheexecu-
76