Performance Testing of Data Delivery Techniques for Ajax Applications

34 Pages·2008·1.77 MB·English

Delft University of Technology
Software Engineering Research Group
Technical Report Series
Performance Testing of Data Delivery
Techniques for A Applications
JAX
Engin Bozdag, Ali Mesbah and Arie van Deursen
Report TUD-SERG-2008-009
SERG
TUD-SERG-2008-009
Published,producedanddistributedby:
SoftwareEngineeringResearchGroup
DepartmentofSoftwareTechnology
FacultyofElectricalEngineering,MathematicsandComputerScience
DelftUniversityofTechnology
Mekelweg4
2628CDDelft
TheNetherlands
ISSN1872-5392
SoftwareEngineeringResearchGroupTechnicalReports:
http://www.se.ewi.tudelft.nl/techreports/
FormoreinformationabouttheSoftwareEngineeringResearchGroup:
http://www.se.ewi.tudelft.nl/
Note: Thispaperisapre-printof:
EnginBozdag,AliMesbah,andArievanDeursen. PerformanceTestingofDataDeliveryTechniques
forAJAXApplications. JournalofWebEngineering(JWE),2008,Toappear.
(cid:13)c copyright 2008, by the authors of this report. Software Engineering Research Group, Department of
Software Technology, Faculty of Electrical Engineering, Mathematics and Computer Science, Delft Uni-
versityofTechnology. Allrightsreserved. Nopartofthisseriesmaybereproducedinanyformorbyany
meanswithoutpriorwrittenpermissionoftheauthors.
JournalofWebEngineering,Vol.0,No.0(2008)000–000
SERG (cid:13)c RintonPress Bozdagetal.–PerformanceTestingofDataDeliveryTechniquesforAJAXapplications
PERFORMANCETESTINGOFDATADELIVERYTECHNIQUES
FORAJAXAPPLICATIONS
EnginBozdag
SoftwareEngineeringResearchGroup,DelftUniversityofTechnology
Mekelweg4,2628CDDelft,TheNetherlands
[email protected]
AliMesbah
SoftwareEngineeringResearchGroup,DelftUniversityofTechnology
Mekelweg4,2628CDDelft,TheNetherlands
[email protected]
ArievanDeursen
SoftwareEngineeringResearchGroup,DelftUniversityofTechnology&CWI
Mekelweg4,2628CDDelft,TheNetherlands
[email protected]
Received(receiveddate)
Revised(reviseddate)
Abstract
AJAX applications are designed to have high user interactivity andlow
user-perceivedlatency.Real-timedynamicwebdatasuchasnewsheadlines,
stocktickers,andauctionupdatesneedtobepropagatedtotheusersassoon
aspossible. However, AJAX stillsuffersfromthelimitationsoftheWeb’s
request/responsearchitecturewhichpreventsserversfrompushingreal-time
dynamicwebdata. Suchapplicationsusuallyuseapullstyletoobtainthe
latestupdates,wheretheclientactivelyrequeststhechangesbasedonapre-
definedinterval.Itispossibletoovercomethislimitationbyadoptingapush
styleofinteractionwheretheserverbroadcastsdatawhenachangeoccurs
ontheserverside. Boththeseoptionshavetheirowntrade-offs. Thispaper
firstintroducesthecharacteristicsofbothpullandpushapproaches. Itthen
presents the design and implementation of our distributed test framework,
calledCHIRON,wheredifferentAJAXapplicationsbasedoneachapproach
canbeautomaticallytestedon.Finally,wepresentanddiscusstheresultsof
ourempiricalstudycomparingdifferentweb-baseddatadeliveryapproaches.
Keywords:Performancetesting,Ajax,Webdatadelivery,Comet,Push/pull,Empiricalstudy
Communicatedby:tobefilledbytheEditorial
1 Introduction
Recently, there has been a shift in the direction of web development. A new breed of web applica-
tion,dubbed AJAX (AsynchronousJavaScriptandXML)[14]isemerginginresponsetothelimited
1
TUD-SERG-2008-009 1
Bozdagetal.–PerformanceTestingofDataDeliveryTechniquesforAJAXapplications SERG
2 PerformanceTestingofDataDeliveryTechniquesforAJAXApplications
degreeofinteractivityinlarge-grainstatelessWebinteractions. Theintentistomakewebpagesfeel
moreresponsivebyexchangingsmallamountsofdatawiththeserverbehindthescenesandmaking
changestoindividualuserinterfacecomponents. Thisway, theentirewebpagedoesnothavetobe
reloadedeachtimetheusermakesachange. AJAX isaseriousoptionnotonlyfornewlydeveloped
applications,butalsoforexistingwebsitesiftheiruserfriendlinessisinadequate[21,22,20,23].
Theterm AJAX spreadrapidlyfromaWeblogtotheWallStreetJournalwithinweeks. Thenew
web applications under the AJAX banner have redefined end users’ expectations of what is possible
withinaWebbrowser. However,AJAXstillsuffersfromthelimitationsoftheWeb’srequest/response
architecture. The classical model of the web called REST [12] requires all communication between
the browser and the server to be initiated by the client, i.e., the end user clicks on a button or link
andtherebyrequestsanewpagefromtheserver. No‘permanent’connectionisestablishedbetween
client/serverandtheserverisrequiredtomaintainnostateinformationabouttheclients. This“pull”
schemehelpsscalability[12], butprecludesserversfromsendingasynchronousnotifications. There
aremanyusecaseswhereitisimportanttoupdatetheclientuserinterfaceinresponsetoserver-side
changes. Forexample:
• Anauctionwebsite,wheretheusersneedtobeavertedthatanotherbidderhasmadeahigher
bid. Figure 1 shows a screen-shot taken from eBay. In a site such as eBay, the user has to
continuously press the ‘refresh’ button of his or her browser, to see if somebody has made a
higherbid.
• Astockticker, wherestockpricesarefrequentlyupdated. Figure2showsascreen-shottaken
fromMSN’sMoneyCentralsitea. Therightcolumncontainsastockticker. Thesitecurrently
usesapull-basedmechanismtoupdatethestockdata.
• Achatapplication,wherenewsentmessagesaredeliveredtoallthesubscribers.
• A news portal, where news items are pushed to the subscriber’s browser when they are pub-
lished.
Today, suchwebapplicationsrequiringreal-timeeventnotificationanddatadeliveryareusually
implementedusingapullstyle,wheretheclientcomponentactivelyrequeststhestatechangesusing
client-side timeouts. An alternative to this approach is the push-based style, where the clients sub-
scribe to their topic of interest, and the server publishes the changes to the clients asynchronously
everytimeitsstatechanges.
However, implementing such a push solution for web applications is not trivial, mainly due to
the limitations of the HTTP protocol. It is generally accepted that a push solution that keeps an
open connection for all clients will cause scalability problems. However, as far as we know, no
empiricalstudyhasbeenconductedintotheactualtrade-offsinvolvedinapplyingapush-versuspull-
based approach to browser-based or AJAX applications. Such a study will answer questions about
data coherence, scalability, network usage and latency. It will also allow web engineers to make
rationaldecisionsconcerningkeyparameterssuchaspublishandpullintervals,inrelationto,e.g.,the
anticipatednumberofclients.
Inthispaper,whichisanextensionofourpreviouswork[5],wefocusonthefollowingchallenges:
a http://moneycentral.msn.com
2 TUD-SERG-2008-009
SERG Bozdagetal.–PerformanceTestingofDataDeliveryTechniquesforAJAXapplications
E.Bozdag,A.Mesbah,andA.vanDeursen 3
Fig.1.AscreenshottakenfromeBay.Theuserhastoconstantlyclickthe“Refresh”buttontoseeanyupdates.
• Howcanwesetupanautomated,controllable,repeatable,anddistributedtestenvironment,so
that we can obtain empirical data with high accuracy for comparing data delivery approaches
forAJAXapplications?
• How does a push-based web data delivery approach compare to a pull-based one, in terms of
datacoherence,scalability,networkperformance,andreliability?
Thispaperisfurtherorganizedasfollows.
Westartout,inSection2,byexploringcurrenttechniquesforreal-timeHTTP-baseddatadelivery
ontheweb. Subsequently,inSection3,wediscussthepush-based BAYEUX protocolandthe DWR
library,thetwoopensourcepushimplementationsthatwewilluseinourexperiments. InSection4,
wepresenttheexperimentaldesignbyarticulatingourresearchquestionsandoutliningtheproposed
approach. The independent and dependent variables of our experiment are also discussed in this
section. A detailed presentation of our distributed testing framework called CHIRONbas well as the
environment and applications that we use to conduct our experiments, is shown in Section 5. In
Section 6 the results of our empirical study involving push and pull data delivery techniques are
covered, followed by a discussion of the findings of the study and threats to validity in Section 7.
Finally, in Section 8, we survey related work on this area, after which we conclude our paper in
Section9withasummaryofourkeycontributionsandsuggestionsforfuturework.
bInGreekmythology,CHIRON,wastheonlyimmortalcentaur.Hebecamethetutorforanumberofheroes,includingAJAX.
TUD-SERG-2008-009 3
Bozdagetal.–PerformanceTestingofDataDeliveryTechniquesforAJAXapplications SERG
4 PerformanceTestingofDataDeliveryTechniquesforAJAXApplications
Fig.2.StocktickerfromMSNMoneyCentral
2 Web-basedReal-timeEventNotification
2.1 AJAX
AJAX[14]isanapproachtowebapplicationdevelopmentutilizingacombinationofestablishedweb
technologies: standards-basedpresentationusingXHTMLandCSS,dynamicdisplayandinteraction
using the Document Object Model, data interchange and manipulation, asynchronous data retrieval
using XMLHttpRequest, and JavaScript binding everything together. XMLHttpRequest is an API
implementedbymostmodernwebbrowserscriptingenginestotransferdatatoandfromawebserver
using HTTP, by establishing an independent communication channel in the background between a
webclientandserver.
Itisthecombinationofthesetechnologiesthatenablesustoadoptprincipalsoftwareengineering
paradigms, suchas component-and event-based approaches, forweb applicationdevelopment. Our
earlierwork[21]onanarchitecturalstyleforAJAX,calledSPIAR,givesanoverviewofthenewway
webapplicationscanbearchitectedusingAJAX. TheevolutionofwebandtheadventofWeb2.0,and
AJAXinparticular,ismakingtheusersexperiencesimilartousingadesktopapplication. Wellknown
examplesincludeGmailc, GoogleMapsd, orGoogleDocumentse, inwhich, forexample, multiple
userscaneditthesamespreadsheetsimultaneously,allviatheirwebbrowser.
The classical page-sequence web, based on the REST style, makes a server-initiated HTTP re-
quest impossible. Every request has to be initiated by a client, precluding servers from sending
asynchronous notifications without a request from the client [18]. There are several solutions used
inpracticethatstillallowtheclienttoreceive(near)real-timeupdatesfromtheserver. Inthissection
c http://www.gmail.com
d http://maps.google.com
e http://docs.google.com
4 TUD-SERG-2008-009
SERG Bozdagetal.–PerformanceTestingofDataDeliveryTechniquesforAJAXapplications
E.Bozdag,A.Mesbah,andA.vanDeursen 5
weanalyzesomeofthesesolutions.
2.2 HTTP Pull
Mostwebapplicationscheckwiththeserveratregularuser-definableintervalsknownasTimetoRe-
fresh(TTR).Thischeckoccursblindlyregardlessofwhetherthestateoftheapplicationhaschanged.
In order to achieve high data accuracy and data freshness, the pulling frequency has to be high.
This, inturn, induceshighnetworktrafficandpossiblyunnecessarymessages. Theapplicationalso
wastessometimequeryingforthecompletionoftheevent,therebydirectlyimpactingtheresponsive-
ness to the user. Ideally, the pulling interval should be equal to the Publish Rate (PR), i.e., rate at
whichthestatechanges. Ifthefrequencyistoolow,theclientcanmisssomeupdates.
Thisschemeisfrequentlyusedinwebsystems,sinceitisrobust,simpletoimplement,allowsfor
offlineoperation,andscaleswelltohighnumberofsubscribers[15]. MechanismssuchasAdaptive
TTR [4] allow the server to change the TTR, so that the client can pull on different frequencies,
dependingonthechangerateofthedata. ThisdynamicTTRapproachinturnprovidesbetterresults
thanastaticTTRmodel[28]. However,itwillneverreachcompletedataaccuracy,anditwillcreate
unnecessarytraffic.
2.3 HTTP Streaming
HTTPStreamingisabasicandoldmethodthatwasintroducedonthewebfirstin1992byNetscape,
under the name ‘dynamic document’ [24]. HTTP Streaming comes in two forms namely, Page and
ServiceStreaming.
2.3.1 PageStreaming
Thismethodsimplyconsistsofstreamingserverdataintheresponseofalong-livedHTTPconnection.
Mostwebserversdosomeprocessing,sendbackaresponse,andimmediatelyexit.Butinthispattern,
theconnectioniskeptopenbyrunningalongloop. Theserverscriptuseseventregistrationorsome
other technique to detect any state changes. As soon as a state change occurs, it streams the new
dataandflushesit, butdoesnotactuallyclosetheconnection. Meanwhile, thebrowsermustensure
the user-interface reflects the new data, while still waiting for response from the server to finish.
Implementationofthis
2.3.2 ServiceStreaming
ServiceStreamingreliesontheXMLHttpRequestobject.Thistime,itisanXMLHttpRequestconnec-
tionthatislong-livedinthebackground,insteadoftheinitialpageload. Thisbringssomeflexibility
regarding the length and frequency of connections. The page will be loaded normally (once), and
streaming can be performed with a predefined lifetime for connection. The server will loop indefi-
nitely just like in page streaming, and the browser has to read the latest response (responseText) to
updateitsstateontheDocumentObjectModel(DOM).
2.4 Comet or Reverse AJAX
Currently,majorAJAXpushtoolssupportServiceStreaming. TheapplicationoftheServiceStream-
ingschemeunderAJAXisnowknownasReverseAJAXorCOMET[25]. COMETenablestheserver
tosendamessagetotheclientwhentheeventoccurs,withouttheclienthavingtoexplicitlyrequest
it. Suchaclientcancontinuewithotherworkwhileexpectingnewdatafromtheserver. Thegoalis
TUD-SERG-2008-009 5
Bozdagetal.–PerformanceTestingofDataDeliveryTechniquesforAJAXapplications SERG
6 PerformanceTestingofDataDeliveryTechniquesforAJAXApplications
Fig.3.StreamingmodeforCOMET
Fig.4.LongpollingmodeforCOMET
to achieve a real-time update of the state changes and offer a solution to theproblems mentioned in
Section2.2.
The COMET scheme is available thanks to the ‘persistent connection’ feature brought by
HTTP/1.1. With HTTP/1.1, unless specified otherwise, the TCP connection between the server and
thebrowseriskeptalive,untilanexplicit‘closeconnection’messageissentbyoneoftheparties,or
atimeout/networkerroroccurs. Priortopersistentconnections,aseparateTCPconnectionwasestab-
lishedtofetcheachURL,increasingtheloadonHTTPserversandcausingcongestionontheInternet.
With persistent connections, fewer TCP connections are opened and closed, leading to savings both
inCPUtimeforroutersandhosts(clients,servers,proxies,gateways,tunnels,orcaches),aswellas
inmemoryusageforTCPprotocolcontrolblocksinhosts.
HTTP/1.1 reduces the total number of TCP connections in use. However, it still states that the
protocol must follow the request/response scheme, where the client makes a request and the server
returnsaresponseforthisparticularrequest. Thus,onceacompleteresponseisreturned,thereisno
furtherwayfortheservertosenddatabacktotheclientbrowser.
COMETimplementationshavemainlyadoptedthefollowingtechniquestoovercomethisproblem:
Streaming Figure3showshowthestreamingmodeoperates.Aftertheinitialrequest,theserverdoes
notclosetheconnection, nordoesitgiveafullresponse. Asthenewdatabecomesavailable,
theserverreturnsittotheclientinHTTPchunkedmode[32], usingthesamerequestandthe
connection. TypicalCometserverimplementationsimplementthisfeaturebyopeningahidden
iframe element in the browser after page load, establishing a long-lived connection inside the
hiddeniframe. Dataispushedincrementallyfromtheservertotheclientoverthisconnection,
andrenderedincrementallybythewebbrowser[27].
6 TUD-SERG-2008-009
SERG Bozdagetal.–PerformanceTestingofDataDeliveryTechniquesforAJAXapplications
E.Bozdag,A.Mesbah,andA.vanDeursen 7
The problem with this approach is that some web servers might identify this open-request as
idleandclosetheconnection. HTTPchunkedmodemightalsobenotsupportedineveryrouter
thatislocatedonthepath.
Longpolling Figure 4 shows the operation of the long polling mode. In this mode, once again the
server holds on to the client request, however this time until data becomes available. If an
eventoccurs,theserversendsthedatatotheclientandtheclienthastoreconnect. Otherwise,
the server holds on to the connection for a finite period of time, after which it asks the client
to reconnect again. Long polling (also known as Asynchronous-Polling) is a mixture of pure
serverpushandclientpull. Ifthepublishinterval(orthetimeoutvalue)islow,thesystemacts
morelikeapurepull-basedstyle. Ifthepublishintervalishigh,itwillactmorelikeapurepush
approach.
3 COMETImplementations
COMETD and DWR are currently two actively developed open source libraries that bring COMET
supporttoAJAXapplications.Inthefollowingsubsectionswetakeacloserlookatthesetwolibraries.
3.1 COMETD Framework and the BAYEUX Protocol
Asaresponsetothelackofcommunicationstandards[21]forAJAXapplications,theCOMETDgroupf
released a COMET protocol draft called BAYEUX [26]. The BAYEUX message format is defined in
JSON (JavaScript Object Notation)g which is a data-interchange format based on a subset of the
JavaScript Programming Language. The protocol has recently been implemented and included in a
numberofwebserversincludingJettyh andIBMWebspherei.
This protocol follows the ‘topic-based’ [11] publish-subscribe scheme, which groups events ac-
cordingtotheirtopic(name)andmapsindividualtopicstodistinctcommunicationchannels. Partici-
pantssubscribetoindividualtopics,whichareidentifiedbykeywords.Likemanymoderntopic-based
engines, BAYEUX offersaformofhierarchicaladdressing, whichpermitsprogrammerstoorganize
topicsaccordingtocontainmentrelationships. Italsoallowstopicnamestocontainwildcards,which
offers the possibility to subscribe and publish to several topics whose names match a given set of
keywords. BAYEUXdefinesthefollowingphasesinordertoestablishaCOMETconnection:
1. The client performs a handshake with the server, receives a client ID and list of supported
connectiontypes,suchasIFrameorlong-polling(SeeSection2.4).
2. TheclientsendsaconnectionrequestwithitsIDanditspreferredconnectiontype.
3. Theclientlatersubscribestoachannelandreceivesupdates
Although the BAYEUX specification supports both streaming and long polling modes, the
COMETDframeworkcurrentlyonlyimplementsthelongpollingapproach. Pleasenotethat,although
there are other frameworks that support the BAYEUX protocol, in this paper we will use the terms
BAYEUXandCOMETDinterchangeably.
f http://www.cometd.com
g http://www.json.org
h http://www.mortbay.org
i http://www.ibm.com/software/websphere
TUD-SERG-2008-009 7
Bozdagetal.–PerformanceTestingofDataDeliveryTechniquesforAJAXapplications SERG
8 PerformanceTestingofDataDeliveryTechniquesforAJAXApplications
3.2 Direct Web Remoting (DWR)
DirectWebRemoting(DWR)j isaJavaopensourcelibrarywhichallowsscriptingcodeinabrowser
to use Java functions running on a web server just as if they were in the browser. DWR works by
dynamically generating JavaScript based on Java classes. To the user it feels like the execution is
takingplaceonthebrowser, butinrealitytheserverisexecutingthecodeand DWR ismarshalling
the data back and forwards. DWR works similar to the RPC mechanism (e.g., Java RMI [29]), but
withoutrequiringanyplugins. Itconsistsoftwomainparts:
• A Java Servlet running on the server that processes requests and sends responses back to the
browser.
• AJavaScriptenginerunninginthebrowserthatsendsrequestsandcandynamicallyupdatethe
DOMwithreceivedresponsesfromtheserver.
Fromversion2.0andaboveDWRsupportsCOMETandcallsthistypeofcommunication“Active
Reverse AJAX”[10]. Currently, DWR doesnotsupport BAYEUX,andhasadopteditsownprotocol
toexchangedata. DWRsupportsthelongpollingaswellasthestreamingmode. BecauseCOMETD
has no streaming implementation, we will only use the long polling mode of both libraries in our
experiment,inordertobeabletomakeacomparison.
4 ExperimentalDesign
Inthissectionwefirstpresentourresearchquestions. Laterwedescribeourproposedapproach,and
the independent and dependent variables which we will use to conduct the experiment and come to
answerstoourquestions.
4.1 Goal and Research Questions
WesetupourexperimentsinordertoevaluateseveraldependentvariablesusingtheGQM/MEDEAk
frameworkproposedbyBriandetal.[6]. First,wedescribethegoal,perspectiveandenvironmentof
ourexperiment:
Goal: Toobtainarigorousunderstandingoftheactualperformancetradeoffsbetweenapush-based
andapull-basedapproachtoAJAXdatadelivery.
Perspective: Inparticular,weaimatanautomated,repeatableexperiment,inwhichasmany(com-
binations) of the factors that influence performance (such as the number of users, the number of
publishedmessages,theintervalsbetweenmessages,etc.) canbetakenintoaccount.
Environment:Theexperimentsaretargetedatdistributedenvironments,particularythosewithUNIX/Linux
nodes. FurtherrelevantfactorsofthesesystemswillbedescribedinSection5.2.
Wehaveformulatedanumberofquestionsthatwewouldliketofindanswersforineachapproach.
Ourresearchquestionscanbesummarizedas:
RQ1 Howfastarestatechanges(newmessages)ontheserverpropagatedtotheclients?
RQ2 Whatisthescalabilityandoverallperformanceoftheserver?
j http://getahead.org/dwr
kGoalQuestionMetric/MEtricDEfinitionApproach
8 TUD-SERG-2008-009