EDISON Science Gateway: A Cyber-Environment for Domain-Neutral Scientific Computing

12 Pages·2014·4.14 MB·English

IEICETRANS.INF.&SYST.,VOL.E97–D,NO.8AUGUST2014
1953
PAPER SpecialSectiononCyberworlds
EDISON Science Gateway: A Cyber-Environment for
Domain-Neutral Scientific Computing
HoonRYU†a),Jung-LokYU†,DuseokJIN†,Jun-HyungLEE†,DukyunNAM†,JongsukLEE†,
KumwonCHO†,Nonmembers,Hee-JungBYUN††,Member,andOkhwanBYEON†b),Nonmember
SUMMARY We discuss a new high performance computing service ing TCAD softwares but are not familiar of command in-
(HPCS)platformthathasbeendevelopedtoprovidedomain-neutralcom- structions, however, CLI clients may not be the most effi-
putingserviceunderthegovernmentalsupportfrom“EDucation-research cient way to access TCAD services as they first needs to
IntegrationthroughSimulationOntheNet”(EDISON)project.Withafirst
becomefamiliarwiththesetofinstructionstoconfigureand
focusontechnicalfeatures,wenotonlypresentin-depthexplanationsof
theimplementationdetails,butalsodescribethestrengthsoftheEDISON runTCADsoftwaresinclusters. Theissueofprovidingdo-
platformagainstthesuccessfulnanoHUB.orggateway.Tovalidatetheper- main scientists with “easier” TCAD services has thus mo-
formanceandutilityoftheplatform,weprovidebenchmarkingresultsfor tivated the strong needs for a ScienceGateway coupled to
theresourcevirtualizationframework,andprovethestabilityandprompt-
graphicaluserinterface(GUI)clientssuchaswebportals.
nessoftheEDISONplatforminprocessingsimulationrequestsbyana-
Theconceptofthesciencegatewaycanbedefinedasa
lyzingseveralstatisticaldatasetsobtainedfromathree-monthtrialservice
intheinitiativeareaofcomputationalnanoelectronics. Wefirmlybelieve HPCservice(HPCS)platformthatintegratesacommunity-
thatthisworkprovidesagoodopportunityforunderstandingthescience specific set of TCAD softwares, other applications to sup-
gatewayprojectongoingforthefirsttimeinRepublicofKorea,andthat port simulation services (e.g., visualizations), and collec-
thetechnicaldetailspresentedherecanbeservedasanusefulguidelinefor
tions of related dataset coupled to user-friendly clients to
anypotentialdesignsofHPCSplatforms.
keywords: sciencegateway,highperformancecomputingservice(HPCS), provide an easy access to HPC-integrated resources[6],
computationalscience,technologycomputer-aideddesign(TCAD)simula- [7]. Several HPCS platforms have been designed to sup-
tions port online simulations in various fields of computational
science[8]–[12]. Of these, nanoHUB.org project carried
1. Introduction out under the support from US National Science Founda-
tion (NSF)[8]–[10] should be one of successful examples
Highperformancecomputing(HPC)serviceshavebecome thatrepresentstheneedsforsciencegatewaysprovidingon-
general and available to public with a remarkable progress linesimulationserviceviaGUIclientssinceithassupported
incomputingpower,networks,storagesandonlineenviron- anannualuser-basethatnowexceeds230Kfocusingonthe
ments that have driven during the last decade[1],[2]. As onlinesimulationsofnanoelectronics.
a consequence, “computational science” itself has become Thereisnodoubtthattheabove-mentionedHPCSplat-
establishedasanimportantkeywordforresearchesanded- forms including nanoHUB.org project are leading science
ucationsinvariousfieldsofscienceandengineering,where gateways as well as providing public services for online
researchershavenotonlysolvedadvancedproblemsthatare TCAD simulations. They are, however, dedicated for sim-
quite computationally expensive, but successfully demon- ulations in a particular domain area (e.g., nanoelectronics,
stratedtheirresultshaveimpactsthatarecriticalenoughto disastersandchemistryetc). Moreover, allthebackendre-
guidethedesignprocessesneededbyexperimentalists[3]– sourcesintegratedtotheabove-mentionedplatformsarelo-
[5]. catedinUSsuchthattheservicequalitymaynotbegoodin
Needs for Science gateway: Computational scientists some(especiallyAsia-pacific)areaduetothenetworkingis-
who build their own technology computer-aided design sue,althoughthestrongnetworkshouldbethemostcritical
(TCAD) softwares running in parallel on HPC resources, componentinmaintainingoftheservicequalityforTCAD
commonly uses command line interface (CLI) clients such simulationsthattypicallyneedtoprocesslargedatasets.
asTerminalasagatewaytoaccessclusters. Tomanymore Goals and impacts of this work - EDISON project:
people who want to research or get educated using exist- Based on the well-known strong IT infrastructure in Re-
public of Korea, we have recently launched “EDucation-
ManuscriptreceivedNovember10,2013.
researchIntegrationthroughSimulationOntheNet”(EDI-
ManuscriptrevisedFebruary12,2014.
†TheauthorsarewiththeNationalInstituteofSupercomput- SON)projectunderthefinancialsupportoftheMinistryof
ing and Networking, Korea Institute of Science and Technology Education, Science and Technology of Republic of Korea,
Information,RepublicofKorea. where the main objective of the project is to construct an
††TheauthoriswiththeDepartmentofIT,SuwonUniversity,
application-domain neutral HPCS-hub of TCAD softwares
RepublicofKorea.
thatcannotonlyprovideaqualifiedserviceofonlinesimu-
a)E-mail:[email protected]
lations,butbeeasilyaccessiblebyresearchers,studentsand
b)E-mail:[email protected]
DOI:10.1587/transinf.E97.D.1953 developers in Republic of Korea as well as in Asia-pacific
Copyright(cid:2)c 2014TheInstituteofElectronics,InformationandCommunicationEngineers
IEICETRANS.INF.&SYST.,VOL.E97–D,NO.8AUGUST2014
1954
Fig.1 IllustrationofEDISONplatformhierarchy: TheEDISONopenplatformconsistsofato-
talofthreelayers-(1)MiddlewarestackthatconnecttheapplicationframeworktoHPCresources,
(2)Applicationserviceframeworkbasedonaweb-standardRESTfulinterface,and(3)Portalservice
frameworkthatoffersasetofnativeweb-basedGUIclientsforauser-friendlyservice.
area. HPCSplatformshouldsatisfytomeettheabove-mentioned
Themainobjectiveofthispaperworkistodiscussthe purposecanbesummarizedasfollows:
detailsoftheEDISONHPCSopenplatform,whichhasbeen
• Generality: Theplatformmustbeabletomanagevar-
developed and released with an trial service area on com-
ious simulation workflows to support a wide range of
putational nanoelectronics. Especially we demonstrate the
TCADsoftwareincomputationalscience.
practicality and strength of the EDISON platform by ad-
• Expandability: The platform must maintain neutral-
dressing following issues: (1) “Why is the EDISON plat-
ity to computing resources in which TCAD software
formsuitabletosupportdomain-neutralTCADsoftware?”,
willrun. Thatis,theplatformmustprovideanabstract
(2) “What are the strong points of the EDISON platform
interfacethatissynchronouswithanynewresourceen-
from a viewpoint of the service management, especially
vironments.
comparedtotheleadingsciencegateway-nanoHUB.org?”,
• Openness: The platform must provide an open,
and (3) “How good is the performance when the system is
web-standard interface to accommodate various user-
applied to a real-time service?”. We believe that this work
friendlyclientssuchasweb-portals,stand-aloneappli-
provides a good opportunity for understanding the science
cations,andmobileenvironments.
gateway project ongoing for the first time in Republic of
Korea,andthatthetechnicaldetailsdiscussedherearesolid Figure 1 provides an overview of the EDISON open
enoughtobeservedasanusefulguidelineforwhomwantto platform that has been developed to satisfy the above-
startanewimplementationofHPCSplatformsfromscratch. mentioned requirements. The platform broadly consists of
the middleware stack, the application service framework,
2. EDISONPlatform: ImplementationDetails and the portal service framework. The EDISON middle-
ware allows the management of TCAD software metadata,
ThepurposeoftheEDISONplatformistomanagedomain- themanagementofsimulationdatasets,themanagementof
neutral TCAD softwares and related contents by establish- simulation history, and synchronization of heterogeneous
ingaconvergenceenvironmentforsimulationservicesusing (physical and virtual) computing resources. Based on a
cyber-infrastructure. The three critical components that a web-standard REST (Representational State Transfer) ful
RYUetal.:EDISONSCIENCEGATEWAY:ACYBER-ENVIRONMENTFORDOMAIN-NEUTRALSCIENTIFICCOMPUTING
1955
interface[13], EDISON application service framework in-
cludestheuserauthentication,simulationworkflows,meta-
dataqueryofTCADsoftware,storage,andstatisticalanal-
ysis. TheEDISONportalserviceframeworkoffersasetof
web-based GUI clients that are specialized for the simula-
tionworkflowofeachTCADsoftware.
Of all the components described in the platform hier-
archy (Fig.1), the native web-based GUI client (portal ser-
viceframework),theScienceAppstoreservice(application
framework), and virtualized computing resource/job man-
agement framework (middleware stack) are the three key
featuresoftheEDISONplatform,whicheventuallyprovide
theefficiencyinmanagementofsimulationrequestsaswell
as the flexibility in supporting various types of the simula-
tion workflow. In next subsections, we discuss these key
featuresfurtherindetail.
2.1 NativeWeb-BasedInterfaceforSimulations
Tobecomeusefulfortheopencomputingservice, aHPCS
platformshouldnotonlyenableresearchersfromgeograph-
ically distributed locations to share their in-house TCAD
software and related datasets, but also guarantee that any
userscanrunsharedapplicationswithoutbuyingalicenses
for all toolboxes. Moreover, the platform should provide
a set of user-friendly GUI such that researchers/users can
perform the sharing/running process easily. Without loss
of generality, a web-based problem-solving environment
should be the most suitable solution to satisfy the above-
Fig.2 Native Web-interface for Simulations: the PHP-based web-
mentioned motivations since it not only maximizes the ac- interfaceisoverhead-efficientinsupportingthepublicserviceofTCAD
cessibilityviabrowsers,butcanbealsoeasilyimplemented simulationswhencomparedtotheVNC-basedinterface. (a)Theuserin-
usingvariousopenframeworks. terfaceforcontrolofsimulationparameters.(b)Thevisualizationinterface
implementedusingWebGL.
InnanoHUB.org[8],userscancontroltheinputparam-
eters and visualize the results of simulations via the web-
interfacesthatarebasedonthevirtualnetworkingcomput-
ing(VNC)technology. TheVNC-basedinterface,however, 2.2 ScienceAppstoreFramework
is well known not to be generally suitable for the public
service environment since it introduces a substantial over- Ingeneral,thesimulationenvironmentiscomprisedofvari-
headthatresultsinaslowandsometimesunreliablegraph- ouscomponentssuchaspreprocessors(e.g.,controlofsim-
ics protocol when used simultaneously by many users in ulationinputs),TCADsoftware,postprocessors(e.g.,visu-
a wide area network[14]. To avoid the overhead in sup- alizationscheme),computingresources,andjobmanagers.
portingmanyusersasmuchaspossible,therefore,theEDI- Itishoweverhardtoestablishandomain-neutralsimulation
SONplatformallowsuserstoaccesstheserviceviaaweb- environment since the data schema and interfaces of these
native GUI that are developed using a server-side scripting components have not been standardized. Existing simula-
language (PHP)[15]. Figure 2(a) shows the native web- tion environments have been dependent on specific types
interfaceviawhichuserscancontrolasetofsimulationpa- ofTCADsoftware,thusonlysupportinglimitedapplication
rametersintheEDISONplatform. fields[8]–[12].Withthe“ScienceAppstore”frameworkthat
Anotherstrongpointofournativeweb-interface,isthat canbeunderstoodasasetofmetadataschemafortheabove-
it can be easily incorporated with the web-standard tech- mentionedcomponents,theEDISONplatformprovidesthe
nologies for visualizations such as HTML5 Canvas, Web simulationenvironmentthatcanaccommodateagreaterva-
Graphics Library (WebGL) and Scalable Vector Graphics riety of TCAD software in diverse fields of computational
(SVG)[16]–[19]. Figure 2(b) shows the visualization in- science.
terfaces of simulation results that are implemented using Metadata schema forTCAD software: For the service
WebGL,whichsupportshigh-qualityvisualeffectsdirectly of domain-neutral applications, we have carefully exam-
in browsers with lesser overhead compared to the VNC- ined the workflow patterns of various TCAD software in
coupledvisualizations. the fields of computational fluid dynamics, nanoelectron-
ics, chemistry, structuraldynamicsandmedicine, andhave
IEICETRANS.INF.&SYST.,VOL.E97–D,NO.8AUGUST2014
1956
Fig.4 TheScienceAppstoreframework:(a)ThestructureoftheSci-
enceAppstoreframeworkforthestorageandmanagementofTCADsoft-
waremetadata. ItconsistsofaNoSQL-basedmetadatarepositoryandan
Fig.3 AconceptualdiagramrepresentingtheXSDsetofTCADsoft-
APIsetforregistrationandmanagementofthemetadata.(b)Afulllistof
waremetadata: Thehierarchyandcardinalityofstandardizedmetadata
RESTfulAPIoftheScienceAppstoreframework.
schemaareshown. TheXML-basedschemaishighlydocument-centric,
andisparticularlystronginexpandabilityastheaddition/deletionofnew
elements/attributesarequiteflexible.
tionwithwhichuserscaneasilysearchtheirpreferred
TCADsoftwareontheweb-portal(thegeometries,ap-
establishedaschemaof131componentsofsoftwaremeta- plicationsandproblemcategoriesofsimulations).
data. The 78 elements and 53 attributes are again further • Additional Section: This section contains the infor-
classified to a total of six sections with a consideration of mation that supports the validity of TCAD software
interoperationsamongthecomponentsofthesimulationen- (the list of related technical reports. research papers
vironment,whereeachsectioncomposestheXMLschema andothercontents)aswellastheinformationofusers
definition(XSD)[20]ofthestandardizedmetadataset. De- (majors,gradesandpurposesofportalusers).
tails of the XSD are described as follows, and Fig.3 illus- • SystemSection: Thissectioncontainstheinformation
tratesthehierarchyandcardinalityofthemetadataschema. withwhichtheadministratorcanmanagesoftwaresbe-
ingservedontheHPCSplatform(theversioncontrol,
• IdentificationSection:Thissectioncontainstheinfor-
usagefrequencyandstatusofservice).
mationneededtoidentifytheTCADsoftware(thetitle,
version,developerandaffiliation)aswellastheinfor- The Rapid APPlication infrastrucTURE (Rappture)
mation needed to help users understand the softwares [21], an interface for running TCAD simulations that is
(thedescription,featuresandscreenshotsofsimulation currently being utilized in nanoHUB.org, also manages
details). the software metadata based on the XML schema. While
• Code Section: This section contains the information the Rappture focuses on describing the input and out-
needed to create automated job commands (the name put metadata of each TCAD software, the Science App-
andpathoftheexecutablefile, andthepathwherere- store framework manages the metadata in a broader man-
sultfilesarestored)aswellastheinformationneeded ner, categorizing the dataset into workflow components,
to install and run the software (the programming lan- i.e., inputs/outputs of the TCAD software and types of the
guage,compilerandstaticlibrariesrequiredtolink). preprocessor/postprocessor required by each TCAD soft-
• Parameter Section: This section contains the infor- ware,suchthatsimulationsofdiverseworkflowpatternscan
mation needed to create interfaces that allow users to besupportedflexibly.
control the set of simulation parameters (the names, Storage of software metadata and API set: Fig.4(a)
data types, descriptions and default values of control illustratesthestructureoftheScienceAppstoreframework
parameters). thatisusedtostoreandmanagedomain-neutralTCADsoft-
• CategorySection: Thissectioncontainstheinforma- ware. The metadata repository has been developed using
tion needed to set the simulation workflows (the pre- a Not-only SQL (NoSQL) database (DB) class[22] that is
processor and postprocessor) as well as the informa- well-known to be good to preserve unstructured (arbitrary)
RYUetal.:EDISONSCIENCEGATEWAY:ACYBER-ENVIRONMENTFORDOMAIN-NEUTRALSCIENTIFICCOMPUTING
1957
datasets. We have especially employed MongoDB to im-
plement the storage system[23],[24], which allows the in-
dependentandexpandablesynchronizationbetweensimula-
tioncomponentswithoutconstraintsontheschemastructure
yetstillsupportingthepreservationofthedocument-centric
dataschema.
The APIs for the management of the software meta-
data are available as a RESTful interface. The interface
hasbeenimplementedwithNode.js[25],whichisaserver-
sideJavaScriptdesignedforimplementationofscalablein-
ternet applications, notably web-servers. The APIs are
written using event-driven, asynchronous inputs and out-
puts (I/O) to minimize overhead and maximize scalability.
ThelistoftheRESTfulAPIsneededtooperatethereposi- Fig.5 Structure of virtualized computing resources and job man-
agementframework:Theframeworkbroadlyemploysathree-layerhier-
toryisshowninFig.4(b),whichbasicallyprovidesCRUD
archy.Thebottommostabstractionlayersupportstheinteroperationofau-
(create/read/update/delete) and Query functions to manage thentication,virtualization,jobschedulerandstoragedevices. Thecentral
metadata of specific TCAD software, pre/postprocessor, layeristhecoreoftheframeworkthatenablesusermanagement,server
andconfiguration. Indetail,theAPIsetcanbecategorized provisioning and job management. The topmost service layer provides
RESTfulinterfacesviawhichthesystemadministratorcaneasilymanages
asfollows:
computingresourcesandjobrequests.
• Solver:APIsinthisgroupcanbeutilizedtodeployand
managethemetadataofthespecificTCADsoftware.It
providesfunctionstoCRUDthesoftwaremetadata,as • Must enable an efficient utilization of available com-
wellastosearchthesoftwarebasedontheuser-quary. puting resources to support diverse simulation work-
APIshavebeendesignedtoprovidethesystemadmin- loadsthatrunonvariousapplicationenvironments.
istratorsand(TCADsoftware)developerswiththeac- • Must have a web-standard interface such that the sys-
cesstostorages. tem administrators can easily monitor the computing
• Pre/Post-Processor:APIsinthisgroupprovidesfunc- resourceandjobstatus.
tionstoCRUDthemetadataofpre/postprocessors,and
Figure 5 illustrates the hierarchy of the virtualized
to retrieve pre/postprocessors coupled to the specific
computing resource/job management framework. The
TCADsoftwarebasedonthequery.
framework consists of the three layers - the abstraction,
• Configuration: APIsinthisgroupprovidesfunctions
core-framework,andweb-service.Thebottommostabstrac-
to acquire the metadata of additional configurations
tion layer supports the interoperation among various func-
(e.g.,librarypackages)neededtorunthesoftware.
tionalenvironmentssuchastheauthentication,resourcevir-
tualization,managementsofsimulationjobsandstoragede-
2.3 Virtualized Resource and Job Management Frame- vices. The central layer is the core-framework that pro-
work videstheactualfunctionalitiesrequiredforservicessuchas
the user management, (physical/virtual) server provision-
Resourcevirtualizationshouldbetheoneofmostessential ing, and simulation/job requests. Finally, the web-service
componentsforHPCSplatforms. Theubiquitousavailabil- layer supports web-standard RESTful interface, via which
ity of the virtualization technology brings the efficient re- theadministratorcanaccessandmanagethevirtualizedre-
sourcemanagementforpubliccomputingservicessuchthat sourcesandsimulationjobseasily.
oneofthebenefitsis,forexample,thatitenablesthesupport The abstraction layer of the framework provides the
ofmultiplehardwarearchitecturesandapplicationenviron- abstraction interface for the user authentication and man-
mentswithlimitedtypesandnumbersofphysicalmachines, agementsofthevirtualizationplatform,simulationjobsand
providingtheversatilityofsupportingvariousTCADappli- storage devices, and thus eventually enable the interopera-
cationsthatworkonvariousapplicationenvironments[26]. tion between these functionalities and the pre-constructed
Anothercomponentcriticalfortheopencomputingservice infrastructure of the server farm and HPC storages. The
isthejobmanagementframeworkviawhichthesystemad- authentication interface provides the local DB and Light-
ministrators can manage computing resources by monitor- weight Directory Access Protocol (LDAP, [27]) plug-ins.
ing jobs requested by users. In this subsection, we discuss For on-demand virtual machines (VMs) provisioning, the
the virtualized computing resources and job management Xen[28]plug-in,whichisbuiltontopofOpenNebula[29]
frameworkoftheEDISONplatform. cloud management toolkit, is available on a trial basis.
Structureandfunctionalities: Thevirtualizedcomput- OpenPBS[30] and Network File System (NFS) are sup-
ing resource/job management framework of the EDISON ported as plug-ins for the management of simulation jobs
platform has been developed upon the following design andstoragedevices,respectively.
principles: In the core framework, information of user, physical
IEICETRANS.INF.&SYST.,VOL.E97–D,NO.8AUGUST2014
1958
are used for privisioning virtual machines/clusters to
physical servers that are registered by administrators.
Being accessible by both administrators and general
users, the API set enables the request of provisioing
with information of machine specifications such as a
number of processors and size of memory etc. It also
provides users with the access to detailed information
ofprovisionedVMs,andallowstheVMownertosub-
mitrequeststoresume/suspendthemachines.
• File I/O: Generally TCAD simulations not only re-
quire multiple input data, but also produce multiple
result data upon the completion of jobs. APIs in this
group is used to build interfaces via which users can
control simulation inputs and access results produced
whensimulationsarefinished.
• Simulation management: The framework interprets
a “simulation” as an virtual parent object, which has
a subset of individual jobs that become the child ob-
jects, i.e., a simulation for a patameter study consists
Fig.6 RESTful APIs for the virtualized resource and job man- ofmultipleindividualjobsthathasdifferentvaluesfor
agement framework: the APIs are classified into six groups based
on the functionalities: User management, host management, VM the specific parameter. The APIs in this group facili-
provisioning/management,fileI/O,simulationandjobmanagement. tatesthemanagementofjob-setsprovidingtheCRUD
functionality.
• Job management: After creating a simulation object
server(host),VM,VMimage,virtualnetwork,storageand
as a virtual parent, users can manage individual jobs,
simulationjobaremanagedasPlainOldJavaObjects(PO-
e.g., the specific jobs of interest can be individually
JOs)coupledtoHibernate/HSQL[31]. Thiscore-layerpro-
submittedbycallingthe submit API.Theinterfaceal-
vides actual functionalities for the user management (reg-
lowsuserstomonitorthestatusofasubmittedjob, as
istration, authentication and authorization), physical server wellastocancelthejobthroughthestatus/cancelAPI.
management(additionanddeletion),on-demandVMprovi-
The result data can be also accessed when the job is
sioning,jobsubmissionandmonitoring. Thetopmostweb-
successfullycompleted.
servicelayer,whichprovidestheRESTfulinterfacesupport-
ing CRUD operations for the objects managed in the core-
layer,hasbeendevelopedusingtheSpringwebModelView
3. ResultsandDiscussion: EDISON Nanophysics
Controller(MVC)framework[32].
APIs for resource and job management: The frame-
Tobenchmarktheutilityandhearusers’experienceforthe
workprovidesasetof42RESTfulAPIswhereafulllistis
improvement of the currently developed EDISON HPCS
showninFig.6. Theweb-standardinterfacecurrentlyavail-
platform, wehaveperformedatrialserviceforonlinesim-
ableforthemanagementofcomputingresourcesandsimu-
ulations during about 3 months, focusing on the initiative
lationjobs,canbesummarizedasfollows:
serviceareaofthecomputationalnanoelectronics(physics).
• User management/authentication: The system ad- Atotalof272usershavejoinedthetrialservice,performing
ministrator is able to register and delete the user in- a total of 4,220 simulation jobs during the service period
formation. General users can obtain the authentica- (see Table 1 for the service outline.) 8 nodes coupled to
tion through the login/logout interface, and can ac- 1Gnetwork,whereeachnodehastwo6-core2.4GHzIntel
cess other service APIs using the token provided if Xeons(E5650)and96GBmemory,areprovidedasphysical
the authentication is successful. The framework uses resources(96cores). Eachphysicalcoreisthenmappedto
aHTTP(S)basicauthenticationmechanismtoauthen- asetoftwoVMsthatrunonCentOS5.6with2vCPUsand
ticateusersandfordelegationofauthority. 4GB memory. Xen[33] 4.0.1 Hypervisor is employed as a
• Physicalsever(host)management: Toperformsim- VMmonitor(VMM).
ulation jobs requested by users, virtual machines/ Nextsubsectionsdiscussthepracticalityandutilityof
clusters must be provisioned and registered in theEDISONHPCSplatforminbothaqualitativeandquan-
the shared pool. The framework provides APIs titative manner. We not only present detailed descriptions
to register/delete physical servers where virtual of the service environment that has been supported via the
machines/clusters will be created. Access to the set EDISON Nanophysicsportal(http://nano.edison.re.kr)dur-
of APIs in this category is granted to the system ad- ingtheserviceperiod,butalsodemonstratetheperformance
ministratorsonly. of the framework for the resource virtualization and job
• VM provisioning/management: APIs in this group management when coupled to the real TCAD applications
RYUetal.:EDISONSCIENCEGATEWAY:ACYBER-ENVIRONMENTFORDOMAIN-NEUTRALSCIENTIFICCOMPUTING
1959
Fig.7 Web-interfacesforonlinesimulationservices: Userscanperformandaccesssimulations
easilyviauser-friendlyGUIsforthefollowing4steps-(1)InterfacesforsearchingandselectingTCAD
softwareatusers’preference,(2)aninterfaceforinputcontrolsfortheselectedsoftware,(3)aninterface
formonitoringthestatusofrequestedjobs,and(4)interfacesforaccessingthesimulationresults. All
thedetailsininputparameters,supplementaldescriptions,workflowpatternsandpre/postprocessingof
TCADsoftwareshouldbeprovidedasasetofmetadatafromdevelopers.
Table1 EDISONNanophysicstrialservice:Overview. ecutables that work correctly, users can easily select
Objective ServicePeriod SummaryofUsage theirpreferredapplicationsviatheweb-interfacewhich
In-class 2012-Oct-15∼ 168Users, not only shows the list of available TCAD software
Utilizationa 2012-Nov-30(47days) 2,165SimulationJobs (STEP 1.1 in Fig.7), but also provides detailed infor-
EDISON 2013-Mar-04∼ 104Users, mation related to each TCAD software (STEP 1.2 in
Fig.7).
Competitionb 2013-Apr-19(47days) 2,055SimulationJobs
• Control of simulation parameters: Upon the selec-
aUtilized in 7 classes in following 5 domestic institutes: tion of preferred software, users can control the set
Korea Advanced Institute of Science and Technology (KAIST), ofinputparameterstoperformmeaningfulsimulations
Sookmyung Womens’ Univ., Sejong Univ., Seoul National Univ. (STEP2inFig.7). Theinterfacesupportsdirectinputs
(SNU),andKoreaUniv. of parameter values as well as upload of files needed
bEDISONCompetitioninNanophysics(electronics)hasbeen as preconditions of a simulation. If preprocessors are
heldunderthesupportofNationalResearchFoundationofKorea
needed, they can be included in the simulation work-
(http://www.nrf.re.kr)tomotivatestudentstoperformtheirownre-
flow at the moment of the registration. Brief descrip-
searchbasedonwhatlearnedinclassrooms.
tions for parameters are also available such that users
candouble-checkanypotentialmistakeswhiletheyare
innanoelectronicsforservice.
workingonthesimulationsetup. Detailedinformation
of the input control (parameter names, types, descrip-
3.1 ServiceEnvironment
tions,andpreprocessors)mustbepresentedasapartof
themetadatasetbythesoftwaredevelopers.
Web-interface for online simulations: User-friendly web- • Monitoring of submitted jobs: Once the job is sub-
interface is one of the critical features that a qualified ser-
mitted to the backend resource, users can monitoring
vice environment for online simulations should have. Fig-
thestatusofrequestedsimulationjobs.Themonitoring
ure 7 describes the set of web-interfaces that are presented
interface(STEP3inFig.7)provides4typesofthejob
intheEDISONNanophysicsportaltosupportan“easy”on- status(QUEUED/RUNNING/FAIL/SUCCESS).Jobs
lineserviceforTCADsimulations. Tobemorespecific,the
that are queued or running can be cancelled at users’
EDISON Nanophysics portal provides the set of GUIs via
preference.
whichuserscanperformandaccessTCADsimulationsac- • Interpretationofsimulationresults:Forsuccessfully
cordingtothefourstepsasfollows:
completed jobs, users can either visualize in a prede-
• SelectionofaTCADsoftware: Oncetheadministra- finedway,ordirectlydownloadthesimulationresults.
torregisterthesoftwareintheScienceAppstoreframe- Currently, the visualization is supported in two ways;
workwithacompletesetofsoftwaremetadataandex- (1) a in-house tool developed using WebGL[17] for
IEICETRANS.INF.&SYST.,VOL.E97–D,NO.8AUGUST2014
1960
Table2 EDISONNanophysicstrialservice:Thesixapplicationsmostlypopularlyutilized.
Title Descriptions Platforms
pnjuncLAB Solvestheelectrostaticsof2-terminalP/Njunctiondiodes[35];Usefultoeducate C++,LAPACK[36].
semiconductorfundamentalsforundergraduatesinelectricalengineering.
pwLAB Solvesthequantumconfinementofsingleparticlein1Dfinitepotentialwells[37]; C++,LAPACK,
Usefultoeducatecorebasicsinquantummechanicsforundergraduatesinphysics. ARPACK[38].
collisionLAB Solvestheelasticcollisionsbetweentwoobjects;Usefultoeducatethebasicsin Fortran(F90family).
physicsforfreshmenstudentsinmajorsofscienceandengineering.
emesLAB Solvestheelectronicstructureof3Dfinitenanoscaledeviceswithaneffective C++/MPI[40],LAPACK,
massframework[39];Usefulinresearchesonnanoelectronicsmodeling ParallelARPACK.
(materialproperties);Supportlarge-scalecomputationrunninginparallel.
nanowireFET Solvesthe3DquantumtransportproblemsusingtheNon-equilibriumGreen’s C++/MPI,LAPACK,
Function(NEGF)[41];Usefulinresearchesofnanoelectronicsmodeling PETSc[42].
(current-voltagepropertiesofnanoscaledevices);Runsinparallel.
1D data visualizations (STEP 4.1 in Fig.7), and (2) alized computing resources that have been utilized during
Paraview[34] open source toolkit for 2D/3D data vi- thetrialserviceperiod,theexecutiontimeoftheNASParal-
sualizations(STEP4.2inFig.7). WhileParaviewsup- lelBenchmarks(NPB)[43],[44]workloadshasbeenbench-
portsthe1Dvisualization,allowinguserstotakeline- marked against physical resource with a problem size of
cutsof2D/3Ddata,itinvolvesrelativelylargeresource class C. The four NPB workloads are described as follows
overheadduetothedata-renderingengine. ForTCAD (forelaborateddescriptions,seeRefs.[41]and[42].):
softwares dedicated to 1D problems where the data-
• EP (Embarrassingly Parallel): Generate a set of
renderingisnotcriticallyneeded,thevisualizationcan
independent Gaussian random variables using the
bedoneusingourin-housetoolwithlessoverhead.
Marsagliapolarmethod.
TCADapplicationsavailableinEDISON Nanophysics: • MG(MultiGrid):Approximatethesolutiontoathree-
During the service period, 15 applications that are imple- dimensional discrete Poisson equation using the V-
mented and run in a linux environment, have been served cyclemultigridmethod.
foronlinesimulationssupportingeducationsofcorebasics • FT (Fast Fourier Transform): Solve a three-
andresearchesinsemiconductorengineeringfundamentals, dimensional partial differential equation (PDE) using
quantum mechanics, and electronic structure and quantum thefastFouriertransform.
transport in advanced nanoscale devices. Details of some • LU (Lower-Upper symmetric Gauss-Siedel): Solve
applicationsthathavebeenpopularlyutilized,canbefound nonlinear PDEs using the algorithm involving a sym-
from the summary given in Table 2. We note that 10 more metricsuccessiveover-relaxationsolverkernel.
applicationswillbecomeavailableintheportalpriortothe
The performance of VMs that have been provided
fullserviceofEDISON Nanophysicsthatwillbelaunched
for TCAD simulations is demonstrated Fig.8, where
innearfuture.
we expressed the execution time of test workloads
(EP/MG/FT/LU)measuredinVMsasaratiotothatmea-
3.2 Performance: EDISONPlatform suredinphysicalnodes. TheVMperformanceforEP-type
workloads,wherethecommunicationratioamongCPUsbe-
Since an HPCS platform indeed consists of various com- comesminimal, turnsouttobealmostnative, i.e., theexe-
ponentsasdiscussedintheprevious sectionandillustrated cution times in VMs are almost identical to that in physi-
in Fig.1, it should be in principle difficult to define a rea- cal nodes. This, which readers might think is trivial at a
sonable set of performance indicators, especially when we firstglance,indicatesonethingimportantfromaviewpoint
focustoomuchoneverytechnicalcomponentonly. Inthis oftheservice,thattheresourcevirtualizationweemployed
subsection, therefore, we would like to discuss the perfor- is indeed efficient for a simultaneous processing of multi-
manceoftheEDISONplatformfocusingonthetwoview- plerequestsforserialjobs(simulationsrunningonasingle
points-resourcevirtualizationandjobmanagement,which CPU) that do not communicate one another at all. Degra-
aretheimportantfactorstightlycoupledtothecostandqual- dationinperformanceis,however,observedtoincreasefor
ityoftheservice. workloads of larger communication ratio such that FT and
Virtual machines: As addressed the resource virtual- LU-typeworkloads,whenrunin32virtualizedcores,takea
izationiscritical,sinceitservesasakeyfactorenablingthe 2.2and2.5timeslongerwall-timethanin32physicalcores,
efficient management of the computing resource and thus respectively.
could result in a reduction of the cost required to main- TheexperimentalresultontheXen-virtualizationover-
tain the service. To understand the performance of virtu- headisshowninFig.9,whereweexpressedtheCPUover-
RYUetal.:EDISONSCIENCEGATEWAY:ACYBER-ENVIRONMENTFORDOMAIN-NEUTRALSCIENTIFICCOMPUTING
1961
Fig.8 Performance of virtualized computing resources: VMs be-
comealmostnativefortheEP-typeworkloadindicatingthattheresource
virtualizationhasbeenindeedquitesuccessfulinmanagingserialjobre-
questswithjust96physicalcores. Degradationinperformance,however,
isobservedasthecommunicationratioincrease(MG<FT<LU).
Fig.10 UsageandperformanceofEDISONtrialservice:(a)Monthly
usage(top)andtypeofjobsprocessedduringtheserviceperiod(bottom).
(b)Distributionofjobstatus. (c)Aflowshowingrunning(top)andwait-
ing(bottom)timesforsimulationjobsthatarefinishedsuccessfully. The
resultshereclearlyindicatesthattheservicehasbeenquitepromptinpro-
cessingthejobrequests.
whenever the access to resources is needed. When a par-
alleljobinvolvesalargercommunicationratioandalarger
numberofCPUs,thewall-timecannotbutincreasesincethe
Domain-0VMnotonlyreceivesmorerequestsfornetwork
I/O, but also needs to process all the requests called from
Domain-U VMs. Since it is well known that the VMM-
bypassI/OtechniquessuchasPCI-passthroughandsingle-
root I/O virtualization (SR-IOV) significantly reduce the
I/Ooverhead[46],weplantoworktoimprovetheresource
virtualizationframeworkinnearfuture.
Fig.9 ComparisonofXen-virtualizationoverhead(physicalvs.vir-
tual (8 VCPUs)) according to benchmark characteristics: (a) Kernel Overall performance of the service: To discuss the
(version2.6.18)compilationtimewithvaryingnumberofprocesses. (b) quality of the trial service in an overall viewpoint, we
Normalizedfilecopytimeontothesamelocaldiskvolumewithvaryingfile present a couple of statistical datasets in Fig.10 that
size. (c)Networkmicrobenchmarkswithvaryingmessagesize(roundtrip
have been obtained during the service period. First, the
latencyandonewaybandwidth).
monthly distribution of all the 4,220 simulation jobs (top
of Fig.10(a)) shows that the service utilization has been
head (time needed for kernel compilation), disk overhead quiteconsistentwithnoremarkablefluctuationsintheusage
(timeneeded forfilecopy), andnetworkoverhead (latency frequency,indicatingtheplatformhasbeenunderaconsis-
and bandwidth) measured in VMs as a ratio to that mea- tent stress test during the trial service. Roughly 19.2% of
sured in physical nodes. The result here clearly shows all the processed jobs (809 jobs) ran on parallel while the
that the overhead of VMs mainly comes from the disk and other80.8%(3,411jobs)wereserialasshowninbottomof
network I/O, supporting the message from Fig.8, i.e., the Fig.10(a). 86.7% of the submitted requests (3,658 jobs)
workload with a higher communication ratio causes larger have been completed successfully, while the others have
degradationinperformance. ThereasonwhytheVM’sI/O been either cancelled (7.2%, 301 jobs) by users or failed
causesthemajoroverhead,canbefoundfromtheI/Osub- (6.2%,261jobs)duetousers’faultinsettinginputparame-
system of Xen Hypervisor that is based on the split driver ters. Sincenofailedjobshavebeenobservedduetotechni-
model[45]. Here, only one privileged VM (Domain-0) is calissuesoftheplatformsofar,Fig.10(c)supportsthatthe
allowed to access physical I/O resources while all other operationoftheplatformhasbeenquitestable.
VMs (Domain-U) must forward the request to Domain-0 Figure 10(d) shows the flow of times taken to com-
IEICETRANS.INF.&SYST.,VOL.E97–D,NO.8AUGUST2014
1962
plete jobs successfully upon the start (top), and times that Computing: From Parallel Processing to the Intenet of Things,
submitted jobs had to wait for until they start to run (bot- MorganKaufmann,2012.
tom). The successful 3,658 successful jobs have been run- [2] Top500SupercomputerSite.Availablefromhttp://www.top500.org
ning for ∼5,890 seconds (top) and have been waiting for [3] G.Klimeck,S.S.Ahmed,H.Bae,N.Kharche,R.Rahman,S.Clark,
B. Haley, S. Lee, M. Naumov, H. Ryu, F. Saied, M. Prada, M.
∼0.6seconds(bottom)onaverage. Theextremelysmallav-
Korkusinski,andT.B.Boykin,“Atomisticsimulationofrealistically
erage waiting time here indicates that the service has been sizednanodevicesusingNEMO3-D:PARTI-Modelsandbench-
quitepromptinprocessinguserrequestsforsimulations. marks,”IEEETrans.ElectronDevice,vol.54,no.9,pp.2079–2089,
2007.
[4] B.Weber,S.Mahapatra,H.Ryu,S.Lee,A.Fuhrer,T.C.G.Reusch,
4. Conclusions
D.L.Thompson,W.C.T.Lee,G.Klimeck,L.C.L.Hollenberg,and
M.Y.Simmons, “Ohm’sLawSurvivestotheAtomicScale,” Sci-
In this paperwork, we have discussed the current status of ence,vol.335,pp.64–67,2012.
the EDISON science gateway project ongoing in Repub- [5] M. Fuechsle, J.A. Miwa, S. Mahapatra, H. Ryu, S. Lee, O.
lic of Korea under governmental support. Along with an Warschkow,L.C.L.Hollenberg,G.Klimeck,andM.Y.Simmons,“A
single-atomtransistor,”Nat.Nanotechnol.,vol.7,pp.242–246,2012.
overviewofthehierarchy,weprovidein-depthexplanations
[6] N.Wilkins-Diehr,Specialissue: sciencegateways-commoncom-
for technical details of the EDISON platform focusing on
munity interfaces to grid resources Concurrency and Comput.:
the three key features, i.e., native web-interface, Science
Pract.andExp.,vol.19,pp.743–749,2007.
Appstore,andvirtualizedresource/jobmanagementframe- [7] N. Wilkins-Diehr, D. Gannon, G. Klimeck, S. Oster, and S.
work, which handle job requests overhead-efficiently and Pamidighantam, “TeraGrid science gateways and their impact on
supportdomain-neutralTCADsoftwarewithgreatexpand- science,”Computer,vol.41,pp.32–41,2006.
[8] nanoHUBportal(Nanoelectronics).
ability.
Availablefromhttp://nanoHUB.org
The performance of resource virtualization has been
[9] G. Klimeck, M. McLennan, S. Brophy, G. Adams III, and M.
demonstrated by benchmarking the performance against Lundstrom, “nanoHUB.org: advancing education and research in
physical nodes with NAS parallel workloads, and we ob- nanotechnology,”IEEEComput.Eng.Sci.,vol.10,pp.17–23,2008.
serve that VMs exhibit degradation in performance if they [10] A.Strachan,G.Klimeck,andM.Lundstrom,“Cyber-enabledsim-
arecoupled toworkloads of ahigher communication ratio. ulationsinnanoscalescienceandengineering,”Comput.Sci.Eng.,
vol.12,pp.12–17,2010.
Basedontheexperimentalresultonthevirtualizationover-
[11] neesHUBportal(Earthquakeengineering).
head,weconfirmthattheoverheadmainlycomesfromthe Availablefromhttp://neesHUB.org
diskandnetworkI/O,andsuggestastrategyforimproving [12] WebMOportal(Chemistry).Availablefromhttp://www.webmo.net
theVMperformanceasanear-futureworkingitem.Thesta- [13] C.Pautasso,O.Zimmermann,andF.Leymann,“RESTfulWebSer-
bility and promptness of the EDISON platform in support- vicesvs.“Big”Webservices: Makingtherightarchitecturaldeci-
sion,”Proc.InternationalConferenceonWorldWideWeb,2008.
ingsimulations,havebeenalsoconfirmedfromthestatisti-
[14] E. Fernandez, E. Heymann, and M.A. Senar, “Resource manage-
caldatasetobtainedfroma3-monthtrialserviceinthearea
mentforinteractivejobsinagridenvironment,”Proc.IEEEInter-
ofcomputationalnanoelectronics(EDISON Nanophysics). nationalConferenceonClusterComputing,2006.
In near future, we are going to start a full service [15] ThePHPgroup.Availablefromhttp://kr1.php.net/history
ofEDISON Nanophysics(http://nano.edison.re.kr)andex- [16] I.HicksonandD.Hyatt, “HTML5: Avocabularyandassociated
pandtheserviceareatocomputationalchemistryandcom- APIsforHTMLandXHTML,”W3CWorkingDraft19,2010.
[17] C.Marrin,TheWebGLSpecification-Version1,KhronosWebGL
putational fluid dynamics. Through the establishment of a
WorkingGroup,2011.
virtuoussystemwithconvergencebetweenresearchanded-
[18] KhronosWebCLWorkingGroup.
ucation, we firmly believe that this work will lay the foun- Availablefromhttps://www.khronos.org/webcl
dationforenhancingthenationalcompetitivenessinscience [19] W3CSVGWorkingGroup.
andengineering. Availablefromhttp://www.w3.org/Graphics/SVG
[20] W3CXSD.Availablefromhttp://www.w3.org/TR/xmlschema11-1
[21] M.McLennan,TheRapptureToolkit,
Acknowledgements Availablefromhttp://rappture.org
[22] M.Stonebraker,SQLDatabasesv.NoSQLDatabases,Communica-
This work has been carried out under the support from the tionsoftheACM.vol.53,no.4,pp.10–11,2010.
EDucation-research Integration through Simulation On the [23] R.Cattell,“ScalableSQLandNoSQLDataStores,”ACMSIGMOD
Record,vol.39,no.4,pp.12–27,2010.
Net (EDISON) project funded by the Ministry of Science,
[24] E.Plugge, P.Membrey, andT.Hawkins, TheDefinitiveGuideto
ICT and Future Planning, Republic of Korea (Grant Num-
MongoDB,Apress,2010.
ber: 2011-0020576). H. Ryu acknowledges the extensive [25] Node.js.Availablefromhttp://en.wikipedia.org/wiki/Nodejs
use of TACHYON-II HPC cluster for testing and bench- [26] M.F.Mergen,V.Uhlig,O.Krieger,andJ.Xenidis,“Virtualization
marking TCAD software, and would like to dedicate this for high-performance computing,” ACM SIGOPS Operating Sys-
work to J.H. Sohn for her consistent encouragement and temsReview,vol.40,no.2,pp.8–11,2006.
[27] T. Howes and M. Smith, LDAP: Programming directory-enabled
supports.
applicationswithlightweightdirectoryaccessprotocol,Macmillan
Publishing,1997
References [28] P.Barham,B.Dragovic,K.Fraser,S.Hand,T.Harris,A.Ho,R.
Neugebauer,I.Pratt,andA.Warfield,“Xenandtheartofvirtual-
[1] K.Hwang,C.F.Geoffrey,andJ.J.Dongarra,DistributedandCloud ization,”Proc.ACMSymposiumonOperatingSystemsPrinciples,