Handbook on Data Centers

1309 Pages·2015·42.214 MB·English

by  Samee U. Khan, Albert Y. Zomaya (eds.)

Handbook on Data Centers
Samee U. Khan • AlbertY. Zomaya
Editors
Handbook on Data Centers
Editors
SameeU.Khan AlbertY.Zomaya
DepartmentofElectrical SchoolofInformationTechnologies
andComputerEngineering TheUniversityofSydney
NorthDakotaStateUniversity Sydney
Fargo NewSouthWales
NorthDakota Australia
USA
ISBN978-1-4939-2091-4 ISBN978-1-4939-2092-1(eBook)
DOI10.1007/978-1-4939-2092-1
LibraryofCongressControlNumber:2014959415
SpringerNewYorkHeidelbergDordrechtLondon
© SpringerScience+BusinessMediaNewYork2015
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartofthe
materialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsorthe
editorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforanyerrors
oromissionsthatmayhavebeenmade.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Data centers play an important role in modern IT infrastructures. A data center is
a home to computational power, storage, and applications necessary to support an
enterprisebusiness.DatacentersprocessbillionsofInternettransactionseveryday.
Nowadays, data centers are the beating hearts of the companies they serve. Large
datacenterswiththousandsofservershavebeendeployedbyrenownedICTorgani-
zations,likeIBM,Microsoft,Amazon,Yahoo,E-Bay,andGoogle,toprovidecloud
computingservices.Thephenomenalincreaseinthesizeandnumberofdatacen-
tersandresultantincreaseinoperationalcosthasstimulatedtheresearchinvarious
domainsofdatacenters,suchasenergyefficiency,resourcemanagement,network-
ing,andsecurity.ThisdetailedHandbookonDataCenterscoversinasuccinctand
orderlymannerallaspectspertainingtodatacenterstechnologies.
Giventhefastgrowingexpansionofdatacenters,itisnotsurprisingthatavariety
ofmethodsarenowavailabletoresearchersandpractitionersstrivingtoimprovedata
center performance. This handbook aims to organize all major concepts, theories,
methodologies, trends, challenges and applications of data centers into a coherent
andunifiedrepository.Moreover,thehandbookprovidesresearchers,scholars,stu-
dents and professionals with a comprehensive, yet concise source of reference to
datacentersdesign,energyefficiency,resourcemanagement,andscalabilityissues.
The handbook consists of nine parts, where each part consists of several chapters.
Thefollowingtopicsspanningthedatacentertechnologiesarecoveredindetail:(a)
energyefficiency,(b)networking,(c)cloudcomputing,(d)hardware,(e)modeling
andsimulation,(f)security,(g)dataservices,(h)monitoring,and(i)resourceman-
agement.Eachpartdescribesthestateoftheartmethods, aswellastheextensions
andnovelmethodologiesdevelopedrecentlyfordatacenters.
Thefirstpartdescribesenergyefficiencymethodsfordatacentersandaddresses
thedesigningofenergyefficientschedulingmechanismsforhighperformancecom-
putingenvironments.Theauthorshavediscussedenergy-awarealgorithmsfortask
graphscheduling, replicaplacement, check-pointing, numericalprocessing, andto
estimateenergyconsumptionforthegivenworkload.Also, thispartaddressesthe
problemofminimizingenergyconsumptionwithschedulelengthconstraintonmul-
ticore processors. The following chapters covers the energy efficiency aspects of
datacenterspresentedinthehandbook:(a)Energy-EfficientandHigh-Performance
v
vi Preface
ProcessingofLarge-ScaleParallelApplicationsinDataCenters,(b)Energy-aware
algorithmsfortaskgraphscheduling,replicaplacementandcheckpointstrategies,
(c) Energy efficiency in HPC Data Centers: Latest Advances to Build the Path to
Exascale,(d)Techniquestoachieveenergyproportionalityindatacenters:asurvey,
(e) A Power-Aware Autonomic Approach for Performance Management of Scien-
tificApplicationsinaDataCenterEnvironment,(f)CoolEmAll:ModelsandTools
forPlanningandOperatingEnergyEfficientDataCentres, (g)SmartDatacenter,
(h) Power and Thermal Efficient Numerical Processing, and (k) Providing Green
ServicesinHPCDataCenters:AMethodologybasedonEnergyEstimation.
Thesecondpartofthehandbookprovidesastudyofvariouscommunicationand
networkingmethodologiesfordatacenters.Thenetworkvirtualizationconcepts,the
architectureofopticalnetworksfordatacenters,andnetworkscalabilityissuesare
discussed in detail. Moreover, an emphasis is drawn over packet classification in
multicoreplatforms.Theroutingtechniquesfordatacenternetworksarediscussed
indetailalongwithstudyonTCPcongestioncontrolindatacenternetworks.The
followingchaptersprovideadetailedoverviewintermsofnetworkingtechnologies
fordatacentersareincludedinthispart:(a)NetworkVirtualizationinDataCenters:
A Data Plane Perspective, (b) Optical data center networks: Architecture, perfor-
mance,andenergyefficiency,(c)ScalableNetworkCommunicationusingUnreliable
RDMA,(d)PacketClassificationonMulti-corePlatforms,(e)OpticalInterconnects
for Data Center Networks, (f)TCP Congestion Control in Data Center Networks,
and(g)RoutingTechniquesinDataCenterNetworks.
Thethirdpartofthehandbookdiscussestheroleofdatacentersincloudcomput-
ingandhighlightsvariouschallengesfacedinensuringthedataintegrity,reliability,
and privacy in cloud computing environments. The role of trusted third parties in
performingmonitoringandauditingofservicelevelagreementsisillustrated,aswell
as integrity of big data in cloud computing is discussed. The data intensive appli-
cationsincloudarediscussedalongwithstoragechallenges.Thispartincludesthe
followingchapters:(a)AuditingforDataIntegrityandReliabilityinCloudStorage,
(b)I/OandFileSystemsforData-IntensiveApplications, (c)Cloudresourcepric-
ingundertenantrationality,(d)OnlineResourceManagementforCarbon-Neutral
Cloud Computing, (e)A Big Picture of IntegrityVerification of Big Data in Cloud
Computing,(f)AnOut-of-CoreTask-basedMiddlewareforData-IntensiveScientific
Computing,(g)BuildingScalableSoftwareforDataCenters:AnApproachtoDis-
tributedComputingatEnterpriseLevel,and(h)CloudStorageoverMultipleData
Centers.
Thefourthpartofthehandbookshedalightondatacentersemerginghardware
technologies.Theissuespertainingtotheefficientdatastorageonredundantarray
ofindependentdisks(RAID)technologies, thedatasynchronizationchallengeson
many-cores, and hardware approaches to transactional memory on chip multipro-
cessorsaredetailed.Thefollowingchaptersareincludedinthispart:(a)Realizing
Accelerated Cost-Effective Distributed RAID, (b) Efficient Hardware-Supported
Synchronization Mechanisms for Many-cores, and (c) Hardware Approaches to
TransactionalMemoryinChipMultiprocessors
Preface vii
The fifth part of the handbook discusses modeling and simulation techniques
for data centers and include following chapters: (a) Data Center Modeling and
SimulationUsingOMNeT++,(b)Power-ThermalModelingandControlofEnergy-
EfficientServersandDatacenters,(c)Thermalmodelingandmanagementofstorage
systemsindatacenters,and(d)ModelingandSimulationofDataCenterNetworks.
This portion highlights various techniques to model power consumption, thermal
response,storagesystems,andcommunicationindatacenternetworks.Theauthors
utilizeddiscrete-timesimulatorOMNet++tomodelthedatacentertraffic.
The sixth part of the handbook provides a discussion on various security and
privacytechniquesfordatacenters.Amodelisproposedtodetectandmitigatethe
covertchannelsindatacenters.Theprivacyissuesregardingdatacenteroutsourcing
are discussed and a survey of various privacy attacks and their counter measures
is presented. The following chapters are included in this portion: (a) C2Hunter:
Detection and Mitigation of Covert Channels in Data Centers, (b) Selective and
Private Access to Outsourced Data Centers, and (c) Privacy in Data Centers: A
SurveyofAttacksandCountermeasures.
The seventh part details the data services and their management in data center.
Thequalityofservicerequirementsforprocessingstreamdataforcityapplicationsis
discussed.Thedatamanagementandqueryingofbigdataissurveyedindetail.The
authors also discussed the various methods of constructing on the fly data centers
in wireless ad hoc network environments. The following chapters are included in
this part: (a) Quality-of-Service in Data Center Stream Processing for Smart City
Applications, (b) Opportunistic Databank: A context aware on-the-fly data center
for mobile networks, (c) Data Management:State-of-the-Practice at Open-Science
DataCenters,and(d)DataSummarizationTechniquesforBigData—ASurvey.
Theeighthpartofhandbookillustratesvarioushardwareandsoftware-basedmon-
itoringsolutionsfordatacenters.Theuseofwirelesssensornetworkstechnologyto
sensethethermalactivitiesandefficientcirculationofcoolingairisexplored.Traffic
monitoringisanothervitaltopicdiscussedwithasurveyonnetworkintrusiondetec-
tionsystemsfordatacenters.Thefollowingchaptersarecoveredinthisportion:(a)
CentralManagementofDatacenters,(b)MonitoringofDataCentersusingWireless
SensorNetworks,(c)NetworkIntrusionDetectionSystemsinDataCenters,and(d)
SoftwareMonitoringinDataCenters.
Lastly,theninthpartofbookchapterelaboratestheresourcemanagementaspects
of data centers and present a detailed case study on usage patterns in multi-tenant
datacenters.Adiscussionisalsoperformedonschedulingofdistributedtransactional
memoryandthevarioustradeoffsandtechniquesindistributedmemorymanagement
areelaboratedindetail.Moreover,anemphasisisdrawnontheresourcescheduling
in data-centric systems. This part includes the following chapters: (a) Usage Pat-
terns in Multi-tenant Data Centers: a Large-Case Field Study, (b) On Scheduling
inDistributedTransactionalMemory:TechniquesandTradeoffs,(c)Dependability-
OrientedResourceManagementSchemesforCloudComputingDataCenters,and
(d)ResourceSchedulinginData-CentricSystems.
Contents
PartI EnergyEfficiency
Energy-Efficient and High-Performance Processing of Large-Scale
ParallelApplicationsinDataCenters................................ 3
KeqinLi
Energy-Aware Algorithms for Task Graph Scheduling, Replica
PlacementandCheckpointStrategies................................ 37
GuillaumeAupy,AnneBenoit,PaulRenaud-GoudandYvesRobert
Energy Efficiency in HPC Data Centers: LatestAdvances to Build
thePathtoExascale ............................................... 81
SébastienVarrette,PascalBouvry,MateuszJarusandArielOleksiak
Techniques toAchieve Energy Proportionality in Data Centers:
ASurvey ......................................................... 109
MadhurimaPore,ZahraAbbasi,SandeepK.S.Gupta
andGeorgiosVarsamopoulos
A Power-AwareAutonomicApproach for Performance Management
ofScientificApplicationsinaDataCenterEnvironment ............... 163
RajatMehrotra,IoanaBanicescu,SrishtiSrivastava
andSherifAbdelwahed
CoolEmAll: Models andTools for Planning and Operating Energy
EfficientDataCentres ............................................. 191
MichavordemBerge,JochenBuchholz,LeandroCupertino,
GeorgesDaCosta,AndrewDonoghue,GeorginaGallizo,MateuszJarus,
LaraLopez,ArielOleksiak,EnricPages,WojciechPiatek,
Jean-MarcPierson,TomaszPiontek,DanielRathgeb,JaumeSalom,
LauraSisó,EugenVolk,UweWössnerandThomasZilio
ix
x Contents
SmartDataCenter ................................................ 247
MuhammadUsmanShahidKhanandSameeU.Khan
PowerandThermalEfficientNumericalProcessing ................... 263
WeiLiuandAlbertoNannarelli
ProvidingGreenServicesinHPCDataCenters:AMethodologyBased
onEnergyEstimation.............................................. 287
MohammedElMehdiDiouri,OlivierGlück,LaurentLefêvre
andJean-ChristopheMignot
PartII Networking
NetworkVirtualizationinDataCenters:ADataPlanePerspective ...... 327
WeirongJiangandViktorK.Prasanna
OpticalDataCenterNetworks:Architecture,Performance,andEnergy
Efficiency ........................................................ 351
YuichiOhsitaandMasayukiMurata
ScalableNetworkCommunicationUsingUnreliableRDMA............ 393
RyanE.Grant,MohammadJ.Rashti,PavanBalajiandAhmadAfsahi
PacketClassificationonMulti-corePlatforms ........................ 425
YunR.Qu,ShijieZhouandViktorK.Prasanna
OpticalInterconnectsforDataCenterNetworks ...................... 449
KhurramAzizandMohsinFayyaz
TCPCongestionControlinDataCenterNetworks .................... 485
RohitP.Tahiliani,MohitP.TahilianiandK.ChandraSekaran
RoutingTechniquesinDataCenterNetworks ........................ 507
ShaistaHabib,FawazS.BokhariandSameeU.Khan
PartIII CloudComputing
AuditingforDataIntegrityandReliabilityinCloudStorage ........... 535
BingweiLiuandYuChen
I/OandFileSystemsforData-IntensiveApplications .................. 561
YanlongYin,HuiJinandXian-HeSun
CloudResourcePricingUnderTenantRationality .................... 583
XinJinandYu-KwongKwok
Contents xi
OnlineResourceManagementforCarbon-NeutralCloudComputing ... 607
KishwarAhmed,ShaoleiRen,YuxiongHeandAthanasiosV.Vasilakos
A Big Picture of Integrity Verification of Big Data in Cloud
Computing ....................................................... 631
ChangLiu,RajivRanjan,XuyunZhang,ChiYangandJinjunChen
AnOut-of-CoreTask-basedMiddlewareforData-IntensiveScientific
Computing ....................................................... 647
ErikSaule,HasanMetinAktulga,ChaoYang,EsmondG.Ng
andÜmitV.Çatalyürek
Building Scalable Software for Data Centers: An Approach
toDistributedComputingatEnterpriseLevel ........................ 669
FernandoTurradoGarcía,AnaLucilaSandovalOrozco
andLuisJavierGarcíaVillalba
CloudStorageoverMultipleDataCenters ........................... 691
ShuaiMu,MaomengSu,PinGao,Yongwei Wu,KeqinLi
andAlbertY.Zomaya
PartIV Hardware
RealizingAcceleratedCost-EffectiveDistributedRAID................ 729
AleksandrKhasymski,M.MustafaRafique,AliR.Butt,
SudharshanS.VazhkudaiandDimitriosS.Nikolopoulos
Efficient Hardware-Supported Synchronization Mechanisms
forManycores .................................................... 753
JoséL.Abellán,JuanFernándezandManuelE.Acacio
Hardware Approaches to Transactional Memory in Chip
Multiprocessors................................................... 805
J.RubénTitos-GilandManuelE.Acacio
PartV ModelingandSimulation
DataCenterModelingandSimulationUsingOMNeT++.............. 839
AsadW.MalikandSameeU.Khan
Power-Thermal Modeling and Control of Energy-Efficient Servers
andDatacenters................................................... 857
JungsooKim,MohamedM.Sabry,MartinoRuggieroandDavidAtienza