Parallel Computing Hits the Power Wall: Principles, Challenges, and a Survey of Solutions

99 Pages·2019·3.208 MB·English

by  Arthur Francisco Lorenzon, Antonio Carlos Schneider Beck Filho

SPRINGER BRIEFS IN COMPUTER SCIENCE
Arthur Francisco Lorenzon
Antonio Carlos Schneider Beck Filho
Parallel
Computing Hits
the Power Wall
Principles,
Challenges, and a
Survey of Solutions
123
SpringerBriefs in Computer Science
Serieseditors
StanZdonik,BrownUniversity,Providence,RI,USA
ShashiShekhar,UniversityofMinnesota,Minneapolis,MN,USA
XindongWu,UniversityofVermont,Burlington,VT,USA
LakhmiC.Jain,UniversityofSouthAustralia,Adelaide,SA,Australia
DavidPadua,UniversityofIllinoisUrbana-Champaign,Urbana,IL,USA
XueminShermanShen,UniversityofWaterloo,Waterloo,ON,Canada
BorkoFurht,FloridaAtlanticUniversity,BocaRaton,FL,USA
V.S.Subrahmanian,DepartmentofComputerScience,UniversityofMaryland,
CollegePark,MD,USA
MartialHebert,CarnegieMellonUniversity,Pittsburgh,PA,USA
KatsushiIkeuchi,Meguro-ku,UniversityofTokyo,Tokyo,Japan
BrunoSiciliano,DipartimentodiIngegneriaElettricaedelleTecnologie
dell’Informazione,UniversitàdiNapoliFedericoII,Napoli,Italy
SushilJajodia,GeorgeMasonUniversity,Fairfax,VA,USA
NewtonLee,InstituteforEducationResearchandScholarships,LosAngeles,
CA,USA
SpringerBriefs present concise summaries of cutting-edge research and practical
applicationsacrossawidespectrumoffields.Featuringcompactvolumesof50to
125pages,theseriescoversarangeofcontentfromprofessionaltoacademic.
Typicaltopicsmightinclude:
(cid:129) Atimelyreportofstate-of-theartanalyticaltechniques
(cid:129) A bridge between new research results, as published in journal articles, and a
contextualliteraturereview
(cid:129) Asnapshotofahotoremergingtopic
(cid:129) Anin-depthcasestudyorclinicalexample
(cid:129) Apresentationofcoreconceptsthatstudentsmustunderstandinordertomake
independentcontributions
Briefsallowauthorstopresenttheirideasandreaderstoabsorbthemwithminimal
time investment. Briefs will be published as part of Springer’s eBook collection,
withmillionsofusersworldwide.Inaddition,Briefswillbeavailableforindividual
print and electronic purchase. Briefs are characterized by fast, global electronic
dissemination, standard publishing contracts, easy-to-use manuscript preparation
and formatting guidelines, and expedited production schedules. We aim for pub-
lication 8–12 weeks after acceptance. Both solicited and unsolicited manuscripts
areconsideredforpublicationinthisseries.
Moreinformationaboutthisseriesathttp://www.springer.com/series/10028
Arthur Francisco Lorenzon
Antonio Carlos Schneider Beck Filho
Parallel Computing Hits
the Power Wall
Principles, Challenges, and a Survey
of Solutions
123
ArthurFranciscoLorenzon AntonioCarlosSchneiderBeckFilho
DepartmentofComputerScience InstituteofInformatics,CampusdoVale
FederalUniversityofPampa(UNIPAMPA) FederalUniversityofRioGrande
Alegrete,RioGrandedoSul,Brazil doSul(UFRGS)
PortoAlegre,RioGrandedoSul,Brazil
ISSN2191-5768 ISSN2191-5776 (electronic)
SpringerBriefsinComputerScience
ISBN978-3-030-28718-4 ISBN978-3-030-28719-1 (eBook)
https://doi.org/10.1007/978-3-030-28719-1
©TheAuthor(s),underexclusivelicensetoSpringerNatureSwitzerlandAG2019
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse
ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG.
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Thisbookisdedicatedtothememoryof
MárciaCristinaandAuroraCera.
Preface
Efficiently exploiting thread-level parallelism from modern multicore systems has
been challenging for software developers. While blindly increasing the number
of threads may lead to performance gains, it can also result in a disproportionate
increase in energy consumption. In the same way, optimization techniques for
reducing energy consumption, such as DVFS and power gating, can lead to
huge performance loss if used incorrectly. In this book, we present and discuss
several techniques that address these challenges. We start by providing a brief
theoreticalbackgroundonparallelcomputinginsoftwareandthesourcesofpower
consumption. Then, we show how different parallel programming interfaces and
communicationmodelsmayaffectenergyconsumptionindifferentways.Next,we
discuss tuning techniques to adapt the number of threads/operating frequency to
achievethebestcompromisebetweenperformanceandenergy.Wefinishthisbook
withadetailedanalysisofarepresentativeexampleofanadaptiveapproach.
Alegrete,Brazil ArthurFranciscoLorenzon
PortoAlegre,Brazil AntonioCarlosSchneiderBeckFilho
vii
Acknowledgments
TheauthorswouldliketothankthefriendsandcolleaguesatInformaticsInstitute
oftheFederalUniversityofRioGrandedoSulandgiveaspecialthankstoallthe
peopleintheEmbeddedSystemsLaboratory,whohavecontributedtothisresearch
since2013.
The authors would also like to thank the Brazilian research support agencies,
FAPERGS,CAPES,andCNPq.
ix
Contents
1 RuntimeAdaptability:TheKeyforImprovingParallelApplications.. 1
1.1 Introduction .............................................................. 1
1.2 ScalabilityAnalysis...................................................... 3
1.2.1 VariablesInvolved............................................... 5
1.3 ThisBook ................................................................ 7
2 FundamentalConcepts....................................................... 9
2.1 ParallelComputinginSoftware......................................... 9
2.1.1 CommunicationModels......................................... 9
2.1.2 ParallelProgrammingInterfaces................................ 10
2.1.3 MulticoreArchitectures ......................................... 12
2.2 PowerandEnergyConsumption........................................ 13
2.2.1 DynamicVoltageandFrequencyScaling....................... 14
2.2.2 PowerGating..................................................... 15
3 TheImpactofParallelProgrammingInterfacesonEnergy............. 17
3.1 Methodology ............................................................. 17
3.1.1 Benchmarks...................................................... 17
3.1.2 MulticoreArchitectures ......................................... 19
3.1.3 ExecutionEnvironment.......................................... 20
3.1.4 Setup ............................................................. 22
3.2 Results.................................................................... 23
3.2.1 PerformanceandEnergyConsumption......................... 23
3.2.2 Energy-DelayProduct........................................... 30
3.2.3 TheInfluenceoftheStaticPowerConsumption............... 34
3.3 Discussion................................................................ 38
4 TuningParallelApplications................................................. 41
4.1 DesignSpaceExplorationofOptimizationTechniques................ 41
4.2 DynamicConcurrencyThrottling....................................... 42
4.2.1 ApproacheswithnoRuntimeAdaptation
andnoTransparency............................................. 43
4.2.2 ApproacheswithRuntimeAdaptationand/orTransparency .. 45
xi
xii Contents
4.3 DynamicVoltageandFrequencyScaling............................... 49
4.3.1 ApproacheswithnoRuntimeAdaptation
andnoTransparency............................................. 49
4.3.2 ApproacheswithRuntimeAdaptationand/orTransparency .. 50
4.4 DCTandDVFS .......................................................... 51
4.4.1 ApproacheswithnoRuntimeAdaptation
andnoTransparency............................................. 51
4.4.2 ApproacheswithRuntimeAdaptationand/orTransparency .. 52
5 CaseStudy:DCTwithAurora .............................................. 55
5.1 TheNeedforAdaptabilityandTransparency .......................... 55
5.2 Aurora:SeamlessOptimizationofOpenMPApplications............. 56
5.2.1 IntegrationtoOpenMP.......................................... 56
5.2.2 SearchAlgorithm................................................ 60
5.3 EvaluationofAurora..................................................... 63
5.3.1 Methodology..................................................... 63
5.3.2 Results............................................................ 66
6 Conclusions.................................................................... 79
References.......................................................................... 81