Compiling Algorithms for Heterogeneous Systems

105 Pages·2018·5.357 MB·English

by  Steven Bell, Jing Pu, James Hegarty

Compiling Algorithms
for Heterogeneous Systems

Synthesis Lectures on
Computer Architecture
Editor
MargaretMartonosi,PrincetonUniversity
FoundingEditorEmeritus
MarkD.Hill,UniversityofWisconsin,Madison
SynthesisLecturesonComputerArchitecturepublishes50-to100-pagepublicationsontopics
pertainingtothescienceandartofdesigning,analyzing,selectingandinterconnectinghardware
componentstocreatecomputersthatmeetfunctional,performanceandcostgoals.Thescopewill
largelyfollowthepurviewofpremiercomputerarchitectureconferences,suchasISCA,HPCA,
MICRO,andASPLOS.
CompilingAlgorithmsforHeterogeneousSystems
StevenBell,JingPu,JamesHegarty,andMarkHorowitz
2018
ArchitecturalandOperatingSystemSupportforVirtualMemory
AbhishekBhattacharjeeandDanielLustig
2017
DeepLearningforComputerArchitects
BrandonReagen,RobertAdolf,PaulWhatmough,Gu-YeonWei,andDavidBrooks
2017
On-ChipNetworks,SecondEdition
NatalieEnrightJerger,TusharKrishna,andLi-ShiuanPeh
2017
Space-TimeComputingwithTemporalNeuralNetworks
JamesE.Smith
2017
HardwareandSoftwareSupportforVirtualization
EdouardBugnion,JasonNieh,andDanTsafrir
2017
iv
DatacenterDesignandManagement:AComputerArchitect’sPerspective
BenjaminC.Lee
2016
APrimeronCompressionintheMemoryHierarchy
SomayehSardashti,AngelosArelakis,PerStenström,andDavidA.Wood
2015
ResearchInfrastructuresforHardwareAccelerators
YakunSophiaShaoandDavidBrooks
2015
AnalyzingAnalytics
RajeshBordawekar,BobBlainey,andRuchirPuri
2015
CustomizableComputing
Yu-TingChen,JasonCong,MichaelGill,GlennReinman,andBingjunXiao
2015
Die-stackingArchitecture
YuanXieandJishenZhao
2015
Single-InstructionMultiple-DataExecution
ChristopherJ.Hughes
2015
Power-EfficientComputerArchitectures:RecentAdvances
MagnusSjälander,MargaretMartonosi,andStefanosKaxiras
2014
FPGA-AcceleratedSimulationofComputerSystems
HariAngepat,DerekChiou,EricS.Chung,andJamesC.Hoe
2014
APrimeronHardwarePrefetching
BabakFalsafiandThomasF.Wenisch
2014
On-ChipPhotonicInterconnects:AComputerArchitect’sPerspective
ChristopherJ.Nitta,MatthewK.Farrens,andVenkateshAkella
2013
v
OptimizationandMathematicalModelinginComputerArchitecture
TonyNowatzki,MichaelFerris,KarthikeyanSankaralingam,CristianEstan,NilayVaish,and
DavidWood
2013
SecurityBasicsforComputerArchitects
RubyB.Lee
2013
TheDatacenterasaComputer:AnIntroductiontotheDesignofWarehouse-Scale
Machines,Secondedition
LuizAndréBarroso,JimmyClidaras,andUrsHölzle
2013
Shared-MemorySynchronization
MichaelL.Scott
2013
ResilientArchitectureDesignforVoltageVariation
VijayJanapaReddiandMeetaSharmaGupta
2013
MultithreadingArchitecture
MarioNemirovskyandDeanM.Tullsen
2013
PerformanceAnalysisandTuningforGeneralPurposeGraphicsProcessingUnits
(GPGPU)
HyesoonKim,RichardVuduc,SaraBaghsorkhi,JeeChoi,andWen-meiHwu
2012
AutomaticParallelization:AnOverviewofFundamentalCompilerTechniques
SamuelP.Midkiff
2012
PhaseChangeMemory:FromDevicestoSystems
MoinuddinK.Qureshi,SudhanvaGurumurthi,andBipinRajendran
2011
Multi-CoreCacheHierarchies
RajeevBalasubramonian,NormanP.Jouppi,andNaveenMuralimanohar
2011
APrimeronMemoryConsistencyandCacheCoherence
DanielJ.Sorin,MarkD.Hill,andDavidA.Wood
2011
vi
DynamicBinaryModification:Tools,Techniques,andApplications
KimHazelwood
2011
QuantumComputingforComputerArchitects,SecondEdition
TzvetanS.Metodi,ArvinI.Faruque,andFredericT.Chong
2011
HighPerformanceDatacenterNetworks:Architectures,Algorithms,andOpportunities
DennisAbtsandJohnKim
2011
ProcessorMicroarchitecture:AnImplementationPerspective
AntonioGonzález,FernandoLatorre,andGrigoriosMagklis
2010
TransactionalMemory,2ndedition
TimHarris,JamesLarus,andRaviRajwar
2010
ComputerArchitecturePerformanceEvaluationMethods
LievenEeckhout
2010
IntroductiontoReconfigurableSupercomputing
MarcoLanzagorta,StephenBique,andRobertRosenberg
2009
On-ChipNetworks
NatalieEnrightJergerandLi-ShiuanPeh
2009
TheMemorySystem:YouCan’tAvoidIt,YouCan’tIgnoreIt,YouCan’tFakeIt
BruceJacob
2009
FaultTolerantComputerArchitecture
DanielJ.Sorin
2009
TheDatacenterasaComputer:AnIntroductiontotheDesignofWarehouse-Scale
Machines
LuizAndréBarrosoandUrsHölzle
2009
vii
ComputerArchitectureTechniquesforPower-Efficiency
StefanosKaxirasandMargaretMartonosi
2008
ChipMultiprocessorArchitecture:TechniquestoImproveThroughputandLatency
KunleOlukotun,LanceHammond,andJamesLaudon
2007
TransactionalMemory
JamesR.LarusandRaviRajwar
2006
QuantumComputingforComputerArchitects
TzvetanS.MetodiandFredericT.Chong
2006
Copyright©2018byMorgan&Claypool
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmittedin
anyformorbyanymeans—electronic,mechanical,photocopy,recording,oranyotherexceptforbriefquotations
inprintedreviews,withoutthepriorpermissionofthepublisher.
CompilingAlgorithmsforHeterogeneousSystems
StevenBell,JingPu,JamesHegarty,andMarkHorowitz
www.morganclaypool.com
ISBN:9781627059619 paperback
ISBN:9781627057301 ebook
ISBN:9781681732633 hardcover
DOI10.2200/S00816ED1V01Y201711CAC043
APublicationintheMorgan&ClaypoolPublishersseries
SYNTHESISLECTURESONCOMPUTERARCHITECTURE
Lecture#43
SeriesEditor:MargaretMartonosi,PrincetonUniversity
FoundingEditorEmeritus:MarkD.Hill,UniversityofWisconsin,Madison
SeriesISSN
Print1935-3235 Electronic1935-3243
Compiling Algorithms
for Heterogeneous Systems
Steven Bell
StanfordUniversity
Jing Pu
Google
James Hegarty
Oculus
Mark Horowitz
StanfordUniversity
SYNTHESISLECTURESONCOMPUTERARCHITECTURE#43
M
&C Morgan&cLaypool publishers
ABSTRACT
Mostemergingapplicationsinimagingandmachinelearningmustperformimmenseamounts
ofcomputationwhileholdingtostrictlimitsonenergyandpower.Tomeetthesegoals,archi-
tectsarebuildingincreasinglyspecializedcomputeenginestailoredforthesespecifictasks.The
resultingcomputersystemsareheterogeneous,containingmultipleprocessingcoreswithwildly
differentexecutionmodels.Unfortunately,thecostofproducingthisspecializedhardware—and
the software to control it—is astronomical. Moreover, the task of porting algorithms to these
heterogeneousmachinestypicallyrequiresthatthealgorithmbepartitionedacrossthemachine
andrewrittenforeachspecificarchitecture,whichistimeconsumingandpronetoerror.
Over the last several years, the authors have approached this problem using domain-
specificlanguages(DSLs):high-levelprogramminglanguagescustomizedforspecificdomains,
such as database manipulation, machine learning, or image processing. By giving up general-
ity,theselanguagesareabletoprovidehigh-levelabstractionstothedeveloperwhileproducing
high-performanceoutput.Thepurposeofthisbookistospurtheadoptionandthecreationof
domain-specificlanguages,especiallyforthetaskofcreatinghardwaredesigns.
Inthefirstchapter,ashorthistoricaljourneyexplainstheforcesdrivingcomputerarchi-
tecture today. Chapter 2 describes the various methods for producing designs for accelerators,
outliningthepushformoreabstractionandthetoolsthatenabledesignerstoworkatahigher
conceptual level. From there, Chapter 3 provides a brief introduction to image processing al-
gorithmsandhardwaredesignpatternsforimplementingthem.Chapters4and5describeand
compare Darkroom and Halide, two domain-specific languages created for image processing
thatproducehigh-performancedesignsforbothFPGAsandCPUsfromthesamesourcecode,
enabling rapid design cycles and quick porting of algorithms. The final section describes how
theDSLapproachalsosimplifiestheproblemofinterfacingbetweenapplicationcodeandthe
acceleratorbygeneratingthedriverstackinadditiontotheacceleratorconfiguration.
Thisbookshouldserveasausefulintroductiontodomain-specializedcomputingforcom-
puterarchitecturestudentsandasaprimerondomain-specificlanguagesandimageprocessing
hardwareforthosewithmoreexperienceinthefield.
KEYWORDS
domain-specificlanguages,high-levelsynthesis,compilers,imageprocessingaccel-
erators,stencilcomputation