Table Of Content

HWU 2011 01-fm-i-iv-9780123859631 2011/9/7 20:30 Page i #1 GPU Computing Gems Jade Edition HWU 2011 01-fm-i-iv-9780123859631 2011/9/7 20:30 Page ii #2 Morgan Kaufmann’s Applications of GPU Computing Series Computing is quickly becoming the third pillar of scientific research, due in large part to the performance gains achieved through graphics processing units (GPUs), which have become ubiquitous in handhelds,laptops,desktops,andsupercomputerclusters.MorganKaufmann’sApplicationsofGPU Computing series offers training, examples, and inspiration for researchers, engineers, students, and supercomputingprofessionalswhowanttoleveragethepowerofGPUsincorporatedintotheirsimu- lationsorexperiments.Eachhigh-quality,peer-reviewedbookiswrittenbyleadingexpertsuniquely qualifiedtoprovideparallelcomputinginsightsandguidance. EachGPUComputingGemsvolumeoffersasnapshotofthestateofparallelcomputingacrossa carefullyselectedsubsetofindustrydomains,givingyouawindowintothelead-edgeresearchoccur- ring across the breadth of science, and the opportunity to observe others’ algorithm work that might applytoyourownprojects.Findoutmoreathttp://mkp.com/gpu-computing-gems. RecommendedParallelComputingTitles ProgrammingMassivelyParallelProcessors AHands-onApproach ByDavidB.KirkandWen-meiW.Hwu ISBN:9780123814722 GPUComputingGems:EmeraldEdition Editor-in-Chief:Wen-meiW.Hwu ISBN:9780123849885 TheArtofMultiprocessorProgramming ByMauriceHerlihyandNirShavit ISBN:9780123705914 AnIntroductiontoParallelProgramming ByPeterPacheco ISBN:9780123742605 HeterogeneousComputingwithOpenCL ByBenedictR.Gaster,LeeHowes,DavidR.Kaeli,PerhaadMistry,DanaSchaa ISBN:9780123877666 CUDAApplicationDesignandDevelopment ByRobFarber ISBN:9780123884268 ComingWinter2011 DistributedandCloudComputing ByKaiHwang,GeoffreyFox,andJackDongarra ISBN:9780123858801 ComingFall2011 HWU 2011 01-fm-i-iv-9780123859631 2011/9/7 20:30 Page iii #3 GPU Computing Gems Jade Edition Wen-mei W. Hwu AMSTERDAM•BOSTON•HEIDELBERG•LONDON NEWYORK•OXFORD•PARIS•SANDIEGO SANFRANCISCO•SINGAPORE•SYDNEY•TOKYO MorganKaufmannPublishersisanimprintofElsevier HWU 2011 01-fm-i-iv-9780123859631 2011/9/5 15:22 Page iv #4 AcquiringEditor:ToddGreen DevelopmentEditor:RobynDay ProjectManager:PaulGottehrer Designer:DennisSchaefer MorganKaufmannisanimprintofElsevier 225WymanStreet,Waltham,MA02451,USA (cid:13)c 2012Elsevier,Inc.Allrightsreserved. Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicor mechanical,includingphotocopying,recording,oranyinformationstorageandretrievalsystem,without permissioninwritingfromthepublisher.Detailsonhowtoseekpermission,furtherinformationaboutthe Publisher’spermissionspolicies,andourarrangementswithorganizationssuchastheCopyrightClearance CenterandtheCopyrightLicensingAgencycanbefoundatourwebsite:www.elsevier.com/permissions. ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher (otherthanasmaybenotedherein). Notices Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperience broadenourunderstanding,changesinresearchmethodsorprofessionalpractices,maybecome necessary. Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluating andusinganyinformationormethodsdescribedherein.Inusingsuchinformationormethodstheyshouldbe mindfuloftheirownsafetyandthesafetyofothers,includingpartiesforwhomtheyhaveaprofessional responsibility. Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assume anyliabilityforanyinjuryand/ordamagetopersonsorpropertyasamatterofproductsliability, negligenceorotherwise,orfromanyuseoroperationofanymethods,products,instructions,orideas containedinthematerialherein. LibraryofCongressCataloging-in-PublicationData Applicationsubmitted BritishLibraryCataloguing-in-PublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary. ISBN:978-0-12-385963-1 ForinformationonallMKpublicationsvisitour websiteatwww.mkp.com PrintedintheUnitedStatesofAmerica 11 12 13 14 15 10 9 8 7 6 5 4 3 2 1 HWU 2011 03-loc-ix-xiv-9780123859631 2011/9/7 12:54 Page ix #1 Editors, Reviewers, and Authors Editor-In-Chief Wen-MeiW.Hwu,UniversityofIllinoisatUrbana-Champaign Managing Editor AndrewSchuh,UniversityofIllinoisatUrbana-Champaign NVIDIA Editor NadeemMohammad,NVIDIACorporation Area Editors ThomasBradley,NVIDIACorporation(Section5) FrankJargstorff,NVIDIACorporation(Section2) PauliusMicikevicius,NVIDIACorporation(Section1) RichardTonge,NVIDIACorporation(Section4) PengWang,NVIDIACorporation(Section3) CliffWooley,NVIDIACorporation(Section6) Reviewers JohnAshley,NVIDIACorporation NathanBell,NVIDIACorporation AviBleiweiss,NVIDIACorporation JonathanCohen,NVIDIACorporation AndrewCorrigan,NavalResearchLab MikeGiles,OxfordUniversity DilipSequeira,NVIDIACorporation DavidSullins,NVIDIACorporation StanimireTomov,UniversityofTennessee,Knoxville CliffWooley,NVIDIACorporation GernotZiegler,NVIDIACorporation ix HWU 2011 03-loc-ix-xiv-9780123859631 2011/9/7 12:54 Page x #2 x Editors, Reviewers, and Authors Authors EmmanuelAgullo,INRIA(Chapter34) DanA.Alcantara,UCDavis(Chapter4) Je´re´mieAllard,INRIALille&LilleUniversity(Chapter21) NinaAmenta,UCDavis(Chapter4) MihaiAnitescu,ArgonneNationalLaboratory(Chapter20) V.G.Asouti,NationalTechnicalUniversityofAthens(Chapter17) Ce´dricAugonnet,INRIA(Chapter34) JamesBalasalle,UniversityofDenver(Chapter6) NathanBell,NVIDIACorporation(Chapter26) ThomasBradley,NVIDIACorporation(Chapter25) TobiasBrandvik,UniversityofCambridge(Chapter14) GiuseppeCaggianese,Universita` degliStudidellaBasilicata(Chapter22) BryanCatanzaro,UCBerkeley(Chapter27) CrisCecka,StanfordUniversity(Chapter16) DanielCederman,SwedenChalmersUniversityofTechnology(Chapter35) RuinanChang,UCSanDiego(Chapter19) JikeChong,ParasiansLLC&UCBerkeley(Chapter25) JonathanCohen,NVIDIACorporation(Chapters11,32) SylvainCollange,Ećolenormalesupe´rieuredeLyon(Chapter9) HadrienCourtecuisse,INRIALille&LilleUniversity(Chapter21) DavidD.Cox,HarvardUniversity(Chapter33) EricDarve,StanfordUniversity(Chapter16) MarcDaumas,Universite´ dePerpignan(Chapter9) AndrewA.Davidson,UCDavis(Chapter11) DavidDefour,Universite´ dePerpignan(Chapter9) GregoryDiamos,GeorgiaInstituteofTechnology(Chapter30) AndreaDiBlas,OracleCorporation(Chapter1) MatthewF.Dixon,UCDavis(Chapter25) JackDongarra,UniversityofTennessee,Knoxville(Chapter34) PeterEastman,StanfordUniversity(Chapter29) DanielEgloff,QuantAleaGmbH(Chapter23) HWU 2011 03-loc-ix-xiv-9780123859631 2011/9/7 12:54 Page xi #3 Editors, Reviewers, and Authors xi UgoErra,Universita` degliStudidellaBasilicata(Chapter22) AhmedFasih,OhioStateUniversity(Chapter27) FrançoisFaure,INRIARhońe-Alpes&GrenobleUniversity(Chapter21) AldoFrezzotti,PolitecnicodiMilano(Chapter15) MichaelGarland,NVIDIACorporation(Chapters2and3) GianPietroGhiroldi,PolitecnicodiMilano(Chapter15) K.C.Giannakoglou,NationalTechnicalUniversityofAthens(Chapter17) LivioGibelli,PolitecnicodiMilano(Chapter15) MikeGiles,OxfordUniversity(Chapter10) PawanHarish,InternationalInstituteofInformationTechnology,Hyderabad(Chapter7) MarkHarris,NVIDIACorporation(Chapter3) JohnC.Hart,UniversityofIllinoisatUrbana-Champaign(Chapter2) ZhengyuHe,GeorgiaInstituteofTechnology(Chapter5) JanS.Hesthaven,BrownUniversity(Chapter18) TobyHeyn,UniveristyofWisconsin,Madison(Chapter20) JaredHoberock,NVIDIACorporation(Chapters2and26) BoHong,GeorgiaInstituteofTechnology(Chapter5) JohnHumphrey,EMPhotonics,Inc.(Chapter12) PaulIvanov,UCBerkeley(Chapter27) YuntaoJia,UniversityofIllinoisatUrbana-Champaign(Chapter2) TimKaldewey,IBM(Chapter1) I.C.Kampolis,NationalTechnicalUniversityofAthens(Chapter17) EricKelmelis,EMPhotonics,Inc.(Chapter12) AndrewKerr,GeorgiaInstituteofTechnology(Chapter30) KurtKeutzer,UCBerkeley(Chapter25) AndreasKlo¨ckner,NewYorkUniversity(Chapters18and27) TorbenLarsen,AalborgUniversity(Chapter28) YunsupLee,UCBerkeley(Chapter27) AdrianLew,StanfordUniversity(Chapter16) ShaojingLi,UCSanDiego(Chapter19) VitaliyLomakin,UCSanDiego(Chapter19) MarioA.Lopez,UniversityofDenver(Chapter6) HWU 2011 03-loc-ix-xiv-9780123859631 2011/9/7 12:54 Page xii #4 xii Editors, Reviewers, and Authors HatemLtaief,UniversityofTennessee,Knoxville(Chapter34) VictorLu,UniversityofIllinoisatUrbana-Champaign(Chapter2) JamesMalcolm,AccelerEyes(Chapter28) HammadMazhar,UniversityofWisconsin,Madison(Chapter20) MichaelMitzenmacher,HarvardUniversity(Chapter4) RaymondNamyst,INRIA(Chapter34) P.J.Narayanan,InternationalInstituteofInformationTechnology,Hyderabad(Chapter7) DanNegrut,UniversityofWisconsin,Madison(Chapter20) JohnD.Owens,UCDavis(Chapters4,11,36) VijayPande,StanfordUniversity(Chapter29) SuryakantPatidar,NVIDIACorporation(Chapter7) ArmanPazouki,UniveristyofWisconsin,Madison(Chapter20) HagenPeters,Christian-Albrechts-UniversityofKiel(Chapter8) NicolasPinto,MassachusettsInstituteofTechnology(Chapters27and33) DanielPrice,EMPhotonics,Inc.(Chapter12) GallagherPryor,AccelerEyes(Chapter28) GrahamPullan,UniversityofCambridge(Chapter14) SimonJ.Rees,BarclaysCapital(Chapter24) MatthewJ.Rutherford,UniversityofDenver(Chapter6) OleSchulz-Hildebrandt,Christian-Albrechts-UniversityofKiel(Chapter8) AssafSchuster,Technion(Chapter36) ShubhabrataSengupta,UCDavis(Chapter4) EricShaffer,UniversityofIllinoisatUrbana-Champaign(Chapter13) MarkSilberstein,Technion(Chapter36) KyleSpagnoli,EMPhotonics,Inc.(Chapter12) RobertStrzodka,MaxPlanckInstitutInformatik(Chapter31) AlessandroTasora,UniversityofParma(Chapter20) SamuelThibault,INRIA(Chapter34) StanimireTomov,UniversityofTennessee,Knoxville(Chapter34) X.S.Trompoukis,NationalTechnicalUniversityofAthens(Chapter17) PhilippasTsigas,SwedenChalmersUniversityofTechnology(Chapter35) VibhavVineet,InternationalInstituteofInformationTechnology,Hyderabad(Chapter7) HWU 2011 03-loc-ix-xiv-9780123859631 2011/9/7 12:54 Page xiii #5 Editors, Reviewers, and Authors xiii VasilyVolkov,UCBerkeley(Chapter4) JosephWalkenhorst,BarclaysCapital(Chapter24) TimothyWarburton,RiceUniversity(Chapter18) JiadongWu,GeorgiaInstituteofTechnology(Chapter5) SudhakarYalamanchili,GeorgiaInstituteofTechnology(Chapter30) GeorgeZagaris,IllinoisRocstarLLC(Chapter13) YaoZhang,UCDavis(Chapter11) HWU 2011 04-int-xv-xvi-9780123859631 2011/9/7 14:44 Page xv #1 Introduction Wen-meiW.Hwu STATE OF GPU COMPUTING WeareinthegoldenageoftheGPUcomputing.SincetheintroductionofCUDAin2007,morethan 100 million computers with CUDA capable GPUs have been shipped to end users. GPU computing applicationdeveloperscannowexpecttheirapplicationtohaveamassmarket.Withtheintroduction of OpenCL in 2010, researchers can now expect to develop GPU applications that can run on hard- warefrommultiplevendors.Furthermore,frommyownexperienceinteachingCUDAandOpenCL programming,Cprogrammerscanbegintowritebasicprogramsafteronlyattendingonelectureand readingonetextbookchapter.Withsuchalowbarrierofentry,researchersallovertheworldhavebeen engagedindevelopingnewalgorithmsandapplicationstotakeadvantageoftheextremefloatingpoint executionthroughputoftheseGPUs. Today,thereisalargecommunityofGPUcomputingpractitioners.Manyofthemhavereported10 to100timesspeedupoftheirapplicationswithGPUcomputing.Toputthisintoperspective,withthe historical2×performancegrowtheverytwoyears,theseresearchersareexperiencingtheequivalent of time travel of 8 to 12 years. That is, they are getting today the performance they would have to waitfor8to12yearsiftheywentforthe“free-ride”advancementofperformanceinmicroprocessors. Interestingly,suchafreerideisnolongeravailable.Furthermore,oncetheydevelopedtheirapplication inCUDA,theywilllikelyseecontinuedperformancegrowthof2×foreverytwoyearsfromthisday forward. Afterdiscussingwithnumerousresearchers,Ireachedtheconclusionthatmanyofthemaresolv- ing similar algorithm problems in their programming efforts. Although they are working on diverse applications,theyoftenendupdevelopingsimilaralgorithmicstrategies.TheideaofGPUComputing Gemsistoprovideaconvenientmeansforapplicationdevelopersindiverseapplicationareastoben- efitfromeachother’sexperience.WhenweissuedthecallforproposalsforthefirstGPUComputing Gems,wereceivedmorethan280submissions,anoverwhelmingresponse.Intheend,weaccepted86 finalchapters.Manyhigh-qualityproposalswerenotacceptedbecauseofconcernsthattheymaynot beaccessibletoalargeaudience.Withsomanychapters,wewereforcedtodividethesegemsintotwo volumes. In the first volume (Emerald Edition), we published 50 Gems from 10 diverse application areas.Inthisvolume,wehavecollected36Gemarticleswrittenbyresearchersin5diverseapplication areasaswellasprogrammingtoolsandtechniques.Eachgemisfirsteditedbyanareaeditorwhoisa GPUcomputingexpertinthatarea.Thisisfollowedbymyowneditingofthesearticles. Forapplications,eachGemsarticlereportsasuccessfulapplicationexperienceinGPUcomputing. These articles describe the techniques or “secret sauce” that contributed to the success. The authors highlight the potential applicability of their techniques to other application areas. In our editorial xv