Michael M. Resch Yevgeniya Kovalenko Wolfgang Bez · Erich Focht Hiroaki Kobayashi Editors Sustained Simulation Performance 2018 2019 1 23 Sustained Simulation Performance 2018 and 2019 Michael M. Resch • Yevgeniya Kovalenko (cid:129) Wolfgang Bez (cid:129) Erich Focht (cid:129) Hiroaki Kobayashi Editors Sustained Simulation Performance 2018 and 2019 Proceedings of the Joint Workshops on Sustained Simulation Performance, University of Stuttgart (HLRS) and Tohoku University, 2018 and 2019 Editors MichaelM.Resch YevgeniyaKovalenko HighPerformanceComputingCenter HighPerformanceComputing (HLRS) UniversityofStuttgart UniversityofStuttgart Stuttgart,Germany Stuttgart,Germany WolfgangBez ErichFocht EuropeGmbH EuropeGmbH NECHighPerformanceComputing NECHighPerformanceComputing Du¨sseldorf,Germany Stuttgart,Germany HiroakiKobayashi CyberscienceCenter TohokuUniversity Sendai,Japan ISBN978-3-030-39180-5 ISBN978-3-030-39181-2 (eBook) https://doi.org/10.1007/978-3-030-39181-2 Mathematics Subject Classification (2010): 65-XX, 65Exx, 65Fxx, 65Kxx, 68-XX, 68Mxx, 68Uxx, 68Wxx,70-XX,70Fxx,70Gxx,76-XX,76Fxx,76Mxx,92-XX,92Cxx ©SpringerNatureSwitzerlandAG2020 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation, broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional claimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG. Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland Preface Sustained simulation performance is becoming an ever more important issue in High Performance Computing (HPC). Hardware is moving towards the Exaflop, and we will see such systems in the near future in China, Europe, Japan, and the USA.However,sustainedperformanceislaggingbehindsubstantially.Expertsare worried that the level of sustained performance will stay as low as 1% of peak performancefortypicalapplications. The workshopseries on sustained simulation performancehas set out15 years ago to tackle this problem. The papers presented here are hence looking into a variety of issues that have an impact on sustained simulation performance in HPC. The starting point for any such investigation is hardware architecture. The key problem of modern HPC systems is the lack of speed in communication mainly for the main memory. The currently only vector architecture which has the potential to overcome this problem in HPC is the NEC Aurora. Several of the articles in this volume refer to this architecture and its potential for HPC simulation. Based on an excellent architecture, basic software plays a vital role. This includes a variety of topics like operating systems, compilers, schedulers, IO-systems, and programming models. Hardware and software are important for sustained performance, but in the end it is mathematical algorithms that have to be implementedand hence finally decide on how well hardware and software are used. In the coming decades, the optimization of mathematical algorithms might replace Moore’sLaw as the main driving force in sustained performanceon HPC systems. Thecontributionsinthisvolumeshowthatthenumberofproblemsinsustained simulationperformanceishigh.Somesolutionscanbeseenbutformanyproblems we still have to investa lot of research.However,if HPC in generaland Exaflops systems in particular want to be successful in the coming decades the focus of attentionwill haveto shiftfromhardwareto software andalgorithmsandfunding willhavetogointothesefieldsinordertomakesureHPCsystemsdonotbecome v vi Preface heroeswith feet of clay. This book aims to make a contributionnot only to make readers aware of the problem but also to put some potential solutions on the table. Stuttgart,Germany MichaelM.Resch September2019 Contents PartI FutureHPCChallenges R&DofaQuantum-AnnealingAssistedNextGenerationHPC InfrastructureandItsKillerApplications .................................... 3 HiroakiKobayashi MasteringExascaleChallengesforEngineeringApplications.............. 13 BastianKoller,RalfSchneider,AndreasRuopp,andDimitrisLiparas SomeThoughtsonProcessorandHPCHardwareTechnology............. 23 ThomasBönisch PartII PerformanceAnalysisandOptimizationonModernHPC Systems Overall Project View at HLRS as the Basis for Optimizing Applications....................................................................... 33 BjörnDick,ThomasBeisel,andManuelaWossough OntheDetectionandInterpretationofPerformanceVariations ofHPCApplications............................................................. 41 DennisHoppe,LiZhong,StefanAndersson,andDianaMoise Using the NEC Aurora TSUBASA for High-Order Discontinuous-GalerkininAteles .............................................. 57 HaraldKlimachandSabineRoller Performance Evaluation of SX-Aurora TSUBASA by Using BenchmarkPrograms ........................................................... 69 KazuhikoKomatsuandHiroakiKobayashi OptimizedCOAWSTfortheVectorSupercomputerSX-ACE.............. 79 ShivanshuKumar Singh, Kota Sakakura, Sourav Saha, Koji Goto, RaghunandanMathur,OsamuWatanabe,andAkihiroMusa vii viii Contents PartIII Techniques andToolsforNew-GenerationComputing Systems VEO and PyVEO: Vector Engine Offloading for the NEC SX-AuroraTsubasa.............................................................. 95 ErichFocht PotentialofLLVMforSX-Aurora ............................................. 111 SimonMoll,MatthiasKurtenacker,andSebastianHack BadNodesConsideredHarmful:HowtoFindandFixtheProblem ...... 123 Marco Seiz, Johannes Hötzer, Henrik Hierl, Stefan Andersson, andBrittaNestler vTorque:IntroducingVirtualizationCapabilitiestoTorque................ 131 NicoStruckmann IntegratingSDN-EnhancedMPIwithJobSchedulertoSupport SharedClusters................................................................... 149 KeichiTakahashi,SusumuDate,YasuhiroWatashiba,YoshiyukiKido, andShinjiShimojo PartIV LoadBalancingProblemsinHPCSimulations A Method to Reduce Load Imbalances in Simulations ofSolidificationProcesseswithFreeSurface3D ............................. 163 Johannes Müller, Philipp Offenhäuser, Martin Reitzle, andBernhardWeigand LoadBalancingforImmersedBoundariesinCoupledSimulations ....... 185 NedaEbrahimiPour,VerenaKrupp,HaraldKlimach,andSabineRoller PartV ApplicationsandNumericalMethods forLarge-ScaleSystems LargeScaleAgentBasedSocialSimulationswithHighResolution RasterInputsinDistributedHPCEnvironments ............................ 205 SergiyGogolenko AffectingtheRelaxationParameterintheMultifrontalMethod........... 215 Tomoki Nakano, Mitsuo Yokokawa, Yusaku Yamamoto, andTakeshiFukaya EnhancementoftheGWSpace-TimeProgramCodeforAccurate Predictionofthe Electronic Propertiesof OrganicElectronics Materials.......................................................................... 225 SusumuYanagisawa,TakeshiYamashita,andRyusukeEgawa Part I Future HPC Challenges R&D of a Quantum-Annealing Assisted Next Generation HPC Infrastructure and Its Killer Applications HiroakiKobayashi Abstract As the silicon technology driven by so called Moore’s law is facing the physical limitation, we are now moving to the post-Moore’sera in the design of the next generation high-performance computing infrastructures. Under such a situation, Quantum Annealing is expected to be one of emerging information processing technologies in the Post-Moore’s era, which is especially work well for combinatorial optimization problems. In this article, we present our on-going project entitled, “R&D of a Quantum-Annealing Assisted Next Generation HPC InfrastructureanditsApplications,”whichaimstointegratethequantumannealing informationprocessingintoaconventionalHPCsystemasanacceleratorforcom- binatorialoptimization problems. We also show the designs of targetapplications thatintegratecomputationalscienceanddatascienceapproachestobeinstalledon theunderlyinginfrastructure,whichareexpectedtoplayakeyroleintherealization ofthesmartcity(alsonamedSociety5.0inJapan). 1 Introduction In the last several decades, thanks to the silicon technology development so called Moore’s Law, computer performance has been improved exponentially. However, as the physical limitation in the silicon fabrications is approaching, we are facing the end of Moore’s Law. Under such a circumstance, post-Moore’s information processing technologies such as Quantum computing, Brain-Inspired computing etc. is drawing much attention as emerging ones to make a break- through in computing. In such a trend, quantum annealing is considered one of promisinginformationprocessingmechanismsinthePost-Moore’sera,becauseitis commerciallyrealizedandavailablerightnow,althoughsolvableproblemsarestill small. The quantum annealing is a metaheuristic for finding the global minimum H.Kobayashi((cid:2)) TohokuUniversity,Sendai,Japan e-mail:[email protected] ©SpringerNatureSwitzerlandAG2020 3 M.M.Reschetal.(eds.),SustainedSimulationPerformance2018and2019, https://doi.org/10.1007/978-3-030-39181-2_1