ebook img

Principles of Parallel Scientific Computing PDF

302 Pages·2022·19.998 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Principles of Parallel Scientific Computing

Undergraduate Topics in Computer Science Tobias Weinzierl Principles of Parallel Scientific Computing A First Guide to Numerical Concepts and Programming Methods Undergraduate Topics in Computer Science Series Editor Ian Mackie, University of Sussex, Brighton, UK Advisory Editors Samson Abramsky , Department of Computer Science, University of Oxford, Oxford, UK ChrisHankin ,DepartmentofComputing,ImperialCollegeLondon,London,UK Mike Hinchey , Lero – The Irish Software Research Centre, University of Limerick, Limerick, Ireland Dexter C. Kozen, Department of Computer Science, Cornell University, Ithaca, NY, USA Andrew Pitts , Department of Computer Science and Technology, University of Cambridge, Cambridge, UK HanneRiisNielson ,DepartmentofAppliedMathematicsandComputerScience, Technical University of Denmark, Kongens Lyngby, Denmark StevenS.Skiena,DepartmentofComputerScience,StonyBrookUniversity,Stony Brook, NY, USA Iain Stewart , Department of Computer Science, Durham University, Durham, UK ‘Undergraduate Topics in Computer Science’ (UTiCS) delivers high-quality instructional content for undergraduates studying in all areas of computing and information science. From core foundational and theoretical material to final-year topics and applications, UTiCS books take a fresh, concise, and modern approach and are ideal for self-study or for a one- or two-semester course. The texts are all authoredbyestablishedexpertsintheirfields,reviewedbyaninternationaladvisory board,andcontainnumerousexamplesandproblems,manyofwhichincludefully worked solutions. TheUTiCSconceptreliesonhigh-quality,concisebooksinsoftbackformat,and generally a maximum of 275–300 pages. For undergraduate textbooks that are likelytobelonger,moreexpository,Springercontinuestoofferthehighlyregarded Texts in Computer Science series, to which we refer potential authors. More information about this series at https://link.springer.com/bookseries/7592 Tobias Weinzierl Principles of Parallel Scientific Computing A First Guide to Numerical Concepts and Programming Methods 123 TobiasWeinzierl Department ofComputer Science Durham University Durham, UK ISSN 1863-7310 ISSN 2197-1781 (electronic) Undergraduate Topics inComputer Science ISBN978-3-030-76193-6 ISBN978-3-030-76194-3 (eBook) https://doi.org/10.1007/978-3-030-76194-3 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature SwitzerlandAG2021 Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseof illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar ordissimilarmethodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained hereinorforanyerrorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregard tojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland Preface Why This Book IstartedtothinkaboutthismanuscriptwhenIgotappointedasAssociateProfessor atDurhamUniversityandhadbeentoldtoreadnumericalalgorithmsforcomputer scientists plus theparallel programming submodule. InDurham, we teach thisfirst submodule on numerics in the third year of the computer science degree. It spans only ten hours. Ten hours is ridiculously short, but I think it is enough to spark interest in simulation codes and the simulation business. Thinking about it, it is great that our numerics teaching runs parallel to a parallel programming course. Numerical simulation without parallelism today is barely imaginable, so this arrangement gives us the great opportunity to use examples from one area in the other one. We can co-teach the two things: numerics basics and parallel coding. When we later designed our Master in Scientific Computing and Data Analysis (MISCADA), itbecameclearthatco-teaching thetwofieldswouldbeagoodidea for this course, too. With a very heterogeneous student intake—both by means of qualification, cultural background, educational system, and discipline interest—we need to run introductory courses that bring all students quickly to a decent level, while numerical principles as well as parallel programming are omnipresent. Obviously,theM.Sc.-level teaching progressesatadifferent paceand,inDurham, also covers more lectures. Yet, some contentoverlapswith thethirdyear’scourse. There seems to be a limited amount of literature that discusses parallel pro- gramming and (the basics of) numerical techniques in one go and still remains easily accessible. There are many books that cover scientific computing from a mathematics point of view. Many scientists (including me) insist that a formal approach with a strong mathematical background is imperative for research in scientificcomputing.However,wehavetoappreciatethatmanyofourundergrads are enrolled in courses that are lighter on the mathematics side or focus, for example,moreondiscretemaths.Somemathsthusmightbealittlebitrustywhen studentsentertheirthirdyear.Ourmathematicaltextbooksruntheriskthatwelose outonstudentsiftheyorbitaroundtheprofoundmathsfirst.Theymightgetbored beforetheyreachthepointwheretheycanprogramafirstnumericalcodethatdoes “something useful”. v vi Preface Theothertypeofteachingliteraturethatisouttherecombinesthefirstcoursein numericswithaprogrammingtutorial.AlthoughalotofthematerialIhaveseenis really good, this type of approach is ill-suited for our students. Most of them are decent programmers already. To make a long story short, I think that there is a need for courses on numerics witha“rollingupyoursleeves”and“gettheroughideaandmakeitworkincode” attitude.Somestudentsliketogettheirhandsdirtyandwritesomething“thatdoes something” before they start to dive into more details—for which excellent liter- ature does exist. I am also strongly convinced that it makes sense to teach intro- ductoryscientificcomputingincombinationwithparallelisation.Afterall,wewant to have future Ph.D. students and colleagues who can help us with the big simu- lation codes. Mission Statement Since we try to keep the theory light and hands-on, this book is not a compre- hensive overview of the vast field of numerical algorithms and analysis, or even only a certain subfield. It neither is a rock solid introduction to the difficulties, pitfalls and depths of programming in high performance computing (HPC). It is (cid:129) a rush through some basic numerical concepts that are omnipresent when we writesimulationsoftware.Wewanttogetanideaofhowsomenumericalthings work—so we can eventually look them up in a “real” maths book. (cid:129) a numerics primer with implementation glasses on: For all of the concepts, we wanttohaveanideaofhowtorealisethemonarealmachineorwhichimpact they have on our codes. (cid:129) a hands-on text. The best way to understand numerical concepts is to program them or, even better, to see your own code fail and then find a theoretical explanation (together with the fix) for the break down. And it is close to impossible to learn parallel programming solely from textbooks anyway. (cid:129) a guide that makes students start to write simulation codes running on parallel computers. Structure and Style Themanuscriptjumpsfromparallelprogrammingtonumericsandback.Isprinkle insomelight-touchrevisionofthingsthatIconsidertobeimportantbuthavebeen told by my students that many computer science curricula do not cover them anymore. Even if taught at school before, students might have forgotten them. Wherepossible,ItrytomakeallcontentfeedintoonesimpleN-bodysolver.That’s the application leitmotif throughout the manuscript. If readers translate it into a working code, they will end up with an N-body solver with simple gravity, will Preface vii havelearnedfundamentalideasbehindthesolversuchasnumericalterminologyor challengesfacedbyintegratorsforordinarydifferentialequations,andwillhavethe skill set to make this solver run efficiently on a shared memory machine or some GPUs.There’snoneedforareader(orteacher)tosticktomyconceptortoreadthe chapters one by one. Feel free to skip chapters, pick chapters, or use a different application example. Mathematicians will be horrified by my hands-on explanations and lack of formalism. I am fine with this. This book is not yet another maths course. It is a bookthatshall bequickandeasytoread andmotivatestudents tostudy themaths behind the scenes later, oncethe first generation oftheir simulation codeisup and running.Thisbookisalsonotacoursewherestudentsreallylearnsomethingabout N-bodycodesforaparticularapplicationarea.SimpleN-bodycodesjustseemedto be a convenient toy problem that we can understand with A-levels maths. ExpertsdeeplyfamiliarwiththedetailsofOpenMP,e.g.,alsowillbehorrifiedby theinaccuraciesinthetext.Forexample,thevisibilityofavariableintheOpenMP standard is determined through a sequence of rules. These rules are important, but most students quickly will be lost in them. Also, most programmers don’t have to memorise them. They have toknow what happens in 99% ofthe cases. As I want to push students into the field of scientific computing plus HPC, I madeallexamplesinthetextrelyonC/C++.Iamwellawarethatmanycodesstill are written in Fortran, and I am well aware that people nowadays can write inter- esting and fast software with Python, e.g. But C still is very dominant and, fortu- nately, many computer science students are still exposed to C in one form or the other throughout their undergraduate studies. While A-levels maths is sufficient to digest the book’s content, I hence expect students to bring along C programming skills.1 Maybe unusual for a student-facing book, this text does not contain any exer- cises.Therearetworeasonsforthis:Ontheonehand,thereareplentyofbooks,in particularinnumerics,thatarefullofexercises;andmoreandmorewebpageswith (interactive) examples follow. On the other hand, the prime way to exercising scientific computing is to write codes, and I think that there is a big difference between writing small, well-defined example calculations or a bigger piece of software. With this text, I want to encourage students to do the latter. Acknowledgements Thanksareduetonumerousstudentswhoprovidedfeedback.Particularthanksare duetomycolleaguesFlorianRuppandPhilippNeumann.Theyfoundtonsofbugs andhelpedmetofixalotofphrases.ThanksarealsoduetoLawrenceMitchellfor proofreadingsomeofthechapterswithanHPCtouch.HolgerSchulzworkedwith 1The UTICS series hosts, for example, the Pitt-Francis and Whiteley book “Guide to Scientific ComputinginC++”whereChaps.1–5provideareasonablebaseforthepresentbook. viii Preface me on the ExCALIBUR/ExaClaw project throughout the write-up and helped me quite a lot with the OpenMP-focussed chapters. Marion Weinzierl finally was maybe the most critical reader, but eventually it had been her comments and her support over all the years that made writing this manuscript possible. Learning and Teaching Mode MajorpartsofthismanuscripthavebeenwrittenduringthefirstCoronalockdown. Iamsurethatthisunexpectedsituationhasforcedmanylecturerstorevise,rethink and re-evaluate their academic teaching style. I had to. Though I do not know (yet)whetherthisbookisofanyuseorhelpforthebrandnewworldofblendedand virtual teaching, the manuscript style is a result of my personal “rethink your teaching” exercise. TheappendixhostssomeideasonhowIusethebook’scontenttostructuremy classes. The text is written in a language such that students can smoothly work through the content themselves: If I have to make a choice, I run for the sloppy jargon rather than ultra-formal, dryish phrases. Furthermore, I tried to use illus- trationsfortechnicaldetails,asIammyselfavisuallearner.Keylearningoutcomes areextractedandorganisedintoboxes,andtheappendixhostssomechecklistsand cheatsheetsforstudents’coursework.Allofthisallowsmetousethetextasabasis for courses in a “flipped classroom”-style. Obviously, this style can be comple- mented by online/synchronous sessions, videos, exercises and code studies; and, obviously, reading lists that dive deeper into the subject. Enjoy the text, and I hope that it is useful. Durham, UK Tobias Weinzierl January 2022 Contents Part I Introduction: Why to Study the Subject 1 The Pillars of Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 A Third Pillar? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2 Computational X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3 About the Style of the Text and Some Shortcomings . . . . . . . . 9 2 Moore Myths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1 Moore’s Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.2 Dennard Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3 The Three Layers of Parallelism . . . . . . . . . . . . . . . . . . . . . . . 15 3 Our Model Problem (Our First Encounter with the Explicit Euler) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1 The N-Body Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2 Time Discretisation: Running a Movie. . . . . . . . . . . . . . . . . . . 23 3.3 Numerical Versus Analytical Solutions. . . . . . . . . . . . . . . . . . . 25 Wrap-up Part II How the Machine Works 4 Floating Point Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.1 Fixed Point Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.2 Floating Point Numbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.2.1 Normalisation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.2.2 The IEEE Format. . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.2.3 Realising Floating-Point Support. . . . . . . . . . . . . . . . . 40 4.3 Machine Precision. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 4.4 Programming Guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 ix

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.