Scientific Computing: A Historical Perspective

271 Pages·2018·3.993 MB·English

by  Bertil Gustafsson

17
Bertil Gustafsson
Scientifi c
Computing
A Historical Perspective
Editorial Board
T. J.Barth
M.Griebel
D.E.Keyes
R.M.Nieminen
D.Roose
T.Schlick
Texts in Computational
17
Science and Engineering
Editors
TimothyJ. Barth
MichaelGriebel
DavidE. Keyes
RistoM. Nieminen
DirkRoose
TamarSchlick
Moreinformationaboutthisseriesathttp://www.springer.com/series/5151
Bertil Gustafsson
Scientific Computing
A Historical Perspective
123
BertilGustafsson
DepartmentofInformationTechnology
UppsalaUniversity
Uppsala,Sweden
ISSN1611-0994 ISSN2197-179X (electronic)
TextsinComputationalScienceandEngineering
ISBN978-3-319-69846-5 ISBN978-3-319-69847-2 (eBook)
https://doi.org/10.1007/978-3-319-69847-2
LibraryofCongressControlNumber:2018953059
MathematicsSubjectClassification(2010):65-XX,68-XX,01-XX
©SpringerInternationalPublishingAG,partofSpringerNature2018
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
Cover illustration: Figure 2.1 showing the Babylonian clay tablet. (The clay tablet YBC7289. From
the Yale Babylonian Collection, with the assistance and permission of William Hallo, Curator, and
UllaKasten,AssociateCurator.PhotobyBillCasselman,http://www.math.ubc.ca/_cass/Euclid/ybc/ybc.
html.)
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Scienceandtechnologyaretraditionallybasedontheoryandexperiments,butnow-
adaysscientificcomputingisanestablishedthirdbranchofstrategicimportance.It
is based on mathematicalmodels, some of which were developedmany centuries
ago.Mostofthesemodelsareintheformofdifferentkindsofequationsthatmust
besolved,andintheidealcase,onecanfindthesolutionsbyanalyticmeansinthe
formofanexplicitmathematicalform,asforexamplea functionf(x,y)thatcan
beeasilyevaluatedforanyvalueoftheindependentvariablesx andy.However,in
mostrealisticcasesthisisnotpossible,andnumericalmethodsmustbeusedtofind
accurate approximationsof the solutions to the mathematical model. The concept
of scientific computing usually means the whole procedure including analysis
of the mathematical model, development and analysis of a numerical method,
programmingtheresultingalgorithm,andfinallyrunningitona computer.Today,
there are many systems that include the whole chain and require only specifying
the physicalparametersfor the problemto be solved numerically.However,there
arealwaysnewchallengeswithnewtypesofmathematicalmodelsthatrequirenew
numericalmethods.
This book is about some of the most significant problem areas and the history
oftheprocessleadingtoefficientnumericalmethods.Mathematicsintheolddays
was veryclosely connectedto astronomy,which was always the major source for
new mathematical problems up to the nineteenth century. According to Galileo
Galilei(1564–1642),“thebookofnatureiswritteninthelanguageofmathematics,”
and this was particularly true for astronomy. As we shall see, many of the most
famousmathematiciansinthepastcouldaswellhavebeencalledastronomers.For
example,perhapsthemostfamousmathematicianofalltime,CarlFriedrichGauss
(1777–1855),was the directorof the German astronomicalobservatoryGöttingen
Observatoryfor48years.
Today, there is a clear distinction between pure mathematics and applied
mathematicswith scientific computingquite far frompure mathematics.Thiswas
notthecaseintheolddays.Mathematiciansfoundnewmodelsforvariousphysical
problems,buttheyalsocarriedoutthenecessarycomputationstofindtheunknown
v
vi Preface
numberstheywerelookingfor.Atypicalexamplewastopredictthefuturelocation
ofacertainplanetbasedonafewavailableobservations.
Thecontentofthisbookisdividedintofourparts.Thefirstisabouttheveryearly
mathematical/numerical achievements made by the Babylonians and the Greeks.
AfterthatnotmuchhappeneduntiltheseventeenthcenturywhenNewtonandothers
developednew mathematics,and the second partis aboutthe developmentduring
the centuries until the Second World War. Just at the end of the war, scientific
computingtookagiantstepforwardwiththeconstructionofelectroniccomputers.
Nownumericalmethodsthatearlierledtocomputationsofimpossiblesizecouldbe
implementedonthesecomputerswithresultsobtainedafterafewhours.Inthisnew
situation,thedevelopmentofnewmethodsbecamemuchmoreinteresting,andthe
existence of electronic computers provided a strong boost for numerical analysis
and scientific computing. The third part of the book is about this postwar period
until the end of the 1950s. Around that time, scientific computingbecame a third
scientificmethodinadditiontothetraditionalbranchesintheoryandexperiments.
Thefourthpartofthebookcoverstheperioduntilthepresenttime.
The major numericalmethodsare traced back to their origin and to the people
whoinventedthem,aswellastotheoriginofimportanttechniquesforanalysisof
themethods.Thereisalso shortpresentationsofsomeofthemathematicianswho
played a key role in this process. There is certainly not a complete description of
the whole story behind all methods, but rather an attempt to catch the key steps
without going into a complete mathematical derivation. There is also a very brief
presentationofthedevelopmentofelectroniccomputers,particularlytheearlyones.
Differentialequationsarethedominatingtypeofmathematicalmodelsforalmost
everybranchofscience and engineering,andthere are severaldifferentprinciples
thatareusedwhendevelopingnumericalmethodsforthesolutionoftheseproblems.
Therefore,differentialequationsaregivensomeextraspaceinthisbook.
Onedifficultywhengoingbackintimeisthattheoriginalarticlesarenotalways
easy to read and understand. During the active period from the sixteenth to the
nineteenthcentury,Latinwasacommonlanguageformathematicaltexts.Another
majordifficultywasthewaymathematicswasdescribed.Manysymbolsusedtoday
were notintroducedatthat time, andnumericalmethodswere describedin words
inquitealengthyandcomplicatedway.Infact,insomecasesthereisstillacertain
uncertaintyaboutthe exactmeaningof the description,and a consequenceof this
maybethattherealinventorofacertainmethodisnotknownforsure.
Weareawareofthefactthatthewholeareaofscientificcomputingisnotcovered
inthisbook.Therearemanytechniquesandmethodsthatareleftoutorjustbriefly
mentioned,andtherearemanymathematicianswhocouldhavebeenincludedbut
arenot.Thebookisintendedtogivethebigpictureandthehistoricaldevelopment
behindtheriseofscientificcomputingasanewscientificbranch.
Thecontentofthisbookisbasedonalargenumberofsourcesinadditiontothe
originalbooksandarticlesinthereferencelist.Itisimpossibletogiveallofthese
sources, in particularfor some of the notesconcerningimportantmathematicians,
butasfaraspossible,correctnesshasbeenchecked.
Preface vii
Hopefully,thebookwillbeavaluableresourceforallstudentsandprofessionals
interested in the history of numerical analysis and computing and for a broader
readershipalike.
Uppsala,Sweden BertilGustafsson
Acknowledgments
ThisbookwaswrittenaftermyretirementfromthechairattheDivisionofScientific
ComputingatUppsalaUniversity.TheDepartmentofInformationTechnologyhas
stillprovidedfullaccesstothenecessaryinfrastructure,whichismuchappreciated.
Martin Peters and Ruth Allewelt at Springer have been helpful, in particular
whenitcomestocopyrightissues.AnnKostantcorrectedlanguagedeficienciesand
misprints, and I am impressed by her ability to find so many errors and language
detailsthatrequiredcorrection.
Finally, I would like to thank my wife Margaretafor accepting and supporting
myfullengagementinwritingstillanotherbook.
ix
Contents
1 ScientificComputing:AnIntroduction.................................... 1
2 ComputationFarBackinTime ............................................ 5
2.1 TheBabylonians....................................................... 6
2.2 ArchimedesandIterativeMethods ................................... 10
2.3 ChineseMathematics.................................................. 15
3 TheCenturiesBeforeComputers.......................................... 17
3.1 NonlinearAlgebraicEquations ....................................... 19
3.1.1 TheFixedPointMethod..................................... 19
3.1.2 NewtonMethods............................................. 20
3.2 Interpolation ........................................................... 25
3.3 Integrals................................................................ 32
3.4 TheLeastSquaresMethod............................................ 36
3.5 GaussandLinearSystemsofEquations.............................. 42
3.6 SeriesExpansion....................................................... 46
3.6.1 TaylorSeries ................................................. 47
3.6.2 OrthogonalPolynomialExpansions ........................ 48
3.6.3 FourierSeries ................................................ 51
3.6.4 TheGibbsPhenomenon ..................................... 55
3.6.5 TheFourierTransform....................................... 56
3.7 OrdinaryDifferentialEquations(ODE) .............................. 58
3.7.1 TheEulerMethod............................................ 58
3.7.2 AdamsMethods.............................................. 61
3.7.3 Runge–KuttaMethods....................................... 64
3.7.4 RichardsonExtrapolation.................................... 65
3.8 PartialDifferentialEquations(PDE) ................................. 68
3.8.1 TheRitz–GalerkinMethod.................................. 69
3.8.2 Courant’sArticleonFEM................................... 72
3.8.3 Richardson’sFirstPaperonDifferenceMethods........... 75
3.8.4 WeatherPrediction;AFirstAttempt........................ 77
3.8.5 TheCFL-Article ............................................. 79
xi