Applied Scientific Computing: With Python

280 Pages·4.45 MB·English

by  Peter R. Turner, Thomas Arildsen, Kathleen Kavanagh

Texts in Computer Science
Peter R. Turner
Thomas Arildsen
Kathleen Kavanagh
Applied
Scientific
Computing
With Python
Texts in Computer Science
Series editors
David Gries, Department of Computer Science, Cornell University, Ithaca, NY,
USA
Orit Hazzan, Faculty of Education in Technology and Science, Technion—Israel
Institute of Technology, Haifa, Israel
Fred B. Schneider, Department of Computer Science, Cornell University, Ithaca,
NY, USA
More information about this series at http://www.springer.com/series/3191
Peter R. Turner Thomas Arildsen
(cid:129)
Kathleen Kavanagh
Applied Scientific
Computing
With Python
123
PeterR. Turner Kathleen Kavanagh
ClarksonUniversity ClarksonUniversity
Potsdam, NY Potsdam, NY
USA USA
ThomasArildsen
Aalborg University
Aalborg
Denmark
ISSN 1868-0941 ISSN 1868-095X (electronic)
Textsin Computer Science
ISBN978-3-319-89574-1 ISBN978-3-319-89575-8 (eBook)
https://doi.org/10.1007/978-3-319-89575-8
LibraryofCongressControlNumber:2018937706
©SpringerInternationalPublishingAG,partofSpringerNature2018
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
Printedonacid-freepaper
ThisSpringerimprintispublishedbytheregisteredcompanySpringerInternationalPublishingAG
partofSpringerNature
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
This book represents a modern approach to teaching numerical methods—or sci-
entific computing—to students with a broad range of primary interests. The
underlyingmathematicalcontentisnotnew,ofcourse,butthefocusonapplications
and models (or modeling) is. Today’s mathematics or computer science students
have a strong desire to see the relevance of what they are studying in a practical
way. This is even more true for students of other STEM disciplines in their
mathematics and computing classes.
In an introductory text such as this, it is difficult to give complete detail on the
application of the methods to “real-world” engineering, or economics, or physical
science, or social science applications but it is important to connect those dots.
Throughoutthebook,weemphasizeapplicationsandincludeopportunitiesforboth
problem- and project-based learning through examples, exercises, and projects
drawnfrompracticalapplications.Thus,thebookprovidesaself-containedanswer
to the common question “why do I need to learn this?”
That is a question which we believe any student has a right both to ask and to
expect a reasonable and credible answer, hence our focus on Applied Scientific
Computing.Theintentionisthatthisbookissuitableforanintroductoryfirstcourse
in scientific computing for students across a range of major fields of study.
Therefore, we make no pretense at a fully rigorous development of all the
state-of-the-art methods and their analyses. The intention is to give a basic
understanding of the need for, and methods of, scientific computing for different
types of problems. In all cases, there is sufficient mathematical justification and
practical evidence to provide motivation for the reader.
Anytextsuchasthisneedstoprovidepracticalguidanceoncodingthemethods,
too. Coding a linear system solver, for example, certainly helps the student’s
understanding of thepractical issues,andlimitations, of its use.Typically students
findintroductorynumericalmethodsmoredifficultthantheirprofessorsexpect.Part
of the reason is that students are expected to combine skills from different parts
of their experience—different branches of mathematics, programming and other
computerscienceskills,andsomeinsightinapplicationsfields.Arguably,theseare
independent skill sets and so the probabilities of success multiply.
v
vi Preface
Our choice of Python for the computer language is based on the desire to
minimize the overhead of learning a high-level language or of the intricacies (and
cost)ofaspecificapplicationpackage.Thecodingexampleshereareintendedtobe
relatively easily readable; they are not intended to be professional-level software.
The Python code is there to facilitate the learning and understanding of the basic
material, rather than being an objective in itself.
Turning briefly to the content of the book, we gave considerable thought to the
orderingofmaterialandbelievethattheorderwehaveisonegoodwaytoorganize
acourse,thoughofcourseeveryinstructorwillhavehis/herownideasonthat.The
chapters are largely independent and so using them in a different order will not be
problematic.
Webeginwithabriefbackgroundchapterthatsimplyintroducesthemaintopics
application and modeling, Python programming, sources of error. The latter is
exemplified at this point with simple series expansions which should be familiar
fromcalculusbutneednothingmore.Theseseriesexpansionsalsodemonstratethat
the“most obvious” approach that abeginning student might adopt will not always
be practical. As such it serves to motivate the need for more careful analysis of
problemsandtheirsolutions.Chapter2isstillsomewhatfoundationalwithitsfocus
onnumberrepresentationanderrors.Theimpactofhownumbersarerepresentedin
a computer, and the effects of rounding and truncation errors recur in discussing
almost any computational solution to any real-life problem.
FromChap.3onwards,wearemorefocusedonmodeling,applications,andthe
numericalmethodsneededtosolvethem.InChap.3itselfthefocusisonnumerical
calculus.Weputthisbeforeinterpolationtoreflectthestudents’familiaritywiththe
basic concepts and likely exposure to at least some simple approaches. These are
treated without any explicit reference to finite difference schemes.
Chapters4and5aredevotedtolinearandthennonlinearequations.Systemsof
linear equations are another topic with which students are somewhat familiar, at
least in low dimension, and we build on that knowledge to develop more robust
methods. The nonlinear equation chapter also builds on prior knowledge and is
motivated by practical applications. It concludes with a brief treatment of the
multivariable Newton’s method.
The final two chapters are on interpolation and the numerical solution of dif-
ferential equations. Polynomial interpolation is based mostly on a divided differ-
ence approach which then leads naturally to splines. Differential equations start
from Euler’s method, and then Runge Kutta, for initial value problems. Shooting
methods for two-point boundary value problems are also covered, tying in many
ideas from the previous chapters.
All of this material is presented in a gentle way that relies very heavily on
applications and includes working Python code for many of the methods. The
intention is to enable students to overcome the combined demands of the mathe-
matics, the computing, the applications, and motivation to gain a good working
insight into the fundamental issues of scientific computing. The impact of the
inevitable reliance on algebraic manipulation is largely minimized through careful
explanation of those details to help the student appreciate the essential material.
Preface vii
All of us have benefited from many helpful discussions both on the philosophy
andthedetailsofteachingthiscontentovermanyyears–moreforsomethanothers!
Those influencers are too many to list, and so we simply thank them en masse for
all the helpful conversations we have had over the years.
Potsdam, USA Peter R. Turner
Aalborg, Denmark Thomas Arildsen
Potsdam, USA Kathleen Kavanagh
Contents
1 Motivation and Background.... .... .... .... .... .... ..... .... 1
1.1 Mathematical Modeling and Applications... .... .... ..... .... 1
1.2 Applied Scientific Computing.... .... .... .... .... ..... .... 3
1.3 Python Programming . ..... .... .... .... .... .... ..... .... 5
1.4 Background .... .... ..... .... .... .... .... .... ..... .... 6
1.4.1 Series Expansions... .... .... .... .... .... ..... .... 6
1.5 Modeling Errors Versus Errors... .... .... .... .... ..... .... 10
1.6 Conclusions and Connections: Motivation and Background.. .... 11
2 Number Representations and Errors. .... .... .... .... ..... .... 15
2.1 Introduction .... .... ..... .... .... .... .... .... ..... .... 15
2.2 Floating-Point Numbers .... .... .... .... .... .... ..... .... 16
2.2.1 Python Number Representation. .... .... .... ..... .... 19
2.3 Sources of Errors .... ..... .... .... .... .... .... ..... .... 21
2.3.1 Rounding Errors .... .... .... .... .... .... ..... .... 21
2.3.2 Truncation Error .... .... .... .... .... .... ..... .... 22
2.3.3 Ill-Conditioning..... .... .... .... .... .... ..... .... 24
2.4 Measures of Error and Precision.. .... .... .... .... ..... .... 24
2.5 Floating-Point Arithmetic... .... .... .... .... .... ..... .... 27
2.6 Conclusions and Connections: Number Representation
and Errors.. .... .... ..... .... .... .... .... .... ..... .... 31
3 Numerical Calculus.. .... ..... .... .... .... .... .... ..... .... 35
3.1 Introduction .... .... ..... .... .... .... .... .... ..... .... 35
3.2 Numerical Differentiation... .... .... .... .... .... ..... .... 38
3.3 Numerical Integration. ..... .... .... .... .... .... ..... .... 48
3.4 Composite Formulas.. ..... .... .... .... .... .... ..... .... 58
3.5 Practical Numerical Integration... .... .... .... .... ..... .... 68
3.6 Conclusions and Connections: Numerical Calculus ... ..... .... 78
3.7 Python Functions for Numerical Calculus... .... .... ..... .... 79
4 Linear Equations.... .... ..... .... .... .... .... .... ..... .... 81
4.1 Introduction .... .... ..... .... .... .... .... .... ..... .... 81
4.2 Gauss Elimination.... ..... .... .... .... .... .... ..... .... 84
ix
x Contents
4.2.1 Pivoting in Gauss Elimination.. .... .... .... ..... .... 88
4.2.2 Tridiagonal Systems . .... .... .... .... .... ..... .... 93
4.3 LU Factorization and Applications.... .... .... .... ..... .... 98
4.4 Iterative Methods .... ..... .... .... .... .... .... ..... .... 106
4.5 Linear Least Squares Approximation .. .... .... .... ..... .... 115
4.6 Eigenvalues .... .... ..... .... .... .... .... .... ..... .... 127
4.7 Conclusions and Connections: Linear Equations.. .... ..... .... 140
4.8 Python’s Linear Algebra Functions.... .... .... .... ..... .... 141
4.8.1 Linear Equations.... .... .... .... .... .... ..... .... 141
4.8.2 Linear Least Squares. .... .... .... .... .... ..... .... 142
4.8.3 Eigenvalues... ..... .... .... .... .... .... ..... .... 143
4.8.4 Basic Linear Algebra Functions .... .... .... ..... .... 144
5 Iterative Solution of Nonlinear Equations. .... .... .... ..... .... 145
5.1 Introduction .... .... ..... .... .... .... .... .... ..... .... 145
5.1.1 Summary of Convergence of Sequences.. .... ..... .... 146
5.2 The Bisection Method ..... .... .... .... .... .... ..... .... 147
5.3 Fixed Point Iteration.. ..... .... .... .... .... .... ..... .... 153
5.4 Newton’s Method.... ..... .... .... .... .... .... ..... .... 160
5.5 The Secant Method... ..... .... .... .... .... .... ..... .... 167
5.6 Newton’s Method in Higher Dimensions ... .... .... ..... .... 173
5.6.1 Newton’s Method: Two Equations in Two Unknowns .... 176
5.7 Conclusions and Connections: Iterative Solution of Nonlinear
Equations .. .... .... ..... .... .... .... .... .... ..... .... 185
5.8 Python Functions for Equation Solving .... .... .... ..... .... 187
6 Interpolation ... .... .... ..... .... .... .... .... .... ..... .... 189
6.1 Introduction .... .... ..... .... .... .... .... .... ..... .... 189
6.2 Lagrange Interpolation ..... .... .... .... .... .... ..... .... 192
6.3 Difference Representations .. .... .... .... .... .... ..... .... 199
6.3.1 Divided Difference Interpolation.... .... .... ..... .... 199
6.4 Splines .... .... .... ..... .... .... .... .... .... ..... .... 208
6.5 Conclusions and Connections: Interpolation. .... .... ..... .... 225
6.6 Python Interpolation Functions... .... .... .... .... ..... .... 226
7 Differential Equations.... ..... .... .... .... .... .... ..... .... 229
7.1 Introduction and Euler’s Method . .... .... .... .... ..... .... 229
7.2 Runge–Kutta Methods ..... .... .... .... .... .... ..... .... 237
7.3 Multistep Methods ... ..... .... .... .... .... .... ..... .... 245
7.4 Systems of Differential Equations. .... .... .... .... ..... .... 252
7.5 Boundary Value Problems: Shooting Methods... .... ..... .... 258
7.6 Conclusions and Connections: Differential Equations.. ..... .... 264
7.7 Python Functions for Ordinary Differential Equations . ..... .... 265
Further Reading and Bibliography . .... .... .... .... .... ..... .... 267
Index .... .... .... .... .... ..... .... .... .... .... .... ..... .... 269