Numerical Mathematics

663 Pages·2007·4.893 MB·English

by  Alfio Quarteroni, Riccardo Sacco, Fausto Saleri (auth.)

37
Texts in Applied Mathematics
Editors
J.E.Marsden
L.Sirovich
S.S.Antman
Advisors
G.Iooss
P.Holmes
D.Barkley
M.Dellnitz
P.Newton
Texts in Applied Mathematics
1. Sirovich:IntroductiontoAppliedMathematics.
2. Wiggins:IntroductiontoAppliedNonlinearDynamicalSystemsandChaos.
3. Hale/Koc¸ak:DynamicsandBifurcations.
4. Chorin/Marsden:AMathematicalIntroductiontoFluidMechanics,Third
Edition.
5. Hubbard/West:DifferentialEquations:ADynamicalSystemsApproach:
OrdinaryDifferentialEquations.
6. Sontag:MathematicalControlTheory:DeterministicFiniteDimensional
SystemsSecondEdition.
7. Perko:DifferentialEquationsandDynamicalSystems,ThirdEdition.
8. Seaborn:HypergeometricFunctionsandTheirApplications.
9. Pipkin:ACourseonIntegralEquations.
10. Hoppensteadt/Peskin:ModelingandSimulationinMedicineandtheLife
Sciences,SecondEdition.
11. Braun:DifferentialEquationsandTheirApplications,FourthEdition.
12. Stoer/Bulirsch:IntroductiontoNumericalAnalysis,ThirdEdition.
13. Renardy/Rogers:AnIntroductiontoPartialDifferentialEquations.
14. Banks:GrowthandDiffusionPhenomena:MathematicalFrameworksand
Applications.
15. Brenner/Scott:TheMathematicalTheoryofFiniteElementMethods,Second
Edition.
16. VandeVelde:ConcurrentScientificComputing.
17. Marsden/Ratiu:IntroductiontoMechanicsandSymmetry,SecondEdition.
18. Hubbard/West:DifferentialEquations:ADynamicalSystemsApproach:
Higher-DimensionalSystems.
19. Kaplan/Glass:UnderstandingNonlinearDynamics.
20. Holmes:IntroductiontoPerturbationMethods.
21. Curtain/Zwart:AnIntroductiontoInfinite-DimensionalLinearSystems
Theory.
22. Thomas:NumericalPartialDifferentialEquations:FiniteDifference
Methods.
23. Taylor:PartialDifferentialEquations:BasicTheory.
24. Merkin:IntroductiontotheTheoryofStabilityofMotion.
25. Naber:Topology,Geometry,andGaugeFields:Foundations.
26. Polderman/Willems:IntroductiontoMathematicalSystemsTheory:
ABehavioralApproach.
27. Reddy:IntroductoryFunctionalAnalysis:withApplicationstoBoundary
ValueProblemsandFiniteElements.
28. Gustafson/Wilcox:AnalyticalandComputationalMethodsofAdvanced
EngineeringMathematics.
(continuedafterindex)
Alfio Quarteroni Riccardo Sacco Fausto Saleri
Numerical Mathematics
Second Edition
With135Figuresand45Tables
(cid:65)(cid:66)(cid:67)
AlfioQuarteroni RiccardoSacco
SB-IACS-CMS,EPFL DipartimentodiMatematica
1015Lausanne,Switzerland PolitecnicodiMilano
and PiazzaLeonardodaVinci,32
DipartimentodiMatematica-MOX 20133Milano,Italy
PolitecnicodiMilano E-mail:[email protected]
PiazzaLeonardodaVinci,32
20133Milano,Italy FaustoSaleri
E-mail:alfio.quarteroni@epfl.ch
DipartimentodiMatematica–MOX
PolitecnicodiMilano
PiazzaLeonardodaVinci,32
20133Milano,Italy
SeriesEditors E-mail:[email protected]
J.E.Marsden
S.S.Antman
ControlandDynamicalSystems
DepartmentofMathematics
107-81CaliforniaInstituteofTechnology
and
Pasadena,CA91125
InstituteforPhysicalScience
USA
andTechnology
[email protected]
UniversityofMaryland
CollegeOark,MD20742-4015
L.Sirovich
USA
LaboratoryofAppliedMathematics [email protected]
DepartmentofBiomathematics
Mt.SinaiSchoolofMedicine
Box1012
NewYork,NY10029-6574
USA
MathematicsSubjectClassification(2000):15-01,34-01,35-01,65-01
ISBN 0939-2475
ISBN-10 3-540-34658-9SpringerBerlinHeidelbergNewYork
ISBN-13 978-3-540-34658-6SpringerBerlinHeidelbergNewYork
LibraryofCongressControlNumber:2006930676
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broadcasting,
reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationofthispublicationor
partsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,1965,
initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer-Verlag.Violations
areliabletoprosecutionundertheGermanCopyrightLaw.
SpringerisapartofSpringerScience+BusinessMedia.
springer.com
(cid:176)c SpringerBerlinHeidelberg2007
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnotimply,
evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotectivelaws
andregulationsandthereforefreeforgeneraluse.
TypesettingbytheAuthorsandSpiusingSpringerLATEXmacropackage
Coverdesign:design&productionGmbH,Heidelberg
Printedonacid-freepaper SPIN:11304951 37/2244/SPi 543210
Preface
Numericalmathematicsisthebranchofmathematicsthatproposes,develops,
analyzes and applies methods from scientific computing to several fields in-
cluding analysis, linear algebra, geometry, approximation theory, functional
equations, optimization and differential equations. Other disciplines such as
physics, the natural and biological sciences, engineering, and economics and
thefinancialsciencesfrequentlygiverisetoproblemsthatneedscientificcom-
puting for their solutions.
As such, numerical mathematics is the crossroad of several disciplines of
great relevance in modern applied sciences, and can become a crucial tool for
theirqualitativeandquantitativeanalysis.Thisroleisalsoemphasizedbythe
continual development of computers and algorithms, which make it possible
nowadays, using scientific computing, to tackle problems of such a large size
that real-life phenomena can be simulated providing accurate responses at
affordable computational cost.
The corresponding spread of numerical software represents an enrichment
forthescientificcommunity.However,theuserhastomakethecorrectchoice
of the method (or the algorithm) which best suits the problem at hand. As a
matter of fact, no black-box methods or algorithms exist that can effectively
and accurately solve all kinds of problems.
One of the purposes of this book is to provide the mathematical foun-
dations of numerical methods, to analyze their basic theoretical properties
(stability, accuracy, computational complexity), and demonstrate their per-
formances on examples and counterexamples which outline their pros and
cons. This is done using the MATLAB(cid:1) 1 software environment. This choice
satisfies the two fundamental needs of user-friendliness and wide-spread dif-
fusion, making it available on virtually every computer.
Every chapter is supplied with examples, exercises and applications of the
discussed theory to the solution of real-life problems. The reader is thus in
the ideal condition for acquiring the theoretical knowledge that is required to
1 MATLABisatrademarkofTheMathWorks,Inc.
VI Preface
make the right choice among the numerical methodologies and make use of
the related computer programs.
This book is primarily addressed to undergraduate students, with partic-
ular focus on the degree courses in Engineering, Mathematics, Physics and
Computer Science. The attention which is paid to the applications and the
related development of software makes it valuable also for graduate students,
researchers and users of scientific computing in the most widespread profes-
sional fields.
The content of the volume is organized into four Parts and 13 chapters.
Part I comprises two chapters in which we review basic linear algebra and
introduce the general concepts of consistency, stability and convergence of a
numerical method as well as the basic elements of computer arithmetic.
PartIIisonnumericallinearalgebra,andisdevotedtothesolutionoflin-
earsystems(Chapters3and4)andeigenvaluesandeigenvectorscomputation
(Chapter 5).
We continue with Part III where we face several issues about functions
andtheirapproximation.Specifically,weareinterestedinthesolutionofnon-
linear equations (Chapter 6), solution of nonlinear systems and optimization
problems (Chapter 7), polynomial approximation (Chapter 8) and numerical
integration (Chapter 9).
Part IV, which demands a mathematical background, is concerned with
approximation, integration and transforms based on orthogonal polynomials
(Chapter10),solutionofinitialvalueproblems(Chapter11),boundaryvalue
problems(Chapter12)andinitial-boundaryvalueproblemsforparabolicand
hyperbolic equations (Chapter 13).
Part I provides the indispensable background. Each of the remaining Parts
has a size and a content that make it well suited for a semester course.
A guideline index to the use of the numerous MATLAB programs devel-
oped in the book is reported at the end of the volume. These programs are
also available at the web site address:
http://www1.mate.polimi.it/˜calnum/programs.html.
Forthereader’sease,anycodeisaccompaniedbyabriefdescriptionofits
input/output parameters.
We express our thanks to the staff at Springer-Verlag New York for their
expertguidanceandassistancewitheditorialaspects,aswellastoDr.Martin
Peters from Springer-Verlag Heidelberg and Dr. Francesca Bonadei from
Springer-Italiafortheiradviceandfriendlycollaborationallalongthisproject.
We gratefully thank Professors L. Gastaldi and A. Valli for their useful
comments on Chapters 12 and 13.
We also wish to express our gratitude to our families for their forbearance
and understanding, and dedicate this book to them.
Lausanne, Milan Alfio Quarteroni
January 2000 Riccardo Sacco
Fausto Saleri
Preface to the Second Edition
This second edition is characterized by a thourough overall revision.
Regarding the styling of the book, we have improved the readibility of
pictures, tables and program headings.
Regarding the scientific contents, we have introduced several changes in
the chapter on iterative methods for the solution of linear systems as well as
in the chapter on polynomial approximation of functions and data.
Lausanne, Milan Alfio Quarteroni
September 2006 Riccardo Sacco
Fausto Saleri
Contents
Part I Getting Started
1 Foundations of Matrix Analysis........................... 3
1.1 Vector Spaces......................................... 3
1.2 Matrices ............................................. 5
1.3 Operations with Matrices............................... 6
1.3.1 Inverse of a Matrix ............................. 7
1.3.2 Matrices and Linear Mappings ................... 8
1.3.3 Operations with Block-Partitioned Matrices ....... 9
1.4 Trace and Determinant of a Matrix ...................... 10
1.5 Rank and Kernel of a Matrix ........................... 11
1.6 Special Matrices....................................... 12
1.6.1 Block Diagonal Matrices ........................ 12
1.6.2 Trapezoidal and Triangular Matrices.............. 12
1.6.3 Banded Matrices ............................... 13
1.7 Eigenvalues and Eigenvectors ........................... 13
1.8 Similarity Transformations.............................. 15
1.9 The Singular Value Decomposition (SVD) ................ 17
1.10 Scalar Product and Norms in Vector Spaces .............. 18
1.11 Matrix Norms......................................... 22
1.11.1 Relation between Norms and the Spectral Radius
of a Matrix .................................... 25
1.11.2 Sequences and Series of Matrices ................. 26
1.12 Positive Definite, Diagonally Dominant and M-matrices .... 27
1.13 Exercises ............................................. 30
2 Principles of Numerical Mathematics ..................... 33
2.1 Well-posedness and Condition Number of a Problem ....... 33
2.2 Stability of Numerical Methods ......................... 37
2.2.1 Relations between Stability and Convergence ...... 40
2.3 A priori and a posteriori Analysis ....................... 42
X Contents
2.4 Sources of Error in Computational Models................ 43
2.5 Machine Representation of Numbers ..................... 45
2.5.1 The Positional System .......................... 45
2.5.2 The Floating-point Number System............... 46
2.5.3 Distribution of Floating-point Numbers ........... 49
2.5.4 IEC/IEEE Arithmetic .......................... 49
2.5.5 Rounding of a Real Number in its Machine
Representation................................. 50
2.5.6 Machine Floating-point Operations ............... 52
2.6 Exercises ............................................. 54
Part II Numerical Linear Algebra
3 Direct Methods for the Solution of Linear Systems ....... 59
3.1 Stability Analysis of Linear Systems ..................... 60
3.1.1 The Condition Number of a Matrix............... 60
3.1.2 Forward a priori Analysis........................ 62
3.1.3 Backward a priori Analysis ...................... 65
3.1.4 A posteriori Analysis ........................... 65
3.2 Solution of Triangular Systems .......................... 66
3.2.1 Implementation of Substitution Methods .......... 67
3.2.2 Rounding Error Analysis ........................ 69
3.2.3 Inverse of a Triangular Matrix ................... 70
3.3 The Gaussian Elimination Method (GEM) and LU
Factorization.......................................... 70
3.3.1 GEM as a Factorization Method ................. 73
3.3.2 The Effect of Rounding Errors ................... 78
3.3.3 Implementation of LU Factorization .............. 78
3.3.4 Compact Forms of Factorization.................. 80
3.4 Other Types of Factorization............................ 81
3.4.1 LDMT Factorization ............................ 81
3.4.2 Symmetric and Positive Definite Matrices: The
Cholesky Factorization.......................... 82
3.4.3 Rectangular Matrices: The QR Factorization....... 84
3.5 Pivoting.............................................. 87
3.6 Computing the Inverse of a Matrix ...................... 91
3.7 Banded Systems....................................... 92
3.7.1 Tridiagonal Matrices............................ 93
3.7.2 Implementation Issues .......................... 94
3.8 Block Systems ........................................ 96
3.8.1 Block LU Factorization ......................... 97
3.8.2 Inverse of a Block-partitioned Matrix ............. 97
3.8.3 Block Tridiagonal Systems....................... 98
3.9 Sparse Matrices ....................................... 99
Contents XI
3.9.1 The Cuthill-McKee Algorithm ...................102
3.9.2 Decomposition into Substructures ................103
3.9.3 Nested Dissection ..............................105
3.10 Accuracy of the Solution Achieved Using GEM............106
3.11 An Approximate Computation of K(A) ..................108
3.12 Improving the Accuracy of GEM ........................112
3.12.1 Scaling........................................112
3.12.2 Iterative Refinement ............................113
3.13 Undetermined Systems.................................114
3.14 Applications ..........................................117
3.14.1 Nodal Analysis of a Structured Frame.............117
3.14.2 Regularization of a Triangular Grid...............120
3.15 Exercises .............................................123
4 Iterative Methods for Solving Linear Systems ............125
4.1 On the Convergence of Iterative Methods.................125
4.2 Linear Iterative Methods ...............................128
4.2.1 Jacobi, Gauss-Seidel and Relaxation Methods......128
4.2.2 Convergence Results for Jacobi and Gauss-Seidel
Methods ......................................130
4.2.3 Convergence Results for the Relaxation Method....132
4.2.4 A priori Forward Analysis .......................133
4.2.5 Block Matrices.................................134
4.2.6 Symmetric Form of the Gauss-Seidel and SOR
Methods ......................................135
4.2.7 Implementation Issues ..........................137
4.3 Stationary and Nonstationary Iterative Methods...........138
4.3.1 Convergence Analysis of the Richardson Method ...139
4.3.2 Preconditioning Matrices ........................141
4.3.3 The Gradient Method...........................148
4.3.4 The Conjugate Gradient Method .................152
4.3.5 The Preconditioned Conjugate Gradient Method ...158
4.3.6 The Alternating-Direction Method................160
4.4 Methods Based on Krylov Subspace Iterations ............160
4.4.1 The Arnoldi Method for Linear Systems...........164
4.4.2 The GMRES Method ...........................167
4.4.3 The Lanczos Method for Symmetric Systems.......168
4.5 The Lanczos Method for Unsymmetric Systems ...........170
4.6 Stopping Criteria......................................173
4.6.1 A Stopping Test Based on the Increment ..........174
4.6.2 A Stopping Test Based on the Residual ...........175
4.7 Applications ..........................................175
4.7.1 Analysis of an Electric Network ..................176
4.7.2 Finite Difference Analysis of Beam Bending........178
4.8 Exercises .............................................180