Classical Numerical Analysis: A Comprehensive Course

937 Pages·2023·16.523 MB·English

by  Abner J. Salgado, Steven M. Wise

ClassicalNumericalAnalysis
Numericalanalysisisabroadfield,andcomingtogripswithallofitmayseemlike
a daunting task. This text provides a thorough and comprehensive exposition of all
the topics contained in a classical graduate sequence in numerical analysis. With an
emphasisontheoryandconnectionswithlinearalgebraandanalysis,thebookshows
alltherigorofnumericalanalysis.Itshighlevelandexhaustivecoveragewillprepare
studentsforresearchinthefieldandwillbecomeavaluablereferenceastheycontinue
their career. Students will appreciate the simple notation and clear assumptions and
arguments, as well as the many examples and classroom-tested exercises ranging
from simple verification to qualifying exam-level problems. In addition to the many
examples with hand calculations, readers will also be able to translate theory into
practicalcomputationalcodesbyrunningsampleMATLABcodesastheytryoutnew
concepts.
Abner J. Salgado is Professor of Mathematics at the University of Tennessee,
Knoxville.HeobtainedhisPhDinMathematicsin2010fromTexasA&MUniversity.
His main area of research is the numerical analysis of nonlinear partial differential
equations,andrelatedquestions.
StevenM.WiseisProfessorofMathematicsattheUniversityofTennessee,Knoxville.
HeobtainedhisPhDin2003fromtheUniversityofVirginia.Hismainareaofresearch
interest is the numerical analysis of partial differential equations that describe phys-
ical phenomena, and the efficient solution of the ensuing nonlinear systems. He has
authoredmorethan80publications.
Published online by Cambridge University Press
Published online by Cambridge University Press
Classical Numerical Analysis
A Comprehensive Course
ABNER J. SALGADO
UniversityofTennessee,Knoxville
STEVEN M. WISE
UniversityofTennessee,Knoxville
Published online by Cambridge University Press
ShaftesburyRoad,CambridgeCB28EA,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
314–321,3rdFloor,Plot3,SplendorForum,JasolaDistrictCentre,
NewDelhi–110025,India
103PenangRoad,#05–06/07,VisioncrestCommercial,Singapore238467
CambridgeUniversityPressispartofCambridgeUniversityPress&Assessment,
adepartmentoftheUniversityofCambridge.
WesharetheUniversity’smissiontocontributetosocietythroughthepursuitof
education,learningandresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781108837705
DOI:10.1017/9781108942607
©AbnerJ.SalgadoandStevenM.Wise2023
Thispublicationisincopyright.Subjecttostatutoryexceptionandtotheprovisions
ofrelevantcollectivelicensingagreements,noreproductionofanypartmaytake
placewithoutthewrittenpermissionofCambridgeUniversityPress&Assessment.
Firstpublished2023
PrintedintheUnitedKingdombyTJBooksLimited,PadstowCornwall
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary
LibraryofCongressCataloging-in-PublicationData
Names:Salgado,AbnerJ.,author.|Wise,StevenM.(Mathematician),author.
Title:Classicalnumericalanalysis:acomprehensivecourse/AbnerJ.Salgado,
UniversityofTennessee,Knoxville,StevenM.Wise,UniversityofTennessee,Knoxville.
Description:Cambridge,UnitedKingdom;NewYork,NY:CambridgeUniversityPress,2023.|
Includesbibliographicalreferencesandindex.
Identifiers:LCCN2022022842(print)|LCCN2022022843(ebook)|
ISBN9781108837705(hardback)|ISBN9781108942607(epub)
Subjects:LCSH:Numericalanalysis–Textbooks.|
BISAC:MATHEMATICS/MathematicalAnalysis
Classification:LCCQA297.S252023(print)|LCCQA297(ebook)|
DDC518–dc23/eng20220823
LCrecordavailableathttps://lccn.loc.gov/2022022842
LCebookrecordavailableathttps://lccn.loc.gov/2022022843
ISBN978-1-108-83770-5 Hardback
CambridgeUniversityPress&Assessmenthasnoresponsibilityforthepersistence
oraccuracyofURLsforexternalorthird-partyinternetwebsitesreferredtointhis
publicationanddoesnotguaranteethatanycontentonsuchwebsitesis,orwill
remain,accurateorappropriate.
Published online by Cambridge University Press
Contents
Preface page xiii
Acknowledgments xvii
List of Symbols xix
Part I Numerical Linear Algebra 1
1 Linear Operators and Matrices 3
1.1 Linear Operators and Matrices 3
1.2 Matrix Norms 9
1.3 Eigenvaluesand Spectral Decomposition 12
Problems 15
2 The Singular Value Decomposition 20
2.1 Reducedand Full Singular Value Decompositions 21
2.2 Existence and Uniquenessof the SVD 22
2.3 Further Properties of the SVD 25
2.4 Low Rank Approximations 27
Problems 29
3 Systems of Linear Equations 31
3.1 Solution of Simple Systems 32
3.2 LU Factorization 35
3.3 GaussianElimination with Column Pivoting 43
3.4 Implementation of the LU Factorization 50
3.5 Special Matrices 51
Problems 65
Listings 67
4 Norms and Matrix Conditioning 73
4.1 The Spectral Radius 73
4.2 Condition Number 80
4.3 Perturbationsand Matrix Conditioning 82
Problems 86
Published online by Cambridge University Press
vi Contents
5 Linear Least Squares Problem 88
5.1 Linear LeastSquares:Full RankSetting 89
5.2 Projection Matrices 93
5.3 Linear LeastSquares:TheRank-DeficientCase 98
5.4 TheQR Factorizationandthe Gram–Schmidt Algorithm 101
5.5 TheMoore–PenrosePseudo-inverse 106
5.6 TheModifiedGram–Schmidt Process 107
5.7 HouseholderReflectors 110
Problems 115
Listings 119
6 Linear Iterative Methods 121
6.1 Linear IterativeMethods 122
6.2 SpectralConvergenceTheory 124
6.3 Matrix Splitting Methods 125
6.4 Richardson’sMethod 133
6.5 RelaxationMethods 135
6.6 TheHouseholder–JohnCriterion 137
6.7 Symmetrization andSymmetric Relaxation 138
6.8 Convergencein the Energy Norm 140
6.9 A SpecialMatrix 143
6.10 Nonstationary Two-Layer Methods 145
Problems 149
Listings 154
7 Variational and Krylov Subspace Methods 156
7.1 BasicFactsaboutHPD Matrices 156
7.2 GradientDescentMethods 161
7.3 TheSteepestDescentMethod 163
7.4 TheConjugateGradientMethod 169
7.5 TheConjugateGradientMethod asa Three-Layer Scheme 183
7.6 Krylov SubspaceMethodsfor Non-HPD Problems 186
Problems 191
Listings 195
8 Eigenvalue Problems 197
8.1 EstimatingEigenvalues UsingGershgorin Disks 200
8.2 Stability 203
8.3 TheRayleighQuotient for Hermitian Matrices 205
8.4 Power IterationMethods 207
8.5 Reductionto HessenbergForm 211
8.6 TheQR Method 214
8.7 Computationof the SVD 221
Problems 223
Listings 225
Published online by Cambridge University Press
Contents vii
Part II Constructive Approximation Theory 229
9 Polynomial Interpolation 231
9.1 TheVandermondeMatrix andthe VandermondeConstruction 232
9.2 LagrangeInterpolationandthe LagrangeNodal Basis 235
9.3 TheRungePhenomenon 240
9.4 Hermite Interpolation 242
9.5 Complex Polynomial Interpolation 243
9.6 DividedDifferencesandtheNewton Construction 249
9.7 ExtendedDividedDifferences 257
Problems 263
10 Minimax Polynomial Approximation 266
10.1 Minimax: BestApproximation in the ∞-Norm 267
10.2 InterpolationError andthe LebesgueConstant 277
10.3 Chebyshev Polynomials 278
10.4 Interpolationat Chebyshev Nodes 282
10.5 BernsteinPolynomials andthe WeierstrassApproximation Theorem 286
Problems 297
11 Polynomial Least Squares Approximation 300
11.1 LeastSquaresPolynomial Approximations 301
11.2 OrthogonalPolynomials 301
11.3 ExistenceandUniquenessof the LeastSquaresApproximation 302
11.4 Propertiesof OrthogonalPolynomials 305
11.5 Convergenceof LeastSquaresApproximations 307
11.6 Uniform Convergenceof LeastSquaresApproximations 313
Problems 319
12 Fourier Series 320
12.1 LeastSquaresTrigonometricApproximations 321
12.2 Density of TrigonometricPolynomials in the Space C (0,1;C) 324
p
12.3 Convergenceof FourierSeriesin the QuadraticMean 328
12.4 Uniform Convergenceof FourierSeries 331
12.5 Convergenceof FourierSeriesin Sobolev Spaces 340
Problems 343
13 Trigonometric Interpolation and the Fast Fourier Transform 345
13.1 PeriodicInterpolationandPeriodicGrid Functions 347
13.2 TheDiscreteFourierTransform 350
13.3 ExistenceandUniquenessof the Interpolant 354
13.4 AliasError andConvergenceof TrigonometricInterpolation 356
13.5 NumericalIntegrationof PeriodicFunctions 361
13.6 TheFastFourierTransform (FFT) 363
Published online by Cambridge University Press
viii Contents
13.7 FourierMatrices,LeastSquaresApproximation, andBasic
SignalProcessing 366
Problems 371
14 Numerical Quadrature 372
14.1 QuadratureRulesfor WeightedIntegrals 373
14.2 Simple Estimatesfor Interpolatory Quadrature 376
14.3 ThePeanoKernelTheorem 378
14.4 ProperScalingandanError Estimate
Via a ScalingArgument 383
14.5 Newton–CotesFormulas 385
14.6 PeanoError Formulasfor Trapezoidal,Midpoint,
andSimpson’sRules 392
14.7 CompositeQuadratureRules 395
14.8 BernoulliNumbersandEuler–MaclaurinError Formulas 400
14.9 GaussianQuadratureRules 408
Problems 414
Part III Nonlinear Equations and Optimization 417
15 Solution of Nonlinear Equations 419
15.1 Methodsof BisectionandFalsePosition 421
15.2 Fixed PointsandContractionMappings 423
15.3 Newton’sMethod in One SpaceDimension 428
15.4 Quasi-NewtonMethods 433
15.5 Newton’sMethod in Several Dimensions 440
Problems 444
Listings 449
16 Convex Optimization 451
16.1 Some Tools from FunctionalAnalysis 451
16.2 ExistenceandUniquenessof a Minimizer 460
16.3 TheEulerEquation 463
16.4 PreconditionersandGradientDescentMethods 469
16.5 TheGoldenKey 471
16.6 PreconditionedSteepestDescentMethod 473
16.7 PSD with Approximate LineSearch 479
16.8 Newton’sMethod 482
16.9 AcceleratedGradient DescentMethods 489
16.10 NumericalIllustrations 496
Problems 498
Listings 499
Published online by Cambridge University Press
Contents ix
Part IV Initial Value Problems for Ordinary Differential Equations 507
17 Initial Value Problems for Ordinary Differential Equations 509
17.1 Existenceof Solutions 510
17.2 Uniquenessand Regularity of Solutions 515
17.3 TheFlow Map andthe Alekseev–Gro¨bnerLemma 518
17.4 DissipativeEquations 520
17.5 LyapunovStability 521
Problems 524
18 Single-Step Methods 525
18.1 Single-StepApproximation Methods 526
18.2 ConsistencyandConvergence 527
18.3 Linear Slope Functions 532
Problems 534
19 Runge–Kutta Methods 536
19.1 Simple Two-Stage Methods 537
19.2 GeneralDefinitionandBasicProperties 539
19.3 CollocationMethods 544
19.4 DissipativeMethods 550
Problems 554
20 Linear Multi-step Methods 555
20.1 Consistencyof Linear Multi-stepMethods 555
20.2 Adams–Bashforth andAdams–MoultonMethods 562
20.3 Backward DifferentiationFormula Methods 565
20.4 ZeroStability 568
20.5 Convergenceof Linear Multi-stepMethods 574
20.6 DahlquistTheorems 577
Problems 578
Listings 580
21 Stiff Systems of Ordinary Differential Equations and Linear Stability 581
21.1 TheLinear Stability Domain andA-Stability 584
21.2 A-Stability of Runge–KuttaMethods 585
21.3 A-Stability of Linear Multi-stepMethods 589
21.4 TheBoundary LocusMethod 590
Problems 593
Listings 595
22 Galerkin Methods for Initial Value Problems 596
22.1 AssumptionsandBasicDefinitions 596
22.2 CoerciveOperators:TheDiscontinuousGalerkinMethod 599
Published online by Cambridge University Press