Numerical Analysis and Scientific Computation

583 Pages·2022·10.963 MB·English

by  Jeffery J. Leader

Numerical Analysis and
Scientific Computation
Textbooks in Mathematics
Series editors:
Al Boggess, Kenneth H. Rosen
Linear Algebra
An Inquiry-based Approach
Jeff Suzuki
The Geometry of Special Relativity
Tevian Dray
Mathematical Modeling in the Age of the Pandemic
William P. Fox
Games, Gambling, and Probability
An Introduction to Mathematics
David G. Taylor
Linear Algebra and Its Applications with R
Ruriko Yoshida
Maple™ Projects of Differential Equations
Robert P. Gilbert, George C. Hsiao, Robert J. Ronkese
Practical Linear Algebra
A Geometry Toolbox, Fourth Edition
Gerald Farin, Dianne Hansford
An Introduction to Analysis, Third Edition
James R. Kirkwood
Student Solutions Manual for Gallian’s Contemporary Abstract Algebra, Tenth Edition
Joseph A. Gallian
Elementary Number Theory
Gove Effinger, Gary L. Mullen
Philosophy of Mathematics
Classic and Contemporary Studies
Ahmet Cevik
An Introduction to Complex Analysis and the Laplace Transform
Vladimir Eiderman
An Invitation to Abstract Algebra
Steven J. Rosenberg
Numerical Analysis and Scientific Computation, Second Edition
Jeffery J. Leader
https://www.routledge.com/Textbooks-in-Mathematics/book-series/CANDHTEXBOOMTH
Numerical Analysis and
Scientific Computation
Second Edition
Jeffery J. Leader
Fifth edition published 2022
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
© 2022 Jeffery J. Leader
First edition published by Pearson 2004
CRC Press is an imprint of Taylor & Francis Group, LLC
Reasonable efforts have been made to publish reliable data and information, but the author and publisher
cannot assume responsibility for the validity of all materials or the consequences of their use. The authors
and publishers have attempted to trace the copyright holders of all material reproduced in this publica-
tion and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any future
reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, trans-
mitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter
invented, including photocopying, microfilming, and recording, or in any information storage or retrieval
system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.com or
contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400.
For works that are not available on CCC please contact [email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used
only for identification and explanation without intent to infringe.
ISBN: 978-0-367-48686-0 (hbk)
ISBN: 978-1-032-20483-3 (pbk)
ISBN: 978-1-003-04227-3 (ebk)
DOI: 10.1201/9781003042273
Publisher’s note: This book has been prepared from camera-ready copy provided by the authors
To my wife,
Meg.

Contents
Preface ix
Preface to the 2nd Edition, Student xiii
Preface to the 1st Edition xv
Chapter 1. Nonlinear Equations 1
1. Bisection and Inverse Linear Interpolation 1
2. Newton’s Method 11
3. The Fixed Point Theorem 21
4. Quadratic Convergence of Newton’s Method 28
5. Modifications of Newton’s Method 41
6. Brent’s Method 51
7. Effects of Finite Precision Arithmetic 58
8. Polynomial Roots 70
Chapter 2. Linear Systems 81
1. Gaussian Elimination with Partial Pivoting 81
2. The LU Decomposition 93
3. The LU Decomposition with Pivoting 105
4. Newton’s Method for Systems 120
5. The Cholesky Decomposition 131
6. Conditioning 143
7. The QR Decomposition and Least Squares 156
8. The QR Decomposition by Triangularization 168
9. The QR Decomposition by Orthogonalization 180
10. The Singular Value Decomposition 192
Chapter 3. Iterative Methods 199
1. Jacobi and Gauss-Seidel Iteration 199
2. Sparsity 211
3. Iterative Refinement 221
4. Preconditioning 227
5. Krylov Space Methods 233
6. Numerical Eigenproblems 246
7. The QR Method of Francis 254
vii
viii CONTENTS
Chapter 4. Polynomial Interpolation 265
1. Lagrange Interpolating Polynomials 265
2. Piecewise Linear Interpolation 280
3. Cubic Splines 292
4. Computation of the Cubic Spline Coefficients 302
5. Rational Function Interpolation 313
6. The Best Approximation Problem 322
Chapter 5. Numerical Integration 333
1. Closed Newton-Cotes Formulas 333
2. Open Newton-Cotes Formulas 348
3. Gaussian Quadrature 361
4. Gauss-Chebyshev Quadrature 373
5. Adaptivity and Automatic Integration 381
6. Automatic Integration with Gauss Rules 392
7. The Trapezoidal Rule with Extrapolation 401
8. Periodizing Transformations 410
Chapter 6. Differential Equations 421
1. Numerical Differentiation 421
2. Euler’s Method 431
3. Improved Euler’s Method 441
4. Analysis of Explicit One-Step Methods 448
5. Taylor and Runge-Kutta Methods 457
6. Adaptivity and Stiffness 466
7. Multi-step Methods 474
Chapter 7. Nonlinear Optimization 483
1. One-Dimensional Searches 483
2. The Method of Steepest Descent 491
3. Newton Methods for Nonlinear Optimization 500
4. Multiple Random Start Methods 509
5. Direct Search Methods 517
6. The Nelder-Mead Method 525
7. Conjugate Direction Methods 531
8. Hansen’s Method 539
Afterword 549
Index 553
Preface
Preface to the 2nd Edition, Instructor
The fundamental law of computer science: As machines become more powerful, the
efficiency of algorithms grows more important, not less. Nick Trefethen
Surely [one imagines], we compute only when everything else fails, when math-
ematical theory cannot deliver an answer in a comprehensive, pristine form and
thus we are compelled to throw a problem onto a number-crunching computer and
produce boring numbers by boring calculations. This, I believe, is nonsense. Arieh
Iserles
This book represents the numerical analysis course I regularly teach at Rose-
HulmanInstituteofTechnology,whichistakenasanelectivecoursebyjuniorsand
seniorsmajoringincomputationalscience,computerscience,mathematics,physics,
and a variety of engineering disciplines. The students will have had three terms
of calculus as well as differential equations with matrix algebra, but may not have
had any real programming experience. The key features of my approach have been
an immediate immersion in numerical methods, delaying a discussion of computer
arithmetic and round-off error until students have gained some experience using
numerical algorithms; a presentation of numerical linear algebra that more closely
mirrorshowitisactuallyimplementedandusedinmodernpractice;andanempha-
sis on analysis while still bringing out the practical hardware and software issues
involvedinmodernscientificcomputing. Inaddition,optionalMATLAB(cid:13)R subsec-
tionsallowforstudentstoteachthemselvesbasicprogrammingwhileexperimenting
with the methods of the section. I want students to come away with a strong un-
derstandingofthemathematicalunderpinningofmodernscientificcomputing–and
also be prepared to apply that knowledge in a practical way, whether they are do-
ing the coding themselves or, more likely, using what they’ve learned to get better
performance out of the software packages that are common in their field.
Chapter 1 covers root-finding in one-dimension, which should be a comfort-
ingly familiar sort of problem for students. Next is numerical linear algebra (direct
methods in Chapter 2 and iterative methods and eigenvalues in Chapter 3) be-
cause of its crucial importance both on its own and in other algorithms. Chapter
4 is devoted to polynomial interpolation and splines. Chapter 5 covers numerical
quadrature, including Gaussian quadrature and adaptivity; the numerical solution
of ODEs is covered in an intentionally similar manner in Chapter 6. Chapter 7
surveys nonlinear optimization methods.
The MATLAB subsections provide a tutorial resource that presents a slow
introduction to MATLAB that, by the end of the text, will have resulted in a very
broadknowledgeofthesoftwareaswellassubstantiveprogrammingskills. Students
ix