Introduction to numerical analysis

758 Pages·2002·3.548 MB·english

by  J. Stoer, R. Bulirsch

]. Stoer R. Bulirsch
Introduction to
Numerical Analysis
Third Edition
Translated by R. Bartels, W. Gautschi, and C. Witzgall
With 39 Illustrations
Springer
J. Stoer R. Bulirsch
Institut fUr Angewandte Mathematik Institut fUr Mathematik
Universtiit Wurzburg Technische Universitiit
AM Hubland 8000 Munchen
D-97074 Wurzburg Germany
Germany
R. Bartels W. Gautschi C. Witzgall
Department of Computer Department of Computer Center for Applied
Science Sciences Mathematics
University of Waterloo Purdue University National Bureau of
Waterloo, Ontario N2L 3Gl West Lafayette, IN 47907 Standards
Canada USA Washington, DC 20234
USA
Series Editors
J.E. Marsden L. Sirovich
Control and Dynamical Systems, 107-81 Division of Applied Mathematics
California Institute of Technology Brown University
Pasadena, CA 91125 Providence, RI 02912
USA USA
M. Golubitsky S.S. Antman
Department of Mathematics Department of Mathematics
University of Houston and
Houston, TX 77204-3476 Institute for Physical Science
USA and Technology
University of Maryland
College Park, MD 20742-4015
USA
Mathematics Subject Classification (2000): 44-01, 44AIO, 44A15, 65R1O
Library of Congress Cataloging.in.Publication Data
Stoer, Josef.
[EinfOhrung in die numerische Mathematik. English]
Introduction to numerica! analysis / J. Stoer, R. Bulirsch. - 3rd ed.
p. cm. - (Texts in applied mathematics ; 12)
Includes bibliographical references and index.
ISBN 978-1-4419-3006-4 ISBN 978-0-387-21738-3 (eBook)
DOI 10.1007/978-0-387-21738-3
1. Numerica! analysis. I. Bulirsch, Roland, II. Title. III. Series.
QA297 .S8213 2002
519.4-<1c21 2002019729
Printed on acid-free paper.
© 2002, 1980, 1993 Springer Science+Business Media New York
Originally published by Springer-Verlag New York, Inc. in 2002
Softcover reprint ofthe hardcover 3rd edition 2002
AII rights reserved. This work may nat be translated or copied in whole ar in part without the written
pennission of the publisher Springer Science+Business Media, LLC,
except for brief excerpts in connection with reviews or scholarly analysis. Vse in connection with
any fonn of information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilar methodology now known or hereafter developed is forbidden.
The use of general descriptive names, tracle names, trademarks, etc., in this publication. even if the
former are not especially identified, is not to be taken as a sign that such names, as understood by the
Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.
Production managed by Timothy Taylor; manufactu!:!.ns. supernsed by Jacqui Ashri.
Photocomposed copy prepared from the author's U:l~ files using Springer's svsing.sty macro.
9 8 7 6 5 4 3 2 1
ISBN 978-1-4419-3006-4 SPIN 10867658
Series Preface
Mathematics is playing an ever more important role in the physical and
biological sciences, provoking a blurring of boundaries between scientific
disciplines and a resurgence of interest in the modern as well as the classical
techniques of applied mathematics. This renewal of interest, both in re
search and teaching, has led to the establishment of the series Texts in
Applied Mathematics (TAM).
The development of new courses is a natural consequence of a high level
of excitement on the research frontier as newer techniques, such as numeri
cal and symbolic computer systems, dynamical systems, and chaos, mix
with and reinforce the traditional methods of applied mathematics. Thus,
the purpose of this textbook series is to meet the current and future needs
of these advances and to encourage the teaching of new courses.
TAM will publish textbooks suitable for use in advanced undergraduate
and beginning graduate courses, and will complement the Applied Mathe
matical Sciences (AMS) series, which will focus on advanced textbooks and
research-level monographs.
Pasadena, California J .E. Marsden
Providence, Rhode Island L. Sirovich
Houston, Texas M. Golubitsky
College Park, Maryland S.S. Antman
Preface to the Third Edition
A new edition of a text presents not only an opportunity for corrections
and minor changes but also for adding new material. Thus we strived to
improve the presentation of Hermite interpolation and B-splines in Chap-
ter 2, and we added a new Section 2.4.6 on multi-resolution methods and
B-splines, using, in particular, low order B-splines for purposes of illustra-
tion. The intent is to draw attention to the role of B-splines in this area,
andtofamiliarizethereaderwith,atleast,theprinciplesofmulti-resolution
methods, whicharefundamentaltomodernapplicationsinsignal-andim-
age processing.
The chapter on differential equations was enlarged, too: A new Section
7.2.18 describes solving differential equations in the presence of disconti-
nuities whose locations are not known at the outset. Such discontinuities
occur, for instance, in optimal control problems where the character of a
differential equation is affected by control function changes in response to
switching events.
Many applications, such as parameter identification, lead to differential
equations which depend on additional parameters. Users then would like
to know how sensitively the solution reacts to small changes in these pa-
rameters. Techniques for such sensitivity analyses are the subject of the
new Section 7.2.19.
Multiple shooting methods are among the most powerful for solving
boundaryvalueproblemsforordinarydifferentialequations. Wededicated,
therefore,anewSection7.3.8tonewadvancedtechniquesinmultipleshoot-
ing, which especially enhance the efficiency of these methods when applied
to solve boundary value problems with discontinuities, which are typical
for optimal contol problems.
Among the many iterative methods for solving large sparse linear equa-
tions, Krylov space methods keep growing in importance. We therefore
treated these methods in Section 8.7 more systematically by adding new
subsectionsdealingwiththeGMRESmethod(Section8.7.2),thebiorthog-
onalization method of Lanczos and the (principles of the) QMR method
(Section8.7.3),andtheBi-CGandBi-CGSTABalgorithms(Section8.7.4).
Correspondingly,thefinalSection8.10onthecomparisonofiterativemeth-
ods was updated in order to incorporate the findings for all Krylov space
methods described before.
The authors are greatly indebted to the many who have contributed
vii
viii Preface to the Third Edition
to the new edition. We thank R. Grigorieff for many critical remarks on
earlier editions, M. v. Golitschek for his recommendations concerning B-
splines and their application in multi-resolution methods, and Ch. Pflaum
forhiscommentsonthechapterdealingwiththeiterativesolutionoflinear
equations. T. Kronseder and R. Callies helped substantially to establish
the new sections 7.2.18, 7.2.19, and 7.3.8. Suggestions by Ch. Witzgall,
who had helped translate a previous edition, were highly appreciated and
went beyond issues of language. Our co-workers M. Preiss and M. Wenzel
helped us read and correct the original german version. In particular, we
appreciate the excellent work done by J. Launer and Mrs. W. Wrschka
who were in charge of transcribing the full text of the new edition in TEX.
Finally we thank the Springer-Verlag for the smooth cooperation and
expertise that lead to a quick realization of the new edition.
Wu¨rzburg, Mu¨nchen J. Stoer
January 2002 R. Bulirsch
Preface to the Second Edition
Ontheoccasionofthenewedition,thetextwasenlargedbyseveralnew
sections. Two sections on B-splines and their computation were added to
the chapter on spline functions: due to their special properties, their flex-
ibility, and the availability of well tested programs for their computation,
B-splines play an important role in many applications.
Also, the authors followed suggestions by many readers to supplement
the chapter on elimination methods by a section dealing with the solution
of large sparse systems of linear equations. Even though such systems
are usually solved by iterative methods, the realm of elimination methods
has been widely extended due to powerful techniques for handling sparse
matrices. We will explain some of these techniques in connection with the
Cholesky algorithm for solving positive definite linear systems.
The chapter on eigenvalue problems was enlarged by a section on the
Lanczosalgorithm; thesectionsontheLR-andQRalgorithmwererewrit-
ten and now contain also a description of implicit shift techniques.
In order to take account of the progress in the area of ordinary dif-
ferential equations to some extent, a new section on implicit differential
equations and differential-algebraic systems was added, and the section on
stiff differential equations was updated by describing further methods to
solve such equations.
Also the last chapter on the iterative solution of linear equations was
improved. The modern view of the conjugate gradient algorithm as an
iterative method was stressed by adding an analysis of its convergence
rate and a description of some preconditioning techniques. Finally, a new
section on multigrid methods was incorporated: It contains a description
of their basic ideas in the context of a simple boundary value problem for
ordinary differential equations.
ix
x Preface to the Second Edition
Manyofthechangesweresuggestedbyseveralcolleaguesandreaders. In
particular, we would like to thank R. Seydel, P. Rentrop and A. Neumaier
for detailed proposals, and our translators R. Bartels, W. Gautschi and C.
Witzgall for their valuable work and critical commentaries. The original
German version was handled by F. Jarre, and I. Brugger was responsible
for the expert typing of the many versions of the manuscript.
Finally we thank the Springer-Verlag for the encouragement, patience
and close cooperation leading to this new edition.
Wu¨rzburg, Mu¨nchen J. Stoer
May 1991 R. Bulirsch
Contents
Preface to the Third Edition VII
PrefacetotheSecondEditionIX
1 Error Analysis 1
1.1 Representation of Numbers 2
1.2 Roundoff Errors and Floating-Point Arithmetic 4
1.3 Error Propagation 9
1.4 Examples 21
1.5 Interval Arithmetic; Statistical Roundoff Estimation 27
Exercises for Chapter 1 33
References for Chapter 1 36
2 Interpolation 37
2.1 Interpolation by Polynomials 38
2.1.1 Theoretical Foundation: The Interpolation Formula of Lagrange 38
2.1.2 Neville’s Algorithm 40
2.1.3 Newtons Interpolation Formula: Divided Differences 43
2.1.4 The Error in Polynomial Interpolation 48
2.1.5 Hermite Interpolation 51
2.2 Interpolation by Rational Functions 59
2.2.1 General Properties of Rational Interpolation 59
2.2.2 Inverse and Reciprocal Differences. Thiele’s Continued Fraction 64
2.2.3 Algorithms of the Neville Type 68
2.2.4 Comparing Rational and Polynomial Interpolation 73
2.3 Trigonometric Interpolation 74
2.3.1 Basic Facts 74
2.3.2 Fast Fourier Transforms 80
2.3.3 The Algorithms of Goertzel and Reinsch 88
2.3.4 The Calculation of Fourier Coefficients. Attenuation Factors 92
xi
xii Contents
2.4 Interpolation by Spline Functions 97
2.4.1 Theoretical Foundations 97
2.4.2 Determining Interpolating Cubic Spline Functions 101
2.4.3 Convergence Properties of Cubic Spline Functions 107
2.4.4 B-Splines 111
2.4.5 The Computation of B-Splines 117
2.4.6 Multi-Resolution Methods and B-Splines 121
Exercises for Chpater 2 134
References for Chapter2 143
3 Topics in Integration 145
3.1 The Integration Formulas of Newton and Cotes 146
3.2 Peano’s Error Representation 151
3.3 The Euler-Maclaurin Summation Formula 156
3.4 Integration by Extrapolation 160
3.5 About Extrapolation Methods 165
3.6 Gaussian Integration Methods 171
3.7 Integrals with Singularities 181
Exercises for Chapter 3 184
References for Chapter 3 188
4 Systems of Linear Equations 190
4.1 Gaussian Elimination. The Triangular Decomposition of a Matrix 190
4.2 The Gauss-Jordan Algorithm 200
4.3 The Choleski Decompostion 204
4.4 Error Bounds 207
4.5 Roundoff-Error Analysis for Gaussian Elimination 215
4.6 Roundoff Errors in Solving Triangular Systems 221
4.7 Orthogonalization Techniques of Householder and Gram-Schmidt 223
4.8 Data Fitting 231
4.8.1 Linear Least Squares. The Normal Equations 232
4.8.2 The Use of Orthogonalization in Solving Linear Least-Squares
Problems 235
4.8.3 The Condition of the Linear Least-Squares Problem 236
4.8.4 Nonlinear Least-Squares Problems 241
4.8.5 The Pseudoinverse of a Matrix 243
4.9 Modification Techniques for Matrix Decompositions 247
4.10 The Simplex Method 256
4.11 Phase One of the Simplex Method 268
4.12 Appendix: Elimination Methods for Sparse Matrices 272
Exercises for Chapter 4 280
References for Chapter 4 286