Guaranteed Accuracy in Numerical Linear Algebra

544 Pages·1993·15.29 MB·English

by  S. K. Godunov

Guaranteed Accuracy in Numerical Linear Algebra
Mathematics and Its Applications
Managing Editor:
M. HAZEWINKEL
Centre for Mathematics and Computer Science, Amsterdam, The Netherlands
Volume 252
Guaranteed Accuracy in
Numerical Linear Algebra
by
S. K. Godunov
A. G. Antonov
O. P. Kiriljuk
and
V.I. Kostin
Institute of Mathematics.
Novosibirsk. Siberia
Springer-Science+Business Media, B.V.
A C.I.P. Catalogue record for this book is available from the Library of Congress.
ISBN 978-94-010-4863-7 ISBN 978-94-011-1952-8 (eBook)
DOI 10.1007/978-94-011-1952-8
This is an updated and revised translation of the original work
The Guaranteed Precision of Linear Equations Solutions in Euclidean Spaces.
© 1988 Nauka, Novosibirsk, © 1992 second revised edition Nauka, Novosibirsk
Printed on acid-free paper
All Rights Reserved
© 1993 Springer Science+Business Media Dordrecht
Originally published by Kluwer Academic Publishers in 1993
Softcover reprint of the hardcover 3rd edition 1993
No part of the material protected by this copyright notice may be reproduced or
utilized in any form or by any means, electronic or mechanical,
including photocopying, recording or by any information storage and
retrieval system, without written permission from the copyright owner.
Contents
Introduction ix
1 Singular Value Decomposition 1
1.1 Singular Value Decomposition and Singular Values of
Square Matrix . . . . . . . . . . . . . . . . . . . . . .. 3
1.2 Elementary Orthogonal Transformations . . . . . . .. 9
1.3 Singular Value Decomposition of Rectangular Matrices 24
1.4 Norm of Matrix. Singular Values and Singular Vectors 34
1.5 Some Numerical Characteristics of Matrices ...... 49
1.6 Some Properties of Bidiagonal Square Matrices. Singular
Values and Singular Vectors . . . . . . . . . . . . . .. 58
1. 7 Simplification of Matrix Form by Usage of Orthogonal
Transformations . . . . . . . . . . . . . . . . 71
1.8 Simplification of Matrix Form by Deflation . 86
1.9 Extension of Results for Complex Matrices 98
2 Systems of Linear Equations 109
2.1 Condition Number for Square Matrix ........... 111
2.2 Systems of Linear Equations with Simplest Band Matri-
ces of Coefficients . . . . . . . . . . . . . . . . . . . . . . 121
2.3 Generalized Normal Solutions of Systems with Arbitrary
Matrices of Coefficients . . . . . . . . . . . . . . . . . . . 136
2.4 Conditioning of Generalized Normal Solutions of Sys-
tems of Full Rank. . . . . . . . . . . . . . . . . . 153
2.5 Angles between Spaces and Their Conditioning . 162
vi
2.6 Conditioning of the Generalized Normal Solutions
in Case of Not Full Rank ........................ 181
2.7 Generalized Normal r-solution of the Systems of Linear
Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 193
2.8 General Scheme of Finding of r-solution of Linear System 207
3 Deflation Algorithms for Band Matrices 215
3.1 Transformations of Hessenberg Matrices by Chains of
Rotations ................... . 217
3.2 Deflation of Degenerate Bidiagonal Matrices . 235
3.3 Singular Deflation of Non-Degenerate Bidiagonal Matrices247
3.4 Spectral Deflation of Hessenbergian and Symmetric
Tridiagonal Matrices . . . . . . . . . . . . . . . . . . . . 259
3.5 Theory of Perturbations of Singular Deflation of Non
Degenerate Bidiagonal Matrices . . . . . . . . . . . . . . 267
3.6 Theory of Perturbations of Singular Deflation of Degen-
erate Bidiagonal Matrices ................. 290
3.7 Theory of Perturbations of Singular Deflation of Sym-
metric Tridiagonal Matrices . . . . . . . . .296
4 Sturm Sequences of Tridiagonal Matrices 313
4.1 Elementary Proof of Sturm Theorem ... .315
4.2 Algorithm of Computation of Eigenvalues of Symmetric
Tridiagonal Matrix . . . . . . . . . . . . . . . . . .. . 323
4.3 Trigonometric Parametrization of Rational Relations . 336
4.4 Sturm Sequences of Second Kind ........... . 349
4.5 One-Side Sturm Sequences for Tridiagonal Matrices. . 355
4.6 Two-Side Sturm Sequences for Tridiagonal Symmetric
Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
4.7 Examples of Calculations in Problems of Finding Eigen-
values and Sturm Sequences . . . . . . . . . . . .. . 391
4.8 Two-Side Sturm Sequences for Bidiagonal Matrices . 404
Vll
4.9 Examples of Computations of Singular Values and Two-
Side Sturm Sequences of Bidiagonal Matrices. . . . . . . 417
5 Peculiarities of Computer Computations 425
5.1 Modelling of Errors in Computer Arithmetic Operations 428
5.2 Machine Operations on Vectors and Matrices ....... 443
5.3 Machine Realization of Reflections. . . . . . . . . . . . . 452
5.4 Analysis of Errors in Reduction of Matrices into Bi - and
Tridiagonal Form . . . . . . . . . . . . . . . . . . . . . . 462
5.5 Machine Solution of Systems of Equations with Bidiag-
onal Coefficient Matrices . . . . . . . . . . . . . . . . . . 471
5.6 Numerical Examples .................... 478
5.7 Machine Realization of Sturm Algorithm. Estimates of
Errors in Computation of Eigenvalues and Singular Values484
5.8 Computation of Two-Side Sturm Sequence and Com
ponents of Eigenvector of Tridiagonal Symmetric Matrix 493
5.9 Machine Realization of Computations of Two-Side
Sturm Sequences for Bidiagonal Matrices . . . . . 502
5.10 Machine Realization of Deflation Algorithm for
Bidiagonal Matrices . .508
Bibliography 523
Index 529
Introduction
There exists a vast literature on numerical methods of linear algebra.
In our bibliography list, which is by far not complete, we included some
monographs on the subject [46], [15], [32], [39], [11], [21]. The present
book is devoted to the theory of algorithms for a single problem of linear
algebra, namely, for the problem of solving systems of linear equations
with non-full-rank matrix of coefficients. The solution of this problem
splits into many steps, the detailed discussion of which are interest
ing problems on their own (bidiagonalization of matrices, computation
of singular values and eigenvalues, procedures of deflation of singular
values, etc.). Moreover, the theory of algorithms for solutions of the
symmetric eigenvalues problem is closely related to the theory of solv
ing linear systems (Householder's algorithms of bidiagonalization and
tridiagonalization, eigenvalues and singular values, etc.).
It should be stressed that in this book we discuss algorithms which
lead to computer programs having the virtue that the accuracy of com
putations is guaranteed. As far as the final program product is con
cerned, this means that the user always finds an unambiguous solution
of his problem. This solution might be of two kinds:
1. Solution of the problem with an estimate of errors, where abso
lutely all errors of input data and machine round-offs are taken
into account.
2. Rejection of the problem in the case when it is ill-conditioned,
with the guaranteed estimate of parameters proving it impossible
IX
x Introduction
to perform the program with an acceptable error estimate.
For example, in the course of solving of systems of linear equations
with a square matrix of coefficients, a value of the condition number is
essential. Namely, if the condition number is not too large (a threshold
is defined by the level of errors of input data and of the intermediate
computations), then, as a result, a solution with error estimate will be
found. Otherwise, as a result of running the program, the problem is
rejected with the estimate of the condition number from below.
Depending on the problem, there may be several parameters, the
values of which condition the solvability of the problem. A large part
of this book is devoted to revealing these parameters. The contents of
this part can be briefly characterized as 'background error analysis'. In
fact, this analysis is based on the corresponding perturbation theory.
The other, not less important part of our discussion is devoted to
the so-called 'forward error analysis'. Errors of arithmetic operations
arising from round-offs are modelled by equivalent perturbations of
operands. This is the reason why a chapter of this book is devoted to the
theory of placing numbers in computer memory, arithmetic operations
on them, and round-offs. Basing on some idealization, in this chapter
we present some facts concerning arithmetic operations on computers
and examples of error estimates resulting from simple algorithms. The
number of computer types used as examples is rather limited and most
machines (especially those popular in the West) remain outside the
scope of the discussion. However, this does not diminish the importance
of this chapter, because the goal is to present the necessary formalism
which can be applied for any computer.
The algorithms discussed in this book require various accuracies of
arithmetic operations in intermediate computations. For many of them,
the standard machine arithmetics is sufficient, however, some require
more care. For example, in the algorithms of deflation of singular val
ues of bidiagonal matrices, it is necessary for errors to have a relative
character. In order to guarantee this, we use the so-called scaled (or
Introduction Xl
normalized) arithmetics.
Even though the algorithms described impose no restrictions on in
put data, a number of restrictions result from practical considerations.
These are a finite volume of internal memory, finite size of stored num
bers, etc. Depending on the internal memory, sizes of matrices may
reach several hundreds. Let us stress that we do not consider the tech
nology of sparse matrices which makes it possible to save memory dur
ing computation [9] (if, of course, one does not regard bidiagonal and
tridiagonal matrices as sparse).
Clearly, reliable programs for solving problems with dense matrices
of small sizes should be included into routines designed to solve more
complex problems. The algorithms presented in this book may, in our
opinion, play this role.
The set of algorithms satisfying the requirement of guaranteed accu
racy is not exhausted by those presented in this book. For a discussion
of the non-symmetric eigenvalue problem, see, for example, [6]. Re
cently, alternative approaches to the problem of the construction of
reliable algorithms for linear algebra and to estimates of errors of nu
merical solutions of problems have been developed (see, e.g., [7], [8],
[8]).
In the course of work on the manuscript, we benefited from dis
cussions with A.Ya. Bulgakov, A.N. Malyshev, S.V. Kuznetsov, A.D.
Mit chenko , S.V. Fadeev, and Yu.V. Surnin. J. Kowalski-Glikman not
only excellently translated and typeset the book, but also helped us
remove a number of inaccuracies. We would like to thank all of them.
The authors Novosibirsk, 1992