Extrapolation methods : theory and practice

465 Pages·1991·18.964 MB·English

by  Claude Brezinski, Michela Redivo Zaglia

STUDIES IN
COMPUTATIONAL MATHEMATICS 2
Editors:
C. BREZINSKI
University of Lille
Villeneuve d'Ascq, France
L. WUYTACK
University of Antwerp
Wilrijk, Belgium
ELSEVIER
Amsterdam - Boston - London - New York - Oxford - Paris - San Diego
San Francisco - Singapore - Sydney - Tokyo
EXTRAPOLATION METHODS
THEORY AND PRACTICE
Claude BREZINSKI
Universite des Sciences et Technologies de Lille
Villeneuve d'Ascq, France
Michela REDIVO ZAGLIA
Universitä degli Studi di Padova
Padova, Italy
ELSEVIER
Amsterdam - Boston - London - New York - Oxford - Paris - San Diego
San Francisco - Singapore - Sydney - Tokyo
ELSEVIER SCIENCE PUBLISHERS B.V.
Sara Burgerhartstraat 25
P.O. Box 211, 1000 AE Amsterdam, The Netherlands
©1991 ELSEVIER SCIENCE PUBLISHERS B.V. All rights reserved.
This work is protected under copyright by Elsevier Science, and the following terms and conditions apply to its use:
Photocopying
Single photocopies of single chapters may be made for personal use as allowed by national copyright laws. Permission of the
Publisher and payment of a fee is required for all other photocopying, including multiple or systematic copying, copying for
advertising or promotional purposes, resale, and all forms of document delivery. Special rates are available for educational
institutions that wish to make photocopies for non-profit educational classroom use.
Permissions may be sought directly from Elsevier Science Global Rights Department, PO Box 800, Oxford OX5 1DX, UK; phone:
([phone], fax: [phone], e-mail: [email protected]. You may also contact Global Rights directly
through Elsevier's home page (http://www.elsevier.com), by selecting Obtaining Permissions'.
In the USA, users may clear permissions and make payments through the Copyright Clearance Center, Inc., 222 Rosewood Drive,
Danvers, MA 01923, USA; phone: (+1) (978) 7508400, fax: (+1) (978) 7504744, and in the UK through the Copyright Licensing
Agency Rapid Clearance Service (CLARCS), 90 Tottenham Court Road, London W1P 0LP, UK; phone: [phone]; fax:
([phone]. Other countries may have a local reprographic rights agency for payments.
Derivative Works
Tables of contents may be reproduced for internal circulation, but permission of Elsevier Science is required for external resale or
distribution of such material.
Permission of the Publisher is required for all other derivative works, including compilations and translations.
Electronic Storage or Usage
Permission of the Publisher is required to store or use electronically any material contained in this work, including any chapter or
part of a chapter.
Except as outlined above, no part of this work may be reproduced, stored in a retrieval system or transmitted in any form or by any
means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission of the Publisher.
Address permissions requests to: Elsevier Science Global Rights Department, at the mail, fax and e-mail addresses noted above.
Notice
No responsibility is assumed by the Publisher for any injury and/or damage to persons or property as a matter of products liability,
negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material
herein. Because of rapid advances in the medical sciences, in particular, independent verification of diagnoses and drug dosages
should be made.
First edition 1991
Second impression 2002
Library of Congress Cataloging-in-Publication Data
Brezinski, Claude, 1941-
Extrapolation methods : theory and practice / Claude Brezinski,
Michela Redivo Zaglia.
p. cm. -- (Studies in computational mathematics ; 2)
Includes bibliographical references and index.
ISBN 0-444-88814-4
1. Extrapolation. 2. Extrapolation--Data processing. I. Redivo
Zaglia, Michela. II. Title. III. Series.
QA281.B74 1991
511' .42--dc20 91-33014
CIP
This book is sold in conjunction with a diskette.
ISBN SET: 0 444 88814 4
Θ The paper used in this publication meets the requirements of ANSI/NISO Z39.48-1992 (Permanence of
Paper).
Printed in The Netherlands.
PREFACE
In numerical analysis, in applied mathematics and in engineering one has
often to deal with sequences and series. They are produced by iterative
methods, perturbation techniques, approximation procedures depending
on a parameter, and so on. Very often in practice those sequences or
series converge so slowly that it is a serious drawback to their effective
use. This is the reason why convergence acceleration methods have been
studied for many years and applied to various situations. They are based
on the very natural idea of extrapolation and, in many cases, they lead
to the solution of problems which were unsolvable otherwise. Extrapola­
tion methods now constitute a particular domain of numerical analysis
having connections with many other important topics as Pade approxi­
mation, continued fractions, formal orthogonal polynomials, projection
methods to name a few. They also form the basis of new methods for
solving various problems and have many applications as well. Analytical
methods seem to become more and more in favour in numerical analysis
and applied mathematics and thus one can think (and we do hope) that
extrapolation procedures will become more widely used in the future.
The aim of this book is twofold. First it is a self-contained and, as
much as possible, exhaustive exposition of the theory of extrapolation
methods and of the various algorithms and procedures for accelerating
the convergence of scalar and vector sequences. Our second aim is to
convince people working with sequences to use extrapolation methods
and to help them in this respect. This is the reason why we provide many
subroutines (written in FORTRAN 77) with their directions for use. We
also include many numerical examples showing the effectiveness of the
procedures and a quite consequent chapter on applications. In order to
reduce the size of the book the proofs of the theoretical results have been
omitted and replaced by references to the existing literature. However,
on the other hand, some results and applications are given here for the
first time. We have also included suggestions for further research.
vi Preface
The first chapter is a general presentation of extrapolation methods
and algorithms. It does not require any special knowledge and gives
the necessary prerequisites. The second chapter is devoted to the algo­
rithms for accelerating scalar sequences. Special devices for a better use
of extrapolation procedures are given in the third chapter. Chapter four
deals with acceleration of vector sequences while chapter five presents
the so-called continuous prediction algorithms for functional extrapola­
tion. The sixth chapter is quite a big one. It is devoted to applications
of extrapolation methods which range from the solution of systems of
equations, differential equations, quadratures to problems in statistics.
The last chapter presents the subroutines. They have been written in
order to be as portable as possible and can be found on the floppy disk
included in this book with the main programs and the numerical results.
They are all new.
We intend to write a book which can be of interest for researchers in
the field and to those needing to use extrapolation methods for solving
a particular problem. We also hope that it can be used for graduate
courses on the subject.
It is our pleasure to thank our colleagues and students who partici­
pate, directly or not, to the preparation of this monograph. In particular
some of them read the manuscript or parts of it and made several im­
portant comments. We would like to specially express our gratitude
to M. Calvani, G. F. Cariolaro, F. Cordellier, A. Draux, B. Germain-
Bonne, A. M. Litovsky, A. C. Matos, S. Paszkowski, M. Pichat, M. Pinar,
M. Prevost, V. Ramirez, H. Sadok, A. Sidi and J. van Iseghem.
We would like to thank M. Morandi Cecchi for inviting C. Brezinski
to the University of Padova for a one month stay during which the book
was completed.
A special thank is also due to F. J. van Drunen, J. Butterfield and
A. Carter from North-Holland Publishing Company for their very ef­
ficient assistance in the preparation of the book and to M. Agnello,
A. Calore and R. Lazzari from the University of Padova who typed the
textual part of the manuscript.
Claude Brezinski Michela Redivo Zaglia
Universite des Sciences et Universita degli Studi di Padova
Technologies de Lille
Chapter 1
INTRODUCTION TO THE THEORY
1.1 First steps
The aim of this chapter is to be an introduction to convergence accelera­
tion methods which are usually obtained by an extrapolation procedure.
Let (5 ) be a sequence of (real or complex) numbers which converges
n
to 5. We shall transform the sequence (5 ) into another sequence (T )
n n
and denote by T such a transformation.
For example we can have
T = , 71 = 0,1,...
n
or
τ S S +2 — Sn+i
n n n 1
which is the well-known Δ2 process due to Aitken [6].
In order to present some practical interest the new sequence (T ) must
n
exhibit, at least for some particular classes of convergent sequences (5 ),
n
the following properties
1. (T ) must converge
n
2. (T ) must converge to the same limit as (5„)
n
3. (T ) must converge to 5 faster than (5 ), that is
n n
Urn (T„ - S)/(S - S) = 0.
n
n—K»
• In the case 2, we say that
the transformation T is regular for the sequence (5 ).
n
2 Chapter 1. Introduction to the theory
• In the case 3, we say that
the transformation T accelerates the convergence of the sequence
(S ) or that
n
the sequence (T ) converges faster] than (5„)
n
Usually these properties do not hold for all converging sequences
(S ) and, in particular, the last one since, as proved by Delahaye and
n
Germain-Bonne [139], a universal transformation T accelerating all the
converging sequences cannot exist (this question will be developed in
section 1.10). This negative result also holds for some classes of se­
quences such as the set of monotone sequences or that of logarithmic
sequences (that is such that lim (S +i - S)/(S - S) = 1). Thus, this
n n
n—»oo
negative result means that it will be always interesting to find and to
study new sequence transformations since, in fact, each of them is only
able to accelerate the convergence of certain classes of sequences.
What is now the answer to the first two above mentioned properties?
The first example was a linear transformation for which it is easy to see
that, for all converging sequence (5 ), the sequence (T ) converges and
n n
has the same limit as (5 ). Such linear transformations, called summa­
n
tion processes, have been widely studied and the transformations named
after Euler, Cesaro, HausdorfF, Abel and others, are well known. The
positive answer to properties 1 and 2 above for all convergent sequences
is a consequence of the so-called Toeplitz theorem which can be found
in the literature and whose conditions are very easily checked in prac­
tice. Some summation processes are very powerful for some sequences
as is the case with Romberg's method for accelerating the trapezoidal
rule which is explained in any textbook of numerical analysis. However
let us look again at our first transformation and try to find the class of
sequences which it accelerates. We have
1n-s l
-s 2 V s -s )
5, n
and thus
lim (T - S)/{S - S) = 0
n n
if and only if
lim(5 -5)/(5 -5) = -l
n+1 n
1.1. First steps 3
which shows that this transformation is only able to accelerate the con­
vergence of a very restricted class of sequences. This is mainly the case
for all summation processes.
On the other hand, let us now look at our second sequence trans­
formation which is Aitken's Δ2 process. It can be easily proved that
it accelerates the convergence of all the sequences for which it exists
λ 6 [~1,+1[ such that
lim (5 - S)/(S - S) = λ
n+1 n
n—»oo
which is a much wider class than the class of sequences accelerated by
our first linear transformation. But, since in mathematics as in life
nothing can be obtained without pain, the drawback is that the answer
to properties 1 and 2 is no more positive for all convergent sequences.
Examples of convergent sequences (5 ) for which the sequence (T ) ob­
n n
tained by Aitken's process has two accumulation points, are known, see
section 2.3. But it can also be proved that if such a (T ) converges, then
n
its limit is the same as the limit of the sequence (5„), see Tucker [440].
In conclusion, nonlinear sequence transformations usually have better
acceleration properties than linear summation processes (that is, they
accelerate wider classes of sequences) but, on the other hand, they do
not always transform a convergent sequence into another converging
sequence and, even if so, both limits can be different.
In this book we shall be mostly interested by nonlinear sequence
transformations. Surveys on linear summation processes were given by
Joyce [253], Powell and Shah [361] and Wimp [465]. One can also con­
sult Wynn [481], Wimp [463, 464], Niethammer [334], Gabutti [168],
Gabutti and Lyness [170] and Walz [451] among others where interest­
ing developments and applications of linear sequence transformations
can be found.
There is another problem which must be mentioned at this stage.
When using Aitken's process, the computation of T uses 5 , 5 +i and
n n n
5„+2- For some sequences it is possible that lim (T — S)/(S — S) = 0
n n
n—*oo
and that lim (T - S)/(S i - S) or Um (T - S)/(S - S) be differ-
n n+ n n+2
ent from zero. In particular if lim (5 +i — S)/(S — S) = 0 then (T )
n n n
n—>oo
obtained by Aitken's process converges faster than (S ) and (S +i) but
n n
not always faster than (S +2)· Thus, in the study of a sequence trans­
n
formation, it would be better to look at the ratio (T — S)/(S +k — S)
n n
4 Chapter 1. Introduction to the theory
where 5 +fc is the term with the greatest index used in the computation
n
of T . However it must be remarked that
n
T — S _ T — S ^ S — S ^n-ffc-i ~ S
n n n
S +k — S S — S S +\ — S S +k — S
n n n n
which shows that if (T — S)/(S — S) tends to zero and if (5 +i —
n n n
S)/(S — S) is always away from zero and do not tend to it, then the
n
ratio (T — S)/(S +k — S) also tends to zero. In practice, avoiding a null
n n
limit for (5„+i — S)/(S — S) is not a severe restriction since, in such a
n
case, (S ) converges fast enough and does not need to be accelerated.
n
We shall now exemplify some interesting properties of sequence trans­
formations on our two preceding examples. In the study of a sequence
transformation the first question to be asked and solved (before those
of convergence and acceleration) is an algebraic one: it concerns the
so-called kernel of the transformation that is the set of sequences for
which 35 such that Vn, T = 5 (in the sequel Vn would eventually mean
n
Vn > N).
For our linear summation process it is easy to check that its kernel is
the set of sequences of the form
S = S + a(-l)n
n
where a is a scalar.
For Aitken's process the kernel is the set of sequences of the form
S = S + a\n
n
where a and λ are scalars with a φ 0 and λ φ 1.
Thus, obviously, the kernel of Aitken's process contains the kernel of the
first linear summation process.
As we can see, in both cases, the kernel depends on some (almost)
arbitrary parameters, S and a in the first case, 5, a and \{φ 1) in the
second.
If the sequence (5 ) to be accelerated belongs to the kernel of the
n
transformation used then, by construction, we shall have Vn, T — S.
n
Of course, usually, 5 is the limit of the sequence (5 ) but this is not
n
always the case and the question needs to be studied. For example, in
Aitken's process, 5 is the limit of (5 ) if |λ| < 1. If |λ| > 1, (5„) diverges
n
and 5 is often called its anti-limit. If |λ| = 1, (5 ) has no limit at all
n
1.2. What is an extrapolation method? 5
or it only takes a finite number of distinct values and 5 is, in this case,
their arithmetical mean.
The two above expressions give the explicit form of the sequences
belonging to the respective kernels of our transformations. For that
reason we shall call them the explicit forms of the kernel.
However the kernel can also be given in an implicit form that is by means
of a relation which holds among consecutive terms of the sequence. Thus,
for the first transformation, it is equivalent to write that, Vn
Sn+i — S = -(5 - 5)
n
while, for Aitken's process, we have Vn
Sn+1 — 5 = λ(5 - 5).
η
Such a difference equation (see Lakshmikantham and Trigiante [270]) is
called the implicit form of the kernel because it does not give directly
(that is explicitly) the form of the sequences belonging to the kernel
but only implicitly as the solution of this difference equation. Solving
this difference equation, which is obvious in our examples, leads to the
explicit form of the kernel. Of course, both forms are equivalent and
depend on parameters.
We are now ready to enter into the details and to explain what an
extrapolation method is.
1.2 What is an extrapolation method?
As we saw in the previous section, the implicit and explicit forms of
the kernel of a sequence transformation depend on several parameters
5, αι,..., α . The explicit form of the kernel explicitly gives the expres­
ρ
sion of the sequences of the kernel in terms of the unknown parameters
which can take (almost) arbitrary values. The implicit form of the kernel
consists in a relation among consecutive terms of the sequence, involving
the unknown parameters αι,..., a and 5, that is a relation of the form
p
Ä(5 ,..., 5 +, 5) = 0
n η ς
which must be satisfied Vn, if and only if (5 ) belongs to the kernel Κ,χ
n
of the transformation T.