Table Of ContentRegularization of Ill-Posed Problems by Iteration Methods
Mathematics and Its Applications
Managing Editor:
M. HAZEWINKEL
Centre for Mathematics and Computer Science, Amsterdam, The Netherlands
Volume 499
Regularization of
Ill-Posed Problems
by Iteration Methods
by
S.F. Gilyazov
and
N.L. Gol'dman
Science Research Cumputer Celltel;
Moscow State Ulliversity.
Muscuw. Russia
Springer-Science+Business Media, B.Y.
A c.I.P. Catalogue record for this book is available from the Library of Congress.
ISBN 978-90-481-5382-4 ISBN 978-94-015-9482-0 (eBook)
DOI 10.1007/978-94-015-9482-0
Prill ted all acid-free paper
All Rights Reserved
© 2000 Springer Science+Business Media Dordrecht
Originally published by Kluwer Academic Publishers in 2000.
Softcover reprint of the hardcover 1st edition 2000
No part of the material protected by this copyright notice may be reproduced or
utilized in any form or by any means, electronic or mechanical,
including photocopying, recording or by any information storage and
retrieval system, without written permission from the copyright owner.
Table of contents
PREFACE vii
INTRODUCTION 1
ACKNOWLEDGMENTS ix
1 REGULARIZING ALGORITHMS FOR LINEAR
ILL-POSED PROBLEMS: UNIFIED APPROACH 7
1.1 Formulation of the problem and basic definitions ... 7
1.2 Approximation of normal pseudo-solutions for exact data. 13
1.3 Convergence of the approximate family for exact data . . . 21
1.4 Approximation of normal pseudo-solutions for perturbed data 27
1.5 Regularization of linear ill-posed problems with normally
solvable operators . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 33
2 ITERATION STEEPEST DESCENT METHODS FOR
LINEAR OPERATOR EQUATIONS 41
2.1 Basic properties of steepest descent methods ...... . 41
2.2 Convergence of steepest descent methods for exact data. 49
2.3 Asymptotic properties of steepest descent methods and
acceleration of convergence . . . . . . . . . . . . . . . . 60
2.4 Regularizing steepest descent methods . . . . . . . . . 73
2.5 Choice of the regularization parameter by the residual
criterion ......................... . . ..... 84
3 ITERATION CONJUGATE DIRECTION METHODS
FOR LINEAR OPERATOR EQUATIONS 97
3.1 Basic properties of conjugate direction methods ..... . 97
3.2 Convergence of conjugate direction methods for exact data . 107
3.3 Multiparameter conjugate direction methods ..... . .115
3.4 Regularizing conjugate direction methods . . . . . . . . .119
3.5 Choice of the regularization parameter by the residual
principle ......................... . . 127
3.6 The conditions of convergence of the regularizing conjugate
gradient method with the use of additional information .134
v
vi TABLE OF CONTENTS
4 ITERATION STEEPEST DESCENT METHODS FOR
NONLINEAR OPERATOR EQUATIONS 141
4.1 Nonlinear operator equations. . . . . . . . . . . . . . . . . . 141
4.2 Properties of nonlinear operator equations with perturbed data . 147
4.3 Properties of regularizing steepest descent methods . . . . . . . 153
4.4 Convergence of the regularizing steepest descent method . . . . 158
4.5 Convergence rate estimate of the steepest descent method for
nonlinear equations . . . . . . . . . . . . . . . . . . . . . . . . . 165
5 ITERATION METHODS FOR ILL-POSED
CONSTRAINED MINIMIZATION PROBLEMS 171
5.1 The conjugate gradient projection method for exact data . 171
5.2 The regularizing conjugate gradient projection method . . 178
5.3 Sufficient conditions of convergence for the full-sphere. . . 183
5.4 The conjugate gradient projection method for the affine set. . 192
6 DESCRIPTIVE REGULARIZATION ALGORITHMS
ON THE BASIS OF THE CONJUGATE GRADIENT
PROJECTION METHOD 201
6.1 Principles of construction of algorithms for solving ill-posed
problems with shape constraints on the solution . . . . . . . . 201
6.2 Descriptive regularization of the Fredholm integral equation of
the first kind ........................... . 210
6.3 Algorithms for the numerical solution of inverse problems for
parabolic equations . . . . . . . . . . . . . . . . . . . . . . . . . 219
6.4 Descriptive regularization of quasilinear free boundary problems . 248
6.5 Applications in nonlinear thermophysics ............. . 302
BIBLIOGRAPHY 325
INDEX 339
PREFACE
Iteration regularization, i.e., utilization of iteration methods of any form for the
stable approximate solution of ill-posed problems, is one of the most important but
still insufficiently developed topics of the new theory of ill-posed problems.
In this monograph, a general approach to the justification of iteration regulari
zation algorithms is developed, which allows us to consider linear and nonlinear
methods from unified positions. Regularization algorithms are the 'classical' iterative
methods (steepest descent methods, conjugate direction methods, gradient projection
methods, etc.) complemented by the stopping rule depending on level of errors in
input data. They are investigated for solving linear and nonlinear operator equations
in Hilbert spaces. Great attention is given to the choice of iteration index as
the regularization parameter and to estimates of errors of approximate solutions.
Stabilizing properties such as smoothness and shape constraints imposed on the
solution are used. On the basis of these investigations, we propose and establish
efficient regularization algorithms for stable numerical solution of a wide class of
ill-posed problems. In particular, descriptive regularization algorithms, utilizing
a priori information about the qualitative behavior of the sought solution and
ensuring a substantial saving in computational costs, are considered for model
and applied problems in nonlinear thermophysics. The results of calculations
for important applications in various technical fields (a continuous casting, the
treatment of materials and perfection of heat-protective systems using laser and
composite technologies) are given.
This book will be a useful resource for specialists in the theory of partial
differential and integral equations, in numerical analysis, theory and methods of
solving ill-posed problems.
ACKNOWLEDGMENTS
The authors are grateful to Profs. O.M. Alifanov, A.B. Bakushinskii, V.A. Morozov,
F.P. Vasil'ev, V.V. Vasin, A.G. Yagola, and also to the participants of science
research seminars in Moscow State University for useful discussions about this work.
Furthermore, the authors are indebted to Vl.V. Tchernyi for his help with preparing
this manuscript (Chapter 4 and Sections 5.1-5.3).
INTRODUCTION
A considerable number of problems arising in different scientific and technical fields
belong to a class of ill-posed problems. Their solutions need not exist, and even
if they exist they need not be unique and stable, i.e., continuously depending on
the input data. To obtain stable numerical solutions of problems for which the
Hadamard conditions of correctness [83, 84] are not satisfied, the regularization
methods must be applied.
The general principles of regularization for ill-posed problems are known. In
particular, such principles have been established by A.N. Tikhonov, V.K. Ivanov
and M.M. Lavrent'ev [97, 116, 172, 173, 174], see also [98,115,117]. The literature
on various regularization methods based on these general principles is extensive. We
refer to the publications [3, 8, 10, 22, 28, 79, 89, 118, 119, 127, 128, 175, 176, 185]
and to the references given there.
One of the most important, but still insufficiently developed, topics of the new
theory of ill-posed problems is connected with iteration regularization, i.e., with
utilization of iteration methods of any form for the stable approximate solution of
ill-posed problems.
The idea of using iterative schemes for solving ill-posed problems belongs to
M.M. Lavrent'ev [116]. The rapidly growing interest in investigation of regularizing
properties of iteration methods is confirmed by the monographs [4, 8, 9, 23, 28, 175,
180,181,183,185,187] published recently, see also the works [5, 6, 11, 13-17,24-27,
30-32,35-37,85-88,90, 101-103, 107, 113, 124, 138-145, 150, 154, 164-169, 178,
184, 186, 189, 190] and references therein.
This book is based on the publications [38-78, 129-135] and on new results of
the authors. It contains theoretical and applied studies of regularization algorithms
based on nonlinear processes, in particular on methods of the type of steepest descent
and conjugate gradient. Investigation of regularizing properties of such algorithms
is a nonlinear task even in the situation when the original ill-posed problem is linear.
The main results obtained in this book can be stated in the following way.
1. A unified approach to the description and substantiation of iteration regulari
zation algorithms for linear ill-posed problems in Hilbert spaces is developed.
The general scheme proposed makes it possible to obtain a unified proof
of convergence for various types of regularizing algorithms starting from the
Tikhonov regularization up to algorithms based on steepest descent and
conjugate gradient methods. Necessary and sufficient conditions of convergence
of approximate solutions for exact data are established. For perturbed data the
1
S. F. Gilyazov et al., Regularization of Ill-Posed Problems by Iteration Methods
© Springer Science+Business Media Dordrecht 2000
2 INTRODUCTION
conditions of coordination between the regularization parameter and a level of
errors in input data, sufficient for convergence to the normal pseudo-solution,
are given.
2. Asymptotic properties of the iterative steepest descent methods for solving
linear operator equations in Hilbert spaces are established. Previously this
question was investigated only for finite-dimensional spaces. In particular, for
linear operator equations in Hilbert spaces, we prove boundedness (uniform
with respect to the iteration index) of numerical parameters defining the descent
step in each iteration. On the basis of asymptotic properties, the ways of
acceleration of convergence are proposed and justified. A unified proof of
convergence for the family of regularizing methods of steepest descent type
is achieved. Moreover, we prove that these methods are optimal under the
order on a class of the sourcewise represented solutions. The residual principle
for the choice of regularization parameter ensuring some stopping criterion for
the iterative process is established.
3. The family of iteration conjugate direction methods for linear operator equa
tions in Hilbert spaces is investigated. A unified proof of convergence for exact
data is given. We construct multiparameter variant of the conjugate direction
method of integrated gradients, taking into account the structure of the space
and ensuring increase of the rate of convergence. For operator equations given
approximately (with the perturbed operator and right-hand side), we prove
sufficient conditions for coordination between the iteration index and the level of
errors in input data ensuring convergence of the regularizing conjugate direction
algorithms. Convergence of the regularizing algorithm on the basis of the
conjugate direction method for the self-adjoint indefinite operator is justified.
For this family of iteration methods, optimality under the order on a class of
the sourcewise represented solutions and the residual principle for the choice of
the regularization parameter are also established.
4. Iteration steepest descent methods for nonlinear operator equations in Hilbert
spaces are investigated under the assumption of Frechet differentiability of
the operator in the neighborhood of the exact solution of the nonlinear
equation. The regularizing properties of these iteration methods (convergence
and optimality under the order on a class of the sourcewise represented
solutions) are proved. The choice of the regularization parameter on the basis
of some a posteriori principle is proposed and established.
5. Iteration regularizing methods for constrained minimization problems are
investigated. We prove weak convergence of the regularized solutions obtained
by the conjugate gradient projection algorithm for minimization on the convex
closed set. The corresponding result for strong convergence is achieved with
the use of some a priori information about the exact solution. Regularizing
conjugate gradient projection algorithms for minimization on the full-sphere
and the affine set are considered in more detail. In particular, for the affine set,
