Iterative methods for ill-posed problems : an introduction

150 Pages·2011·1.248 MB·English

by  A B BakushinskiiМ†, M Iпё UпёЎ Kokurin, A B Smirnova

Inverse and Ill-Posed Problems Series 54
Managing Editor
Sergey I. Kabanikhin, Novosibirsk, Russia/Almaty, Kazakhstan

Anatoly B. Bakushinsky
Mihail Yu. Kokurin
Alexandra Smirnova
Iterative Methods for
Ill-Posed Problems
An Introduction
De Gruyter
MathematicsSubjectClassification2010:Primary:47A52;Secondary:65J20.
ISBN 978-3-11-025064-0
e-ISBN 978-3-11-025065-7
ISSN 1381-4524
LibraryofCongressCataloging-in-PublicationData
Bakushinskii,A.B.(AnatoliiBorisovich).
[Iterativnyemetodyresheniianekorrektnykhzadach.English]
Iterative methods for ill-posed problems : an introduction / by
AnatolyBakushinsky,MikhailKokurin,AlexandraSmirnova.
p.cm.(cid:2)(Inverseandill-posedproblemsseries;54)
Includesbibliographicalreferencesandindex.
ISBN978-3-11-025064-0(alk.paper)
1. Differential equations, Partial (cid:2) Improperly posed problems.
2.Iterativemethods(Mathematics) I.Kokurin,M.IU.(Mikhail
IUr’evich) II.Smirnova,A.B.(AleksandraBorisovna) III.Title.
QA377.B25513 2011
5151.353(cid:2)dc22
2010038154
BibliographicinformationpublishedbytheDeutscheNationalbibliothek
The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie;
detailedbibliographicdataareavailableintheInternetathttp://dnb.d-nb.de.
”2011WalterdeGruyterGmbH&Co.KG,Berlin/NewYork
Typesetting:Da-TeXGerdBlumenstein,Leipzig,www.da-tex.de
Printingandbinding:Hubert&Co.GmbH&Co.KG,Göttingen
(cid:2)(cid:2)Printedonacid-freepaper
PrintedinGermany
www.degruyter.com
Preface
A variety of processes in science and engineering is commonly modeled by alge-
braic, differential, integral and other equations. In a more difficult case, it can be
systemsofequationscombinedwiththeassociatedinitialandboundaryconditions.
Frequently,thestudyofappliedoptimizationproblemsisalsoreducedtosolvingthe
corresponding equations. Typical examples include Euler’s equation in calculus of
variationsandboundaryvalueproblemsforPontrjagin’smaximalprincipleincon-
trol theory. All such equations, encountered both in theoretical and applied areas,
may naturallybe classifiedasoperator equations. These equations connect the un-
known parameters of the model with some given quantities describing the model.
Theabovequantities,whichcanbeeithermeasuredorcalculatedatthepreliminary
stage,formtheso-calledinputdata.Generally,theinputdataaswellastheunknown
parametersaretheelementsofcertainmetricspaces,inparticular,BanachorHilbert
spaces, with the operator of the model acting from the solution space to the data
space.Thecurrenttextbookwillfocusoniterativemethodsforoperatorequationsin
Hilbertspaces.
Iterativemethodsintheirsimplestformarefirstintroducedinanundergraduate
numericalanalysiscourse,amongwhichNewton’smethodforapproximatingaroot
ofadifferentiablefunctioninonevariableisprobablythebestknown.Thisisatyp-
icaliterativeprocesswidelyusedinapplications.Itcanbegeneralizedtothecaseof
finitesystemsofnonlinearequationswithafinitenumberofunknowns,andalsoto
thecaseofoperatorequationsininfinitedimensionalspaces.Itshould,however,be
notedthatdirectgeneralizationofthiskindisonlypossibleforregularoperatorequa-
tionsandsystemsofequations.Theregularityconditiongeneralizestherequirement
onthederivativetobedifferentfromzeroinaneighborhoodoftheroot.Thisrequire-
mentisusedfortheconvergenceanalysisofNewton’sschemeinaone-dimensional
case.Withouttheregularitycondition,Newton’siterationsarenotnecessarilywell-
defined.Thelackofregularityisamajorobstaclewhenitcomestoapplicabilityof
notonlytheNewtonmethod,butallclassicaliterativemethods,gradient-typemeth-
odsforexample,althoughoftenthesemethodsareformallyexecutableforirregular
problems as well. Still, a lot of important mathematical models give rise to either
irregular operator equations or to operator equations whose regularity is extremely
vi Preface
difficult to investigate, for instance numerous nonlinear inverse problems in PDEs.
Thus,thequestioniswhetherornotitispossibletoconstructiterativemethodsfor
nonlinearoperatorequationswithouttheregularitycondition.
In the last few years the authors have been developing a unified approach to
theconstructionofsuchmethodsforirregularequations.Theapproachunderdevel-
opment is closely related to modern theory of ill-posed problems. The goal of our
textbookistogiveabriefaccountofthisapproach.Thereare16chapters(lectures)
in the manuscript, which is based on the lecture notes prepared by the authors for
graduate students at Moscow Institute of Physics and Technology and Mari State
University,Russia,andGeorgiaStateUniversity,USA.Asetofexercisesappearsat
the end of each chapter. These range from routine tests of comprehension to more
challenging problems helping to get a working understanding of the material. The
bookdoesnotrequireanypriorknowledgeofclassicaliterativemethodsfornonlin-
ear operator equations. The first three chapters investigate the basic iterative meth-
ods,theNewton,theGauss–Newtonandthegradientones,ingreatdetail.Theyalso
give an overview of some relevant functional analysis and infinite dimensional op-
timization theory. Further chapters gradually take the reader to the area of iterative
methods for irregular operator equations. The last three chapters contain a number
ofrealisticnonlineartestproblemsreducedtofinitesystemsofnonlinearequations
withafinitenumberofunknowns,integralequationsofthefirstkind,andparameter
identificationproblemsinPDEs.Thetestproblemsarespeciallyselectedinorderto
emphasize numerical implementation of various iteratively regularized procedures
addressed in this book, and to enable the reader to conduct his/her own computa-
tionalexperiments.
Asitfollowsfromthetitle,thistextbookismeanttoilluminateonlytheprimary
approachestotheconstructionandinvestigationofiterativemethodsforsolvingill-
posed operator equations. These methods are being constantly perfected and aug-
mentedwithnewalgorithms.Appliedinverseproblemsarethemainsourcesofthis
development:tosolvethem,thesuccessfulimplementationofwell-knowntheoreti-
calproceduresisoftenimpossiblewithoutadeepanalysisofthenatureofaproblem
andasuccessfulresolutionofthedifficultiesrelatedtothechoiceofcontrolparame-
ters,whichsometimesnecessitatesmodificationoftheoriginaliterativeschemes.At
times,byanalyzingthestructureofparticularappliedproblems,researchersdevelop
new procedures (iterative algorithms, for instance), aimed at these problems exclu-
sively.Thenew‘problem-oriented’proceduresmayturnouttobemoreeffectivethan
thosedesignedforgeneraloperatorequations.Examplesofsuchproceduresinclude,
but are not limited to, the method of quasi-reversibility (Lattes and Lions, 1967)
for solving unstable initial value problems (IVPs) for the diffusion equation with
reversed time, iteratively regularized schemes for solving unstable boundary value
problems (BVPs), which reduce the original BVP to a sequence of auxiliary BVPs
for the same differential equation with ‘regularized’ boundary conditions (Kozlov
and Mazya, 1990), and various procedures for solving inverse scattering problems.
For applied problems of shape design and shape recovery, the level set method is
widelyused(OsherandSethian,1988).Thereadermayconsult[59,27,63,69,40]
foradetailedtheoreticalandnumericalanalysisofthesealgorithms.
Preface vii
The formulas within the text are doubly numbered, with the first number being
thenumberofthechapterandthesecondnumberbeingthenumberoftheformula
within the chapter. The problems are doubly numbered as well. A few references
aregiventotheextensivebibliographyattheendofthebook;theyareindicatedby
initials in square brackets. Standard notations are used throughout the book; R is
the set of real numbers, N is the set of natural numbers. All other notations are
introducedastheyappear.
Theauthorshopethatthetextbookwillbeusefultograduatestudentspursuing
their degrees in computational and applied mathematics, as well as to researchers
andengineerswhomayencounternumericalmethodsfornonlinearmodelsintheir
work.
AnatolyBakushinsky
MikhailKokurin
AlexandraSmirnova

Contents
Preface............................................................ v
1 Theregularitycondition.Newton’smethod ....................... 1
1.1 Preliminaryresults ......................................... 1
1.2 Linearizationprocedure ..................................... 2
1.3 Erroranalysis.............................................. 4
Problems ...................................................... 6
2 TheGauss–Newtonmethod ..................................... 10
2.1 Motivation ................................................ 10
2.2 Convergencerates .......................................... 12
Problems ...................................................... 14
3 Thegradientmethod ........................................... 16
3.1 Thegradientmethodforregularproblems ...................... 16
3.2 Ill-posedcase.............................................. 18
Problems ...................................................... 20
4 Tikhonov’sscheme ............................................. 23
4.1 TheTikhonovfunctional .................................... 23
4.2 Propertiesofaminimizingsequence........................... 24
4.3 Othertypesofconvergence .................................. 27
4.4 Equationswithnoisydata.................................... 29
Problems ...................................................... 30
5 Tikhonov’sschemeforlinearequations........................... 32
5.1 Themainconvergenceresult ................................. 32
5.2 Elementsofspectraltheory .................................. 34
5.3 Minimizingsequencesforlinearequations ..................... 35
5.4 Aprioriagreementbetweentheregularizationparameterandthe
errorforequationswithperturbedright-handsides............... 37