Krylov Subspace Methods for Linear Systems: Principles of Algorithms

233 Pages·2023·2.989 MB·English

by  Tomohiro Sogabe

Springer Series in Computational Mathematics 60
Tomohiro Sogabe
Krylov Subspace
Methods
for Linear
Systems
Principles of Algorithms
Springer Series in Computational Mathematics
Volume 60
SeriesEditors
RandolphE.Bank,DepartmentofMathematics,UniversityofCalifornia
SanDiego,LaJolla,CA,USA
WolfgangHackbusch,Max-Planck-InstitutfürMathematikinden
Naturwissenschaften,Leipzig,Germany
JosefStoer,InstitutfürMathematik,UniversityofWürzburg,Würzburg,Germany
HarryYserentant,InstitutfürMathematik,TechnischeUniversitätBerlin,Berlin,
Germany
This is basically a numerical analysis series in which high-level monographs are
published.
Wedevelopthisseriesaimingathavingmorepublicationsinitwhicharecloser
to applications. There are several volumes in the series which are linked to some
mathematicalsoftware.
Thisisalistofalltitlespublishedinthisseries:https://www.springer.com/series/
797?detailsPage=titles
Tomohiro Sogabe
Krylov Subspace Methods
for Linear Systems
Principles of Algorithms
TomohiroSogabe
DepartmentofAppliedPhysics
NagoyaUniversity
Nagoya,Japan
ISSN 0179-3632 ISSN 2198-3712 (electronic)
SpringerSeriesinComputationalMathematics
ISBN 978-981-19-8531-7 ISBN 978-981-19-8532-4 (eBook)
https://doi.org/10.1007/978-981-19-8532-4
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNature
SingaporePteLtd.2022
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse
ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Preface
In many fields of scientific computing and data science, we frequently face the
problem of solving large and sparse linear systems of the form Ax = b, which is
one of the most time-consuming parts of all computations. From this fact, many
researchers have devoted themselves to developing efficient numerical algorithms
forsolvingthelinearsystems,andKrylovsubspacemethodsarenowadayspopular
numericalalgorithmsandareknownasoneofthetoptenalgorithmsofthetwentieth
century,othersincludingfastFouriertransformandQuickSort[39].Thoughthebasic
theorywasestablishedinthetwentiethcentury,Krylovsubspacemethodshavebeen
developedbymathematicians,engineers,physicists,andmanyothers.
TherearemanyexcellentbooksonKrylovsubspacemethods,includingthoseby:
•
OweAxelsson,1994[10],
•
RichardBarrettetal.,1994[18],
•
WolfgangHackbusch,1994[92],
•
AreMagnusBruaset,1995[29],
•
RüdigerWeiss,1996[203],
•
AnneGreenbaum,1997[81],
•
YousefSaad,2003[151],
•
HenkA.vanderVorst,2003[196],
•
JörgLiesenandZdeneˇkStrakoš,2012[122],
•
GérardMeurantandJurjenDuintjerTebbens,2020[129].
In [129], detailed historical notes of Krylov subspace methods are described,
whichformamasterpiece(around700pages)ofKrylovsubspacemethodsfornon-
Hermitianlinearsystems.Thefeaturesofthisbookarelistedasfollows:
(1) Many applications of linear systems from computational science and data
science;
(2) Krylov subspace methods for complex symmetric linear systems such as the
COCGmethodandtheCOCRmethod;
(3) Krylovsubspacemethodsfornon-HermitianlinearsystemssuchastheBiCR
method,theGPBiCGmethod,andthe(block)IDR(s)method;
v
vi Preface
(4) KrylovsubspacemethodsforshiftedlinearsystemssuchastheshiftedIDR(s)
method;
(5) Matrixfunctionsasapplicationsofshiftedlinearsystems.
Feature (1) corresponds to Chap. 2, and linear systems are derived from
various applications: partial differential equations (finite difference discretization
methodsandthefiniteelementmethod);computationalphysics:condensedmatter
physics (computation of Green’s function) and lattice quantum chromodynamics
(Wilsonfermionmatrix);machine learning(least-squaresproblems);matrixequa-
tions(Sylvester-typematrixequations);optimization(HessianmatrixoverEuclidean
spaceandRiemannianmanifoldusingtensorcomputationnotations).
Features (2) and (3) correspond to Chap. 3. In this chapter, Krylov subspace
methods are classified into three groups: Hermitian linear systems, complex
symmetriclinearsystems,andnon-Hermitianlinearsystems.ForHermitianlinear
systems,theCGmethodisderivedfromthematrixformoftheLanczosprocess,the
CRmethodisderivedfromtheCGmethod,andtheMINRESmethodisderivedfrom
theLanczosprocess.Forcomplexsymmetriclinearsystems,theCOCGmethod,the
COCRmethod,andtheQMR_SYMmethodaredescribedasextensionsoftheCG
method,theCRmethod,andtheMINRESmethod,respectively.Fornon-Hermitian
linear systems, the BiCG method, the BiCR method, and the QMR method are
describedasextensionsoftheCGmethod,theCRmethod,andtheMINRESmethod,
respectively.ThedetailedderivationsoftheGPBiCGmethodandthe(block)IDR(s)
method,oneofthefeaturesofthisbook,aredescribedinthischapter.Inaddition,
somepreconditioningtechniquesarebrieflydescribed.
Feature(4)correspondstoChap.4.Inthischapter,Krylovsubspacemethodsfor
shiftedlinearsystemsareclassifiedintothreegroups:Hermitian,complexsymmetric,
andnon-Hermitianlinearsystems.Thedetailedderivationsofthesealgorithmsare
describedsystematically.
Feature (5) corresponds to Chap. 5. If one needs a large matrix function, then
KrylovsubspacemethodsandKrylovsubspacemethodsforshiftedlinearsystems
aremethodsofchoicesincethesealgorithmscanproduceanyelementofthematrix
function.Thedefinitionsofmatrixfunctionsandwell-knownalgorithmsformatrix
functionsarealsodescribed.
Anadditionalfeatureofthisbookisthattherearenonumericalexperimentsexcept
sometypicalnumericalexamplesforfurtherunderstandingtheconvergencebehavior
ofKrylovsubspacemethods.TheconvergenceofKrylovsubspacemethodsdepends
highly on the coefficient matrix, and the best algorithm changes if the coefficient
matrix changes. So, if the reader wants to solve linear systems, I recommend the
readertoapplyseveralKrylovsubspacemethods(includingtheBiCGSTABmethod
andtheGMRESmethod)totheirproblemandchoosethebestoneamongthem.
This book is suitable for anyone who studied linear algebra and needs to solve
largeandsparselinearsystems.Ihopethisbookishelpfulforthereadertounderstand
Preface vii
theprinciplesandpropertiesofKrylovsubspacemethodsandtocorrectlyuseKrylov
subspacemethodstosolvetheirproblems.
Nagoya,Japan TomohiroSogabe
September2022
Acknowledgements
IwouldliketothankProf.Dr.LeiDuatDalianUniversityofTechnologyforkindly
sending me the LaTeX code of his block IDR(s) method. I appreciate Masayuki
Nakamura’s work. I could raise the priority of writing this book from his timely
reminders,togetherwithhiskindseasonalgreetings.IthankLokeshwaranManick-
avasagam and Poojitha Ravichandran for their help with the production of this
book. I am grateful to three anonymous reviewers for their careful reading of the
manuscript and highly constructive comments that enhanced the quality of the
manuscript. I would like to express my sincere thanks to three editors from the
Springer Series in Computational Mathematics (SSCM) editorial board: Prof. Dr.
RandolphE.Bank,Prof.Dr.WolfgangHackbusch,Prof.Dr.JosefStoer,andProf.
Dr. Harry Yserentant for providing the author with fruitful comments from broad
anddistinguishedprofessionalviewpoints.
Finally,IappreciateProf.Dr.Shao-LiangZhangwhointroducedmetothefield
ofKrylovsubspacemethodswhenIwasaPh.D.studentattheUniversityofTokyo
about20yearsago,andIwishtothankmylovingwife,YuanLi,forherday-to-day
supportandherkindness.Sherecommendedmetowritethisbookonweekends,and
shewastakingcareoftwoactivesons:KazumaandTakuma,andheroldfather.
ix
Contents
1 IntroductiontoNumericalMethodsforSolvingLinearSystems .... 1
1.1 LinearSystems ............................................. 1
1.1.1 VectorNorm ....................................... 2
1.1.2 MatrixNorm ....................................... 2
1.2 ConditionNumber .......................................... 4
1.3 DirectMethods ............................................ 4
1.3.1 LUDecomposition .................................. 5
1.3.2 LUDecompositionwithPivoting ...................... 6
1.3.3 IterativeRefinement ................................. 8
1.4 DirectMethodsforSymmetricLinearSystems ................. 9
1.4.1 CholeskyDecomposition ............................. 9
1.4.2 LDL(cid:2)Decomposition ............................... 10
1.5 DirectMethodsforLargeandSparseLinearSystems ............ 12
1.6 StationaryIterativeMethods ................................. 13
1.6.1 TheJacobiMethod .................................. 14
1.6.2 TheGauss–SeidelMethod ........................... 15
1.6.3 TheSORMethod ................................... 15
1.6.4 ConvergenceoftheStationaryIterativeMethods ........ 16
1.7 MultigridMethods .......................................... 18
1.8 KrylovSubspaceMethods ................................... 19
1.9 OrthogonalizationMethodsforKrylovSubspaces ............... 23
1.9.1 TheArnoldiProcess ................................. 23
1.9.2 TheBi-LanczosProcess ............................. 26
1.9.3 TheComplexSymmetricLanczosProcess .............. 27
1.9.4 TheLanczosProcess ................................ 28
2 SomeApplicationstoComputationalScienceandDataScience ..... 31
2.1 PartialDifferentialEquations ................................. 31
2.1.1 FiniteDifferenceMethods ............................ 31
2.1.2 TheFiniteElementMethod .......................... 42
2.1.3 WeakForm ........................................ 43
xi