Scientific Computing: Eigenvalues and Optimization

621 Pages·2018·6.781 MB·English

by  John A. Trangenstein

var sdd = ( maxev - minev ) * sqrtrnd - minev;
sdd = Math.max( sdd, 0 );
var mu2 = Math.min( maxadd, sdd );
A[0] += mu2;
A[3] += mu2;
mu += mu2;
19
John A. Trangenstein
Scientifi c
Computing
Vol. II – Eigenvalues
and Optimization
Editorial Board
T. J.Barth
M.Griebel
D.E.Keyes
R.M.Nieminen
D.Roose
T.Schlick
Texts in Computational
19
Science and Engineering
Editors
TimothyJ. Barth
MichaelGriebel
DavidE. Keyes
RistoM. Nieminen
DirkRoose
TamarSchlick
Moreinformationaboutthisseriesathttp://www.springer.com/series/5151
John A. Trangenstein
Scientific
Computing
Vol. II – Eigenvalues
and Optimization
123
JohnA.Trangenstein
ProfessorEmeritusofMathematics
DepartmentofMathematics
DukeUniversity
Durham
NorthCarolina,USA
Additionalmaterialtothisbookcanbedownloadedfromhttp://extras.springer.com.
ISSN1611-0994 ISSN2197-179X (electronic)
TextsinComputationalScienceandEngineering
ISBN978-3-319-69106-0 ISBN978-3-319-69107-7 (eBook)
https://doi.org/10.1007/978-3-319-69107-7
LibraryofCongressControlNumber:2018932366
MathematicsSubjectClassification(2010):15,65
©SpringerInternationalPublishingAG,partofSpringerNature2017
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
Printedonacid-freepaper
ThisSpringerimprintispublishedbytheregisteredcompanySpringerInternationalPublishingAGpart
ofSpringerNature.
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
To mydaughterPamela
Preface
Thisisthesecondvolumeinathree-volumebookaboutscientificcomputing.The
primary goal in these volumes is to present many of the important computational
topics and algorithms used in applications such as engineering and physics,
together with the theory needed to understand their proper operation. However, a
secondarygoalin thedesign ofthis bookisto allowreadersto experimentwith a
numberof interactiveprogramswithin the book,so thatreaderscan improvetheir
understandingoftheproblemsandalgorithms.Thisinteractivityisavailableinthe
HTMLformofthebook,throughJavaScriptprograms.
Theintendedaudienceforthisbookareupperlevelundergraduatestudentsand
beginninggraduatestudents.Duetotheself-containedandcomprehensivetreatment
of the topics, this book should also serve as a useful reference for practicing
numericalscientists. Instructors could use this book for multisemester courses on
numericalmethods.Theycouldalsouseindividualchaptersforspecializedcourses
suchasnumericallinearalgebra,constrainedoptimization,ornumericalsolutionof
ordinarydifferentialequations.
Inordertoreadallvolumesofthisbook,readersshouldhaveabasicunderstand-
ing of both linear algebra and multivariable calculus. However, for this volume it
willsufficetobefamiliarwithlinearalgebraandsinglevariablecalculus.Someof
thebasicideasforbothoftheseprerequisitesarereviewedinthistext,butatalevel
thatwouldbeveryhardtofollowwithoutpriorfamiliaritywiththosetopics.Some
experience with computer programming would also be helpful, but not essential.
Studentsshouldunderstandthepurposeofacomputerprogram,androughlyhowit
operatesoncomputermemorytogenerateoutput.
Manyofthe computerprogrammingexampleswilldescribetheuse ofa Linux
operating system. This is the only publicly available option in our mathematics
department, and it is freely available to all. Students who are using proprietary
operating systems, such as Microsoft and Apple systems, will need to replace
statementsspecifictoLinuxwiththecorrespondingstatementsthatareappropriate
totheirenvironment.
This book also references a large number of programs available in several
programming languages, such as C, CCC, Fortran and JavaScript, as well as
vii
viii Preface
MATLABmodules.Theseprogramsshouldprovideexamplesthatcantrainreaders
to developtheir own programs,fromexistingsoftware wheneverpossibleor from
scratchwhenevernecessary.
Chaptersbeginwithanoverviewoftopicsandgoals,followedbyrecommended
books and relevant software. Some chapters also contain a case study, in which
the techniques of the chapter are used to solve an important scientific computing
problemindepth.
Chapter1presentsnumericalmethodsforfindingeigenvaluesandeigenvectors.
These mathematical tools are important in such diverse problem areas as funda-
mentalmodesofvibrationinengineering,orgrowthanddecayratesineconomics.
Laterin thischapter,we developthesingularvaluedecompositionmorecarefully.
Attheendofthechapter,wealsoshowhowtouseeigenvaluesandeigenvectorsto
solvelinearrecurrences,ortocomputefunctionsofmatrices,suchasexponentials
orsquareroots.ThischapterdependsonthematerialinChap.6ofVolumeI.
Chapter 2 examines some important methods for solving large and typically
sparsesystemsoflinearequations.Thesenumericalmethodsareespeciallyuseful
for implementing implicit methods for partial differential equations (which are
discussed in two other books by this author [172, 173]). This chapter depends on
thematerialinChap.1.
Chapter 3 discusses the numerical solution of systems of nonlinear equations.
Here the mathematicalanalysisdependsstrongly on multivariablecalculus. There
aresomeusefulgeneralizationsofmethodsfromChap.5ofVolumeItothesolution
ofproblemsinthischapter.Therearealsosomeusefulgeneralizationsofiterative
methodsforlinearsystemsthatextendtononlinearsystems.Thechapterendswith
a case study involving the Lennard-Jones potential from chemistry. This chapter
dependsonthematerialinChap.5ofVolumeIandChap.2.
Chapter 4 examines constrained optimization methods. This subject is usually
left to books that are devoted solely to this topic. The chapter begins with
linear programmingproblems,advances to quadratic programmingproblems, and
ends with general nonlinear programmingproblems. This chapter depends on the
materialinChap.3andChap.6ofVolumeI.
Insummary,thisvolumecoversmathematicalandnumericalanalysis,algorithm
selection, and software development. The goal is to prepare readers to build
programsfor solving important problems in their chosen discipline. Furthermore,
theyshoulddevelopenoughmathematicalsophisticationtoknowthelimitationsof
the pieces of their algorithmand to recognizewhen numericalfeaturesare due to
programmingbugsratherthanthecorrectresponseoftheirproblem.
Iamindebtedtomanyteachersandcolleagueswhohaveshapedmyprofessional
experience.IthankJimDouglasJr.forintroducingmetonumericalanalysisasan
undergrad. (Indeed, I could also thank a class in category theory for motivating
me to look for an alternative field of mathematical study.) John Dennis, James
Bunch,andJorgeMoréallprovidedafirmfoundationformytraininginnumerical
analysis, while Todd Dupont, Jim Bramble, and Al Schatz gave me important
training in finite element analysis for my PhD thesis. But I did not really learn to
program until I met Bill Gragg, who also emphasized the importance of classical
Preface ix
analysis in the development of fundamental algorithms. I also learned from my
students,particularlyRandyLeVeque,whowasinthefirstnumericalanalysisclass
Ievertaught.Finally,IwanttothankBillAllardformanyconversationsaboutthe
deficienciesinnumericalanalysistexts.Ihopethatthisbookmovesthefielda bit
inthedirectionthatBillenvisions.
Mostofall,Iwanttothankmyfamilyfortheirloveandsupport.
Durham,NC,USA JohnA.Trangenstein
July7,2017
Contents
1 EigenvaluesandEigenvectors .............................................. 1
1.1 Overview............................................................... 2
1.2 FundamentalConcepts ................................................ 3
1.2.1 Examples..................................................... 3
1.2.2 CharacteristicPolynomial................................... 5
1.2.3 InversesandHermitians ..................................... 9
1.2.4 FundamentalTheoremofAlgebra .......................... 10
1.2.5 GerschgorinCircleTheorem................................ 12
1.2.6 TraceandDeterminant....................................... 14
1.2.7 InvariantSubspace........................................... 17
1.2.8 ChangeofBasis.............................................. 18
1.2.9 SimilarityTransformations.................................. 20
1.2.10 SpectralRadius............................................... 21
1.2.11 DistanceBetweenSubspaces................................ 22
1.3 HermitianMatrices.................................................... 38
1.3.1 Theory........................................................ 38
1.3.2 PerturbationAnalysis........................................ 44
1.3.3 SymmetricTridiagonalMatrices............................ 53
1.3.4 SturmSequences............................................. 55
1.3.5 Bisection...................................................... 65
1.3.6 PowerMethod................................................ 67
1.3.7 QLandQRAlgorithms...................................... 73
1.3.8 DivideandConquer.......................................... 80
1.3.9 dqds........................................................... 84
1.3.10 SolverComparison........................................... 89
1.3.11 HouseholderTridiagonalization............................. 90
1.3.12 LanczosProcess.............................................. 92
1.3.13 SymmetricEigenvalueProblems............................ 97
1.4 GeneralMatrices....................................................... 98
1.4.1 Theory........................................................ 98
1.4.2 PerturbationAnalysis........................................ 123
xi