Matrix, Numerical, and Optimization Methods in Science and Engineering

727 Pages·2021·16.102 MB·English

by  Kevin W. Cassel

Matrix,Numerical,andOptimizationMethodsinScienceandEngineering
Address vector and matrix methods necessary in numerical methods and optimiza-
tion of systems in science and engineering with this unified text. The book treats
the mathematical models that describe and predict the evolution of our processes
and systems, and the numerical methods required to obtain approximate solutions.
It explores the dynamical systems theory used to describe and characterize system
behavior, alongside the techniques used to optimize their performance. The book
integrates and unifies matrix and eigenfunction methods with their applications in
numericalandoptimizationmethods.Consolidating,generalizing,andunifyingthese
topics into a single coherent subject, this practical resource is suitable for advanced
undergraduate students and graduate students in engineering, physical sciences, and
appliedmathematics.
KevinW.Casselisprofessorofmechanicalandaerospaceengineeringandprofessorof
appliedmathematicsattheIllinoisInstituteofTechnology.Heisalsoafellowofthe
AmericanSocietyofMechanicalEngineersandanassociatefellowoftheAmerican
InstituteofAeronauticsandAstronautics.

Matrix, Numerical, and
Optimization Methods in
Science and Engineering
KEVIN W. CASSEL
IllinoisInstituteofTechnology
UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
314–321,3rdFloor,Plot3,SplendorForum,JasolaDistrictCentre,NewDelhi–110025,India
79AnsonRoad,#06–04/06,Singapore079906
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781108479097
DOI:10.1017/9781108782333
©KevinW.Cassel2021
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2021
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
LibraryofCongressCataloging-in-PublicationData
Names:Cassel,KevinW.,1966–author.
Title:Matrix,numerical,andoptimizationmethodsinscience
andengineering/KevinW.Cassel.
Description:Cambridge;NewYork,NY:CambridgeUniversityPress,2021.|
Includesbibliographicalreferencesandindex.
Identifiers:LCCN2020022768(print)|LCCN2020022769(ebook)|
ISBN9781108479097(hardback)|ISBN9781108782333(ebook)
Subjects:LCSH:Matrices.|Dynamics.|Numericalanalysis.|
Linearsystems.|Mathematicaloptimization.|Engineeringmathematics.
Classification:LCCQA188.C372021(print)|LCCQA188(ebook)|
DDC512.9/434–dc23
LCrecordavailableathttps://lccn.loc.gov/2020022768
LCebookrecordavailableathttps://lccn.loc.gov/2020022769
ISBN978-1-108-47909-7Hardback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracy
ofURLsforexternalorthird-partyinternetwebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
TheheavensdeclarethegloryofGod;
theskiesproclaimtheworkofhishands.
Dayafterdaytheypourforthspeech;
nightafternighttheydisplayknowledge.
Thereisnospeechorlanguage
wheretheirvoiceisnotheard.
Theirvoicegoesoutintoalltheearth,
theirwordstotheendsoftheworld.
(Psalm19:1–4)

Contents
Preface pagexi
PartI MatrixMethods 1
1 VectorandMatrixAlgebra 3
1.1 Introduction 5
1.2 Definitions 10
1.3 AlgebraicOperations 13
1.4 SystemsofLinearAlgebraicEquations–Preliminaries 15
1.5 SystemsofLinearAlgebraicEquations–SolutionMethods 26
1.6 VectorOperations 37
1.7 VectorSpaces,Bases,andOrthogonalization 42
1.8 LinearTransformations 51
1.9 NoteonNorms 55
1.10 BrieflyonBases 57
Exercises 58
2 AlgebraicEigenproblemsandTheirApplications 64
2.1 ApplicationsofEigenproblems 64
2.2 EigenvaluesandEigenvectors 69
2.3 RealSymmetricMatrices 78
2.4 NormalandOrthogonalMatrices 91
2.5 Diagonalization 95
2.6 SystemsofOrdinaryDifferentialEquations 100
2.7 SchurDecomposition 121
2.8 Singular-ValueDecomposition 123
2.9 PolarDecomposition 135
2.10 QRDecomposition 138
2.11 BrieflyonBases 141
2.12 Reader’sChoice 141
Exercises 142
viii Contents
3 DifferentialEigenproblemsandTheirApplications 155
3.1 FunctionSpaces,Bases,andOrthogonalization 157
3.2 EigenfunctionsofDifferentialOperators 162
3.3 AdjointandSelf-AdjointDifferentialOperators 179
3.4 PartialDifferentialEquations–SeparationofVariables 189
3.5 BrieflyonBases 209
Exercises 210
4 VectorandMatrixCalculus 218
4.1 VectorCalculus 219
4.2 Tensors 231
4.3 ExtremaofFunctionsandOptimizationPreview 237
4.4 SummaryofVectorandMatrixDerivatives 249
4.5 BrieflyonBases 250
Exercises 250
5 AnalysisofDiscreteDynamicalSystems 253
5.1 Introduction 255
5.2 Phase-PlaneAnalysis–LinearSystems 256
5.3 BifurcationandStabilityTheory–LinearSystems 260
5.4 Phase-PlaneandStabilityAnalysis–NonlinearSystems 273
5.5 PoincaréandBifurcationDiagrams–DuffingEquation 286
5.6 AttractorsandPeriodicOrbits–Saltzman–LorenzModel 299
PartII NumericalMethods 313
6 ComputationalLinearAlgebra 315
6.1 IntroductiontoNumericalMethods 317
6.2 ApproximationandItsEffects 326
6.3 SystemsofLinearAlgebraicEquations–DirectMethods 334
6.4 SystemsofLinearAlgebraicEquations–IterativeMethods 346
6.5 NumericalSolutionoftheAlgebraicEigenproblem 355
6.6 Epilogue 371
Exercises 372
7 NumericalMethodsforDifferentialEquations 376
7.1 GeneralConsiderations 377
7.2 FormalBasisforFinite-DifferenceMethods 384
7.3 FormalBasisforSpectralNumericalMethods 391
7.4 FormalBasisforFinite-ElementMethods 396
7.5 ClassificationofSecond-OrderPartialDifferentialEquations 398
Exercises 405
Contents ix
8 Finite-DifferenceMethodsforBoundary-ValueProblems 407
8.1 IllustrativeExamplefromHeatTransfer 407
8.2 GeneralSecond-OrderOrdinaryDifferentialEquation 412
8.3 PartialDifferentialEquations 415
8.4 DirectMethodsforLinearSystems 420
8.5 Iterative(Relaxation)Methods 426
8.6 BoundaryConditions 430
8.7 Alternating-Direction-Implicit(ADI)Method 434
8.8 MultigridMethods 437
8.9 CompactHigher-OrderMethods 444
8.10 TreatmentofNonlinearTerms 448
Exercises 453
9 Finite-DifferenceMethodsforInitial-ValueProblems 466
9.1 Introduction 466
9.2 Single-StepMethodsforOrdinaryDifferentialEquations 467
9.3 AdditionalMethodsforOrdinaryDifferentialEquations 481
9.4 PartialDifferentialEquations 483
9.5 ExplicitMethods 485
9.6 NumericalStabilityAnalysis 489
9.7 ImplicitMethods 496
9.8 BoundaryConditions–SpecialCases 501
9.9 TreatmentofNonlinearConvectionTerms 502
9.10 MultidimensionalProblems 508
9.11 HyperbolicPartialDifferentialEquations 515
9.12 CoupledSystemsofPartialDifferentialEquations 517
9.13 ParallelComputing 518
9.14 Epilogue 522
Exercises 522
PartIII LeastSquaresandOptimization 527
10 Least-SquaresMethods 529
10.1 IntroductiontoOptimization 529
10.2 Least-SquaresSolutionsofAlgebraicSystemsofEquations 531
10.3 Least-SquareswithConstraints 538
10.4 Least-SquareswithPenaltyFunctions 541
10.5 NonlinearObjectiveFunctions 542
10.6 Conjugate-GradientMethod 543
10.7 GeneralizedMinimumResidual(GMRES)Method 551
10.8 SummaryofKrylov-BasedMethods 555
Exercises 556