Matrix Algorithms in MATLAB

468 Pages·2016·6.13 MB·English

by  Ong U. Routh

Matrix Algorithms in
®
MATLAB

Matrix Algorithms in
®
MATLAB
Ong U. Routh
AMSTERDAM • BOSTON • HEIDELBERG • LONDON
NEWYORK • OXFORD • PARIS • SANDIEGO
SANFRANCISCO • SINGAPORE • SYDNEY • TOKYO
AcademicPressisanImprintofElsevier
Publisher:JoeHayton
SeniorEditorialProjectManager:KattieWashington
ProjectManager:AnushaSambamoorthy
Designer:MariaInesCruz
AcademicPressisanimprintofElsevier
125,LondonWall,EC2Y,5AS
525BStreet,Suite1800,SanDiego,CA92101-4495,USA
50HampshireStreet,5thFloor,CambridgeMA02139,USA
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UK
Copyright©2016ElsevierInc.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicormechanical,including
photocopying,recording,oranyinformationstorageandretrievalsystem,withoutpermissioninwritingfromthepublisher.
Detailsonhowtoseekpermission,furtherinformationaboutthePublisher’spermissionspoliciesandourarrangementswith
organizationssuchastheCopyrightClearanceCenterandtheCopyrightLicensingAgency,canbefoundatourwebsite:
www.elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(otherthanasmaybe
notedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroadenourunderstanding,
changesinresearchmethodsorprofessionalpractices,maybecomenecessary.Practitionersandresearchersmustalwaysrelyon
theirownexperienceandknowledgeinevaluatingandusinganyinformationormethodsdescribedherein.Inusingsuch
informationormethodstheyshouldbemindfuloftheirownsafetyandthesafetyofothers,includingpartiesforwhomtheyhave
aprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeanyliabilityforanyinjury
and/ordamagetopersonsorpropertyasamatterofproductsliability,negligenceorotherwise,orfromanyuseoroperationof
anymethods,products,instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloging-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary.
ISBN:978-0-12-803804-8
ForinformationonallAcademicPresspublications
visitourwebsiteatwww.elsevier.com
ThisbookisdedicatedtomywifeKathy,andtwochildrenJeffreyandJennifer

Contents
List of Figures....................................................................................................xiii
Preface..............................................................................................................xv
License Terms....................................................................................................xix
CHAPTER 1 Introduction 1
Introduction....................................................................................1
1.1 Elements of Linear Algebra................................................................1
1.1.1 Definitions...........................................................................2
1.1.2 Linear Independence and Related Concepts................................5
1.1.3 Solution of Linear Equations....................................................7
1.1.4 Solution of Eigenvalue Problem................................................9
1.2 A Brief Introduction of MATLAB........................................................15
1.3 Types of Matrices...........................................................................21
1.3.1 Square vs. Non-Square Matrices.............................................22
1.3.2 Symmetric vs. Non-Symmetric Matrices...................................22
1.3.3 Full Rank vs. Deficient Rank Matrices .....................................22
1.3.4 Singular vs. Non-Singular Matrices.........................................23
1.3.5 Orthogonal vs. Non-Orthogonal Matrices...................................24
1.3.6 Defective vs. Non-Defective Matrices.......................................25
1.3.7 Positive (semi-)Definite vs. Positive Indefinite Matrices...............28
1.3.8 Zero Structured vs. Full Matrices............................................28
1.4 Overview of Matrix Computations ......................................................36
1.5 Reordering of Sparse Matrices..........................................................46
1.6 Utility Codes.................................................................................50
CHAPTER 2 Direct Algorithms of Decompositions of Matrices by Non-Orthogonal
Transformations 63
Introduction .................................................................................63
2.1 Gauss Elimination Matrix.................................................................64
2.2 LU Decomposition.........................................................................67
2.2.1 LU Decomposition by Gauss Elimination..................................67
2.2.2 LU Decomposition by Crout Procedure.....................................70
2.2.3 Driver of LU Decomposition...................................................73
2.2.4 LU Decomposition of an Upper Hessenberg Matrix.....................77
2.2.5 LU Decomposition of a Band Matrix........................................79
2.3 LDU Decomposition.......................................................................80
2.3.1 LDU Decomposition by Gauss Elimination................................81
2.3.2 LDU Decomposition by Crout Procedure...................................83
2.3.3 Driver of LDU Decomposition.................................................85
2.4 Congruent Decomposition Algorithms for Symmetric Matrices .................87
2.4.1 Reduction of Symmetric Matrix to Diagonal (LDLt).....................88
2.4.2 Cholesky Decomposition (LLt)................................................92
2.4.3 Reduction of Symmetric Matrix to Tri-Diagonal (LTLt)..................95
vii
viii Contents
2.4.4 Reduction of Symmetric Matrix to Block Diagonal (LBLt)........... 101
2.4.5 Modified Cholesky Decomposition (xLLt)................................ 105
2.5 Similarity Decomposition Algorithms................................................ 110
2.5.1 Reduction of Square Matrix to Hessenberg
by Gauss Elimination......................................................... 110
2.5.2 Reduction of Square Matrix to Tri-Diagonal by Gauss Elimination 114
2.5.3 Reduction of Square Matrix to Tri-Diagonal by Lanczos Procedure 118
2.6 Reduction of a Symmetric Matrix to Tri-Diagonal and Another Symmetric
Matrix to Diagonal of ±1s and 0s..................................................... 122
2.6.1 Hyper Rotation and Hyper Reflection..................................... 123
2.6.2 GTJGt Decomposition by Hyperbolic Rotation or Hyperbolic
Reflection........................................................................ 129
CHAPTER 3 Direct Algorithms of Decompositions of Matrices by Orthogonal
Transformations 137
Introduction................................................................................ 137
3.1 Householder Reflection Matrix and Givens Rotation Matrix ................... 138
3.2 QR Decomposition....................................................................... 140
3.2.1 QR Decomposition by Householder Reflections........................ 141
3.2.2 QR Decomposition by Givens Rotations.................................. 142
3.2.3 QR Decomposition by Gram–Schmidt Orthogonalizations........... 143
3.2.4 Driver of QR Decomposition................................................. 145
3.2.5 QR Decomposition of an Upper Hessenberg Matrix................... 150
3.2.6 QR Decomposition of a Band Matrix...................................... 152
3.3 Complete Orthogonal Decomposition (QLZ) ....................................... 153
3.4 Reduction of Matrix to Bi-Diagonal.................................................. 155
3.4.1 QBZ Decomposition by Householder Reflections...................... 155
3.4.2 QBZ Decomposition by Givens Rotations................................ 159
3.4.3 QBZ Decomposition by Golub–Kahan–Lanczos Procedure .......... 162
3.5 Reduction of Square Matrix to Hessenberg by Similarity Transformations. 166
3.5.1 QHQt Decomposition by Householder Reflections..................... 167
3.5.2 QHQt Decomposition by Givens Rotations............................... 169
3.5.3 QHQt Decomposition by Arnoldi Procedure............................. 171
3.6 Reduction of Symmetric Matrix to Tri-Diagonal by Congruent
Transformations........................................................................... 174
3.6.1 QTQt Decomposition by Householder Reflections..................... 174
3.6.2 QTQt Decomposition by Givens Rotations............................... 176
3.6.3 QTQt Decomposition by Lanczos Procedure ............................ 177
3.7 Reduction of a Matrix to Upper Hessenberg and Another Matrix to Upper
Triangular .................................................................................. 181
3.7.1 QHRZ Decomposition by Orthogonal Transformations................ 181
3.7.2 QHRZ Decomposition by Arnoldi Procedure ............................ 184
Contents ix
CHAPTER 4 Direct Algorithms of Solution of Linear Equations 187
Introduction ............................................................................... 187
4.1 Brief Introduction of Pseudo-Inverse................................................ 188
4.2 Linear Constraints to Linear Equations............................................. 191
4.2.1 When A is a Rectangular Matrix........................................... 192
4.2.2 When A is a Square Matrix.................................................. 192
4.2.3 When Bx=b Has No Solution.............................................. 198
4.3 Solution of Five Elementary Linear Equations.................................... 199
4.3.1 Linear Equations of a Zero Matrix: Ox=b.............................. 200
4.3.2 Linear Equations of a Diagonal Matrix: Dx=b......................... 200
4.3.3 Linear Equations of an Orthogonal Matrix: Qx=b.................... 201
4.3.4 Linear Equations of a Lower Triangular Matrix: Lx=b............... 202
4.3.5 Linear Equations of an Upper Triangular Matrix: Ux=b............. 204
4.4 Gauss and Gauss-Jordan Elimination Algorithms ................................ 207
4.5 Householder and Givens Elimination Algorithms................................. 216
4.6 Solution Algorithms Based on Matrix Decompositions.......................... 219
4.7 Linear Systems Arising From Interpolation........................................ 226
4.7.1 Introducing Function Interpolation........................................ 226
4.7.2 Polynomial Interpolation..................................................... 228
4.7.3 Trigonometric Interpolation.................................................. 234
CHAPTER 5 Iterative Algorithms of Solution of Linear Equations 245
Introduction ............................................................................... 245
5.1 Overview of Iterative Algorithms...................................................... 246
5.2 Stationary Iterations: Jacobi, Gauss-Seidel, and More.......................... 247
5.3 General Methodology of Non-Stationary Iterations............................... 256
5.4 Non-Stationary Iterations Applied to Symmetric Matrix........................ 259
5.4.1 CG:ConjugateGradientVisFullOrthogonalBasesofK(A,y ,m)
0
and W=V ...................................................................... 259
5.4.2 CR:ConjugateResidualVisFullOrthogonalBasesofK(A,y ,m)
0
and W=AV..................................................................... 261
5.4.3 ‘C’E: ‘Conjugate’ Error W is Full Orthogonal Bases of K(A,y ,m)
0
and V=AW..................................................................... 264
5.4.4 Numerical Examples for CG,CR, and CE................................. 266
5.5 Non-Stationary Iterations Applied to Unsymmetric Matrix..................... 270
5.5.1 FOM: Full Orthogonalization Method V is Full Orthogonal Bases
of K(A,y ,m) and W=V.................................................... 270
0
5.5.2 GMRES: Generalized Minimum Residual V is Full Orthogonal
Bases of K(A,y ,m) and W=AV ......................................... 272
0
5.5.3 GMERR:GeneralizedMinimumErrorWisFullOrthogonalBases
of K(A′,y ,m) and V=A′W................................................ 274
0
5.5.4 DIOM: Direct Incomplete Orthogonalization Method V is Incom-
plete Orthogonal Bases of K(A,y ,m) and W=? ..................... 275
0
x Contents
5.5.5 DQGMRES: Direct Quasi-Generalized Minimum Residual V is
Incomplete Orthogonal Bases of K(A,y ,m) and W=?............. 278
0
5.5.6 DQGMERR:DirectQuasi-GeneralizedMinimumErrorWisIncom-
plete Orthogonal Bases of K(A′,y ,m) and V=? ..................... 280
0
5.5.7 BCG:Bi-ConjugateGradientVandWareBi-OrthogonalBasesof
K(A,y ,m) and K(A′,y ,m)................................................. 282
0 0
5.5.8 QMR: Quasi-Minimal Residual V is Bi-Orthogonal Bases of
K(A,y ,m) to K(A′,y ,m) and W=?..................................... 284
0 0
5.5.9 QME: Quasi-Minimal Error W is Bi-Orthogonal Bases of
K(A,y ,m) to K(A′,y ,m) and V=?..................................... 286
0 0
5.5.10 TFBCG:TransposedFreeBi-ConjugateGradientV=?andW=?
to Make it Equivalent to BICGSTAB....................................... 287
5.5.11 TFQMR:TransposeFreeQuasi-MinimalResidualV=?andW=?
to Make it Equivalent to TFQMR........................................... 291
5.5.12 Numerical Examples.......................................................... 292
5.6 Special Algorithms for Normal Equations.......................................... 294
5.6.1 Symmetrizing Transformations............................................. 295
5.6.2 Stationary Iterations for Normal Equations.............................. 296
5.6.3 Conjugate Gradient for Normal Equations............................... 300
5.6.4 RemarksonIterationAlgorithmsforConstrainedLinearEquations303
5.7 Other Important Topics................................................................. 304
5.7.1 Preconditioning Techniques................................................. 305
5.7.2 Parallel Computations........................................................ 307
5.7.3 Algebraic Multigrid Method................................................. 308
5.7.4 Domain Decomposition Method............................................ 310
CHAPTER 6 Direct Algorithms of Solution of Eigenvalue Problem 317
Introduction ............................................................................... 317
6.1 Algorithms of Solution of 2×2 Eigenvalue Problems ........................... 318
6.2 Bound Estimation of Symmetric Eigenvalue Problems ......................... 321
6.3 Power Iterations and Subspace Iterations.......................................... 326
6.4 QR Iterations.............................................................................. 327
6.4.1 Reduce A to Hessenberg Form............................................. 328
6.4.2 A QR Iteration Step for a Hessenberg Matrix........................... 328
6.4.3 Deflation......................................................................... 329
6.4.4 Shift............................................................................... 330
6.4.5 MATLAB Implementations of QR Iterations Algorithms.............. 333
6.5 Calculation of Eigenvectors by Inverse Iterations ................................ 336
6.6 Calculation of Invariant Subspace by Eigenvalue Reordering................. 340
6.7 Special Algorithms for Symmetric Tri-Diagonal Matrix.......................... 344
6.7.1 Eigenvalues by Bisection.................................................... 345
6.7.2 A Divide-and-Conquer Algorithm........................................... 347
6.8 Jacobi Iterations for Symmetric Matrix............................................. 353
6.9 Algorithms for Symmetrical Positive Definite Ax= λBx........................ 355
6.10 Algorithms for Unsymmetrical Ax= λBx........................................... 359