Table Of Content

Nathaniel Johnston Advanced Linear and Matrix Algebra Advanced Linear and Matrix Algebra Nathaniel Johnston Advanced Linear and Matrix Algebra 123 Nathaniel Johnston Department ofMathematics and Computer Science Mount Allison University Sackville, NB,Canada ISBN978-3-030-52814-0 ISBN978-3-030-52815-7 (eBook) https://doi.org/10.1007/978-3-030-52815-7 MathematicsSubjectClassification: 15Axx,97H60,00-01 ©SpringerNatureSwitzerlandAG2021 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeor part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway, andtransmissionorinformationstorageandretrieval,electronicadaptation,computersoftware, orbysimilarordissimilarmethodologynowknownorhereafterdeveloped. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthis publication does not imply, even in the absence of a specific statement, that such names are exemptfromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationin thisbookarebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernor the authors or the editors give a warranty, expressed or implied, with respect to the material containedhereinorforanyerrorsoromissionsthatmayhavebeenmade.Thepublisherremains neutralwithregardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland For Devon …whowasveryeageratage2tocontributetothisbook: ndfshfjdskfdshdsfkdfshkdsfhfdskhdfsk The Purpose of this Book Linear algebra, more so than any other mathematical subject, can be approached in numerous ways. Manytextbookspresentthesubjectinaveryconcreteandnumericalmanner,spendingmuchoftheir time solving systems of linear equations and having students perform laborious row reductions on matrices. Many other books instead focus very heavily on linear transformations and other basis-independent properties, almost to the point that their connection to matrices is considered an inconvenient after-thought that students should avoid using at all costs. This book is written from the perspective that both linear transformations and matrices are useful objects in their own right, but it is the connection between the two that really unlocks the magic of linearalgebra.Sometimeswhenwewanttoknowsomethingaboutalineartransformation,theeasiest way to getan answer isto grab onto a basis and look at thecorresponding matrix. Conversely, there aremanyinterestingfamiliesofmatricesandmatrixoperationsthatseeminglyhavenothingtodowith linear transformations, yet can nonetheless illuminate how some basis-independent objects and properties behave. This book introduces many difficult-to-grasp objects such as vector spaces, dual spaces, and tensor products.Becauseitisexpectedthatthisbookwillaccompanyoneofthefirstcourseswherestudents are exposed to such abstract concepts, we typically sandwich this abstractness between concrete examples.Thatis,wefirstintroduceoremphasizeastandard,prototypicalexampleoftheobjecttobe introduced (e.g., Rn), then we discuss its abstract generalization (e.g., vector spaces), and finally we explore other specific examples of that generalization (e.g., the vector space of polynomials and the vector space of matrices). Thisbookalsodelvessomewhatdeeperintomatrixdecompositionsthanmostothersdo.Weofcourse coverthesingularvaluedecompositionaswellasseveralofitsapplications,butwealsospendquitea bitoftime lookingattheJordandecomposition, Schurtriangularization, andspectraldecomposition, and wecompare and contrast them with each other to highlight when each one is appropriate to use. Computationally-motivated decompositions like the QR and Cholesky decompositions are also covered in some of this book’s many “Extra Topic” sections. Continuation of Introduction to Linear and Matrix Algebra This book is the second part of a two-book series, following the book Introduction to Linear and MatrixAlgebra[Joh20].Thereaderisexpectedtobefamiliarwiththebasicsoflinearalgebracovered in that book (as well as other introductory linear algebra books): vectors in Rn, the dot product, matrices and matrix multiplication, Gaussian elimination, the inverse, range, null space, rank, and vii viii Preface determinantofamatrix,aswellaseigenvaluesandeigenvectors.Thesepreliminarytopicsarebriefly reviewed in Appendix A.1. Because these books aim to not overlap with each other and repeat content, we do not discuss some topics that are instead explored in that book. In particular, diagonalization of a matrix via its eigen- valuesandeigenvectorsisdiscussedintheintroductorybookandnothere.However,manyextensions and variations of diagonalization, such as the spectral decomposition (Section 2.1.2) and Jordan decomposition (Section 2.4) are explored here. Features of this Book Thisbookmakesuseofnumerousfeaturestomakeitaseasytoreadandunderstandaspossible.Here we highlight some of these features and discuss how to best make use of them. Notes in the Margin This text makes heavy use of notes in the margin, which are used to introduce some additional terminologyorprovideremindersthatwouldbedistractinginthemaintext.Theyaremostcommonly used to try to address potential points of confusion for the reader, so it is best not to skip them. Forexample,ifwewanttoclarifywhyaparticularpieceofnotationisthewayitis,wedosointhe margin so as to not derail the main discussion. Similarly, if we use some basic fact that students are expected to be aware of (but have perhaps forgotten) from an introductory linear algebra course, the margin will contain a brief reminder of why it’s true. Exercises Several exercises can be found at the end of every section in this book, and whenever possible there are three types of them: (cid:129) There are computational exercises that ask the reader to implement some algorithm or make use of the tools presented in that section to solve a numerical problem. (cid:129) There are true/false exercises that test the reader’s critical thinking skills and reading com- prehension by asking them whether some statements are true or false. (cid:129) Thereareproofexercisesthataskthereadertoproveageneralstatement.Thesetypicallyare either routine proofs that follow straight from the definition (and thus were omitted from the main text itself), or proofs that can be tackled via sometechniquethat wesaw in that section. Roughlyhalfoftheexercisesaremarkedwithanasterisk((cid:1)),whichmeansthattheyhaveasolution providedinAppendixC.Exercisesmarkedwithtwoasterisks((cid:1)(cid:1))arereferencedinthemaintextand are thus particularly important (and also have solutions in Appendix C). To the Instructor and Independent Reader This book is intended to accompany a second course in linear algebra, either at the advanced undergraduate or graduate level. The only prerequisites that are expected of the reader are an introductory course in linear algebra (which is summarized in Appendix A.1) and some familiarity with mathematicalproofs.Itwillhelpthereadertohavebeenexposedtocomplexnumbers,thoughwedo little more than multiply and add them (their basics are reviewed in Appendix A.3). Preface ix The material covered in Chapters 1 and 2 is mostly standard material in upper-year undergraduate linear algebra courses. In particular, Chapter 1 focuses on abstract structures like vector spaces and innerproducts,toshowstudentsthatthetoolsdevelopedintheirpreviouslinearalgebracoursecanbe appliedtoamuchwidervarietyofobjectsthanjustlistsofnumberslikevectorsinRn.Chapter2then explores how we can use these new tools at our disposal to gain a much deeper understanding of matrices. Chapter3coverssomewhatmoreadvancedmaterial—multilinearityandthetensorproduct—whichis aimedparticularlyatadvancedundergraduatestudents(thoughwenotethatnoknowledgeofabstract algebraisassumed).Itcouldserveperhapsascontentforpartofathirdcourse,orasanindependent study in linear algebra. Alternatively, that chapter is also quite aligned with the author’s research interestsasaquantuminformationtheorist,anditcouldbeusedassupplementalreadingforstudents who are trying to learn the basics of the field. Sectioning The sectioning of the book is designed to make it as simple to teach from as possible. The author spends approximately the following amount of time on each chunk of this book: (cid:129) Subsection: 1–1.5 hour lecture (cid:129) Section: 2 weeks (3–4 subsections per section) (cid:129) Chapter: 5–6 weeks (3–4 sections per chapter) (cid:129) Book: 12-week course (2 chapters, plus some extra sections) Of course, this is just a rough guideline, as some sections are longer than others. Furthermore, some instructors may choose to include material from Chapter 3, or from some of the numerous in-depth “Extra Topic” sections. Alternatively, the additional topics covered in those sections can serve as independent study topics for students. Extra Topic Sections Halfofthisbook’ssectionsarecalled“ExtraTopic”sections.Thepurposeofthebookbeingarranged inthiswayisthatitprovidesaclearmainpaththroughthebook(Sections1.1–1.4,2.1–2.4,and3.1– 3.3)that can be supplemented by the Extra Topicsections at the reader’s/instructor’s discretion. It is expected that many courses will not even make it to Chapter 3, and instead will opt to explore some of the earlier Extra Topic sections instead. We want to emphasize that the Extra Topic sections are not labeled as such because they are less important than the main sections, but only because they are not prerequisites for any of the main sections. For example, norms and isometries (Section 1.D) are used constantly throughout advanced mathematics,buttheyarepresentedinanExtraTopicsectionsincetheothersectionsofthisbookdo not depend on them (and also because they lean quite a bit into “analysis territory”, whereas most of the rest of the book stays firmly in “algebra territory”). Similarly, the author expects that many instructors will include the section on the direct sum and orthogonal complements(Section 1.B) as part of their course’s core material, butthis can be done at their discretion. The subsections on dual spaces and multilinear forms from Section 1.3.2 can be omittedreasonablysafelytomakeupsometimeifneeded,ascanthesubsectiononGershgorindiscs (Section 2.2.2), without drastically affecting the book’s flow. For a graph that depicts the various dependencies of the sections of this book on each other, see Figure H. x Preface IntroductiontoLinearandMatrixAlgebra Chapter1: VectorSpaces §1.1 §1.2 §1.3 §1.4 Vectorspaces Lin. transforms Isomorphisms Orthogonality §1.A §1.B §1.C §1.D Tracev2 Directsum QRdecomp. Norms,isometry Chapter2: MatrixDecompositions §2.1 §2.2 §2.3 §2.4 Spectraldecomp. Positivedefinite Singularvalues Jordandecomp. §2.A §2.B §2.C §2.D Quadrat.forms Schurcomplem. Singularvals.v2 Matrixanalysis Chapter3: TensorsandMultilinearity §3.1 §3.2 §3.3 Kroneckerproduct Multilinearity Tensorproduct §3.A §3.B §3.C Matrixmaps Homogen. polynomials Semidef.programming FigureH: Agraphdepictingthedependenciesofthesectionsofthisbookoneachother.Solidarrowsindicatethatthesection isrequiredbeforeproceedingtothesectionthatitpointsto,whiledashedarrowsindicaterecommended(butnotrequired)prior reading.ThemainpaththroughthebookconsistsofSections1–4ofeachchapter.TheextrasectionsA–Dareoptionalandcan beexploredatthereader’sdiscretion,asnoneofthemainsectionsdependonthem. Acknowledgments Thanks are extended to Geoffrey Cruttwell, Mark Hamilton, David Kribs, Chi-Kwong Li, Benjamin Lovitz,NeilMcKay,VernPaulsen,RajeshPereira,SarahPlosker,JamieSikora,andJohnWatrousfor various discussions that have either directly or indirectly improved the quality of this book. Thank you to Everett Patterson, as well as countless other students in my linear algebra classes at Mount Allison University, for drawing my attention to typos and parts of the book that could be improved. Preface xi Parts of the layout of this book were inspired by the Legrand Orange Book template by Velimir Gayevskiy and Mathias Legrand at LaTeXTemplates.com. Finally,thankyoutomywifeKathrynfortoleratingmeduringtheyearsofmymentalabsenceglued to this book, and thank you to my parents for making me care about both learning and teaching. Sackville, NB, Canada Nathaniel Johnston