Applied Linear Algebra and Matrix Analysis

488 Pages·2018·5.215 MB·English

by  Thomas S. Shores

Thomas S. Shores
Applied Linear Algebra
and Matrix Analysis
Second Edition
ThomasS.Shores
DepartmentofMathematics
UniversityofNebraska
Lincoln,NE
USA
ISSN0172-6056 ISSN2197-5604 (electronic)
UndergraduateTextsinMathematics
ISBN978-3-319-74747-7 ISBN978-3-319-74748-4
https://doi.org/10.1007/978-3-319-74748-4
LibraryofCongressControlNumber:2018930352
1stedition:(cid:2)c SpringerScience+BusinessMedia,LLC2007
2ndedition:(cid:2)c SpringerInternationalPublishingAG,partofSpringerNature2018
Preface
Preface to Revised Edition
Times change. So do learning needs, learning styles, students, teachers,
authors, and textbooks. The need for a solid understanding of linear algebra
and matrix analysis is changing as well. Arguably, as we move deeper into an
ageofintellectualtechnology,thisneedisactuallygreater.Witness,forexam-
ple, Google’s PageRank technology, an application that has a place in nearly
every chapter of this text. In the first edition of this text (henceforth refer-
enced as ALAMA), I suggested that for many students “linear algebra will be
as fundamental in their professional work as the tools of calculus.” I believe
now that this applies to most students of technology. Hence, this revision.
Sowhathaschangedinthisrevision?Theobjectivesofthistext,asstated
in the preface to ALAMA, have not:
• Toprovideabalancedblendofapplications,theory,andcomputationthat
emphasizes their interdependence.
• To assist those who wish to incorporate mathematical experimentation
through computer technology into the class. Each chapter has computer
exercises sprinkled throughout and an optional section on applications
and computational notes. Students should use locally available tools to
carry out experiments suggested in projects and use the word processing
capabilities of their computer system to create reports of their results.
• To help students to express their thoughts clearly. Requiring written
reports is one vehicle for teaching good expression of mathematical ideas.
• To encourage cooperative learning. Mathematics educators have become
increasinglyappreciativeofthispowerfulmodeoflearning.Teamprojects
and reports are excellent vehicles for cooperative learning.
• Topromoteindividuallearningbyprovidingacompleteandreadabletext.
I hope that readers will find this text worthy of being a permanent part
of their reference library, particularly for the basic linear algebra needed
in the applied mathematical sciences.
What has changed in this revision is that I have incorporated improve-
ments in readability, relevance, and motivation suggested to me by many
readers. Readers have also provided many corrections and comments which
havebeenaddedtotherevision.Inaddition,eachchapterofthisrevisedtext
concludes with introductions to some of the more significant applications of
linear algebra in contemporary technology. These include graph theory and
networkmodelingsuchasGoogle’sPageRank;alsoincludedaremodelingex-
amples of diffusive processes, linear programming, image processing, digital
signal processing, Fourier analysis, and more.
Thefirstedition madespecificreferencestovarious computeralgebra sys-
tem (CAS) and matrix algebra system (MAS) computer systems. The pro-
liferation of matrix-computing–capable devices (desktop computers, laptops,
PDAs, tablets, smartphones, smartwatches, calculators, etc.) and attendant
software makes these acronyms too narrow. And besides, who knows what’s
next ... bionic chip implants? Instructors have a large variety of systems and
devices to make available to their students. Therefore, in this revision, I will
refertoanysuchdeviceorsoftwareplatformasa“technologytool.” Iwillcon-
fineoccasionalspecificreferencestoafewfreelyavailabletoolssuchasOctave,
theRprogramminglanguage,andtheALAMACalculatorwhichwaswritten
by me specifically for this textbook.
Althoughcalculusisusuallyaprerequisiteforacollege-levellinearalgebra
course, this revision could very well be used in a non-calculus–based course
without loss of matrix and linear algebra content by skipping any calculus-
basedtextexamplesorexercises.Indeed,formanystudentsthetoolsofmatrix
and linear algebra will be as fundamental in their professional work as the
tools of calculus if not more so; thus, it is important to ensure that students
appreciatetheutilityandbeautyofthesesubjectsaswellasthemechanics.To
thisend,appliedmathematicsandmathematicalmodelinghaveanimportant
role in an introductory treatment of linear algebra. In this way, students see
thatconceptsofmatrixandlinearalgebramakeotherwiseintractableconcrete
problems workable.
The text has a strong orientation toward numerical computation and
applied mathematics, which means that matrix analysis plays a central role.
All three of the basic components of linear algebra — theory, computation,
and applications — receive their due. The proper balance of these compo-
nents gives students the tools they need as well as the motivation to acquire
these tools. Another feature of this text is an emphasis on linear algebra as
anexperimentalscience;thisemphasisisfoundincertainexamples,computer
exercises, and projects. Contemporary mathematical technology tools make
ideal “laboratories” for mathematical experimentation. Nonetheless, this text
is independent of specific hardware and software platforms. Applications and
ideas should take center stage, not hardware or software.
An outline of the book is as follows: Chapter 1 contains a thorough
development of Gaussian elimination. Along the way, complex numbers and
the basic language of sets are reviewed early on; experience has shown that
thismaterialisfrequentlylongforgottenbymanystudents,sosuchareviewis
warranted.Basicpropertiesofmatrixarithmeticanddeterminantalgebraare
developedinChapter2.Specialtypesofmatrices,suchaselementaryandsym-
metric, are also introduced. Chapter 3 begins with the “standard” Euclidean
vector spaces, both real and complex. These provide motivation for the more
sophisticated ideas of abstract vector space, subspace, and basis, which are
introducedsubsequentlylargelyinthecontextofthestandardspaces.Chapter
4 introduces geometrical aspects of standard vector spaces such as norm, dot
product, and angle. Chapter 5 introduces eigenvalues and eigenvectors. Gen-
eralnormandinnerproductconceptsforabstractvectorspacesareexamined
in Chapter 6. Each section concludes with a set of exercises and problems.
Each chapter contains a few more optional topics, which are independent
of the non-optional sections. Of course, one instructor’s optional is another’s
mandatory. Optional sections cover tensor products, change of basis and lin-
ear operators, linear programming, the Schur triangularization theorem, the
singular value decomposition, and operator norms. In addition, each chapter
has an optional section of applications and computational notes which has
been considerably expanded from the first edition along with a concluding
section of projects and reports. I employ the convention of marking sections
and subsections that I consider optional with an asterisk.
Thereismorethanenoughmaterialinthisbookforaone-semestercourse.
Tastes vary, so there is ample material in the text to accommodate different
interests. One could increase emphasis on any one of the theoretical, applied,
or computational aspects of linear algebra by the appropriate selection of
syllabus topics. The text is well suited to a course with a three-hour lecture
and laboratory component, but computer-related material is not mandatory.
Every instructor has his/her own idea about how much time to spend on
proofs, how much on examples, which sections to skip, etc.; so the amount of
material covered will vary considerably. Instructors may mix and match any
of the optional sections according to their own interests and needs of their
students, since these sections are largely independent of each other. While it
wouldbeverytime-consumingtocoverthemall,everyinstructoroughttouse
some part of this material. The unstarred sections form the core of the book;
most of this material should be covered. There are 27 unstarred sections and
17 optional sections. I hope the optional sections come in enough flavors to
please any pure, applied, or computational palate.
Of course, no one size fits all, so I will suggest two examples of how one
might use this text for a three-hour one-semester course. Such a course will
typicallymeetthreetimesaweekforfifteenweeks,foratotalof45classes.The
material of most of the unstarred sections can be covered at an average rate
of about one and one-half class periods per section. Thus, the core material
couldbecoveredinabout40orfewerclassperiods.Thisleavestimeforextra
sections and in-class examinations. In a two-semester course or a course of
more than three hours, one could expect to cover most, if not all, of the text.
If the instructor prefers a course that emphasizes the standard Euclidean
spaces, and moves at a more leisurely pace, then the core material of the first
five chapters of the text is sufficient. This approach reduces the number of
unstarred sections to be covered from 27 to 23.
About numbering: Exercises and problems are numbered consecutively
in each section. All other numbered items (sections, theorems, definitions,
examples, etc.) are numbered consecutively in each chapter and are prefixed
by the chapter number in which the item occurs. About examples: In this
text, these are illustrative problems, so each is followed by a solution.
I employ the following taxonomy for the reader tasks presented in this
text. Exercises constitute the usual learning activities for basic skills; these
come in pairs, and solutions to the odd-numbered exercises are given in an
appendix. More advanced conceptual or computational exercises that ask for
explanationsorexamplesaretermedproblems,andsolutionsforproblemsare
not given, but hints are supplied for those problems marked with an asterisk.
Some of these exercises and problems are computer-related. As with pencil-
and-paperexercises,thesearelearningactivitiesforbasicskills.Thedifference
is that some computing equipment is required to complete such exercises and
problems.Atthenextlevelareprojects.Theseassignmentsinvolveideasthat
extend the standard text material, possibly some numerical experimentation
and some written exposition in the form of brief project papers. These are
analogous to laboratory projects in the physical sciences. Finally, at the top
level are reports. These require a more detailed exposition of ideas, consid-
erable experimentation — possibly open ended in scope — and a carefully
written report document. Reports are comparable to “scientific term papers.”
They approximate the kind of activity that many students will beinvolved in
throughout their professional lives and are well suited for team efforts. The
projects and reports in this text also provide templates for instructors who
wishtobuildtheirownproject/reportmaterials.Studentsareopentoallsorts
of technology in mathematics. This openness, together with the availability
of inexpensive high-technology tools, has changed how and what we teach in
linear algebra.
Iwouldlike tothankmycolleagues whoseencouragement, ideas,andsug-
gestionshelpedmecompletethisproject,particularlyKristinPfabeandDavid
Logan. Also, thanks to all those who sent me helpful comments and correc-
tions,particularlyDavidTaylor,DavidCox,andMatsDesaix.Finally,Iwould
like to thank the outstanding staff at Springer for their patience and support
in bringing this project to completion.
A linear algebra page with some useful materials for instructors and stu-
dents using this text can be reached at
http://www.math.unl.edu/∼tshores1/mylinalg.html
Suggestions, corrections, or comments are welcome. These may be sent to
me at [email protected].
Contents
1 LINEAR SYSTEMS OF EQUATIONS .................... 1
1.1 Some Examples ......................................... 1
1.2 Notation and a Review of Numbers ........................ 12
1.3 Gaussian Elimination: Basic Ideas ......................... 24
1.4 Gaussian Elimination: General Procedure................... 37
1.5 *Applications and Computational Notes.................... 52
1.6 *Projects and Reports ................................... 61
2 MATRIX ALGEBRA...................................... 65
2.1 Matrix Addition and Scalar Multiplication.................. 65
2.2 Matrix Multiplication .................................... 72
2.3 Applications of Matrix Arithmetic ......................... 83
2.4 Special Matrices and Transposes...........................103
2.5 Matrix Inverses .........................................118
2.6 Determinants ...........................................141
2.7 *Tensor Products........................................160
2.8 *Applications and Computational Notes....................166
2.9 *Projects and Reports ...................................177
3 VECTOR SPACES ........................................181
3.1 Definitions and Basic Concepts............................181
3.2 Subspaces ..............................................198
3.3 Linear Combinations.....................................206
3.4 Subspaces Associated with Matrices and Operators ..........220
3.5 Bases and Dimension ....................................229
3.6 Linear Systems Revisited .................................239
3.7 *Change of Basis and Linear Operators ....................248
3.8 *Introduction to Linear Programming......................254
3.9 *Applications and Computational Notes....................273
3.10 *Projects and Reports ...................................274
4 GEOMETRICAL ASPECTS OF STANDARD SPACES ...277
4.1 Standard Norm and Inner Product ........................277
4.2 Applications of Norms and Vector Products.................288
4.3 Orthogonal and Unitary Matrices..........................302
4.4 *Applications and Computational Notes....................314
4.5 *Projects and Reports ...................................327
5 THE EIGENVALUE PROBLEM ..........................331
5.1 Definitions and Basic Properties...........................331
5.2 Similarity and Diagonalization ............................343
5.3 Applications to Discrete Dynamical Systems ................354
5.4 Orthogonal Diagonalization...............................366
5.5 *Schur Form and Applications ............................372
5.6 *The Singular Value Decomposition........................375
5.7 *Applications and Computational Notes....................379
5.8 *Project Topics .........................................386
6 GEOMETRICAL ASPECTS OF ABSTRACT SPACES ...391
6.1 Normed Spaces..........................................391
6.2 Inner Product Spaces ....................................398
6.3 Orthogonal Vectors and Projection ........................410
6.4 Linear Systems Revisited .................................418
6.5 *Operator Norms........................................424
6.6 *Applications and Computational Notes....................431
6.7 *Projects and Reports ...................................442
Table of Symbols ..............................................445
Solutions to Selected Exercises ................................447
References.....................................................469
Index..........................................................471
1
LINEAR SYSTEMS OF EQUATIONS
Welcometotheworldoflinearalgebra.Thetwocentralproblemsaboutwhich
much of the theory of linear algebra revolves are the problem of finding all
solutions to a linear system and that of finding an eigensystem for a square
matrix.ThelatterproblemwillnotbeencountereduntilChapter5;itrequires
some background development and the motivation for this problem is fairly
sophisticated. By contrast, the former problem is easy to understand and
motivate. As a matter of fact, simple cases of this problem are a part of most
high-school algebra backgrounds. We will address the problem of existence of
solutionsforalinearsystemandhowtosolvesuchasystemforallofitssolu-
tions. Examples of linear systems appear in nearly every scientific discipline;
we touch on a few in this chapter.
1.1 Some Examples
Here are a few very elementary examples of linear systems:
Example 1.1.
For what values of the unknowns x and y are the following
equations satisfied?
x+2y = 5
4x+y =6.
Solution.
One way that we were taught to solve this problem was the
geometrical approach: every equation of the form ax+by+c = 0 represents
the graph of a straight line. Thus, each equation above represents a line.
We need only graph each of the lines, then look for the point where these
lines intersect, to find the unique solution to the graph (see Figure 1.1). Of
course, the two equations may represent the same line, in which case there
are infinitely many solutions, or distinct parallel lines, in which case there
are no solutions. These could be viewed as exceptional or “degenerate” cases.
Normally, we expect the solution to be unique, which it is in this example.
Wealsolearnedhowtosolvesuchanequationalgebraically:inthepresent
casewemayuseeitherequationtosolveforonevariable,sayx,andsubstitute