Linear Algebra

283 Pages·2017·0.829 MB·English

by  Taylor M

Linear Algebra
Michael Taylor
Contents
1. Vector spaces
2. Linear transformations and matrices
3. Basis and dimension
4. Matrix representation of a linear transformation
5. Determinants and invertibility
Row reduction, matrix products, and Gaussian elimination
Vandermonde determinant
6. Eigenvalues and eigenvectors
7. Generalized eigenvectors and the minimal polynomial
8. Triangular matrices and upper triangularization
Companion matrices
9. Inner products and norms
10. Norm, trace, and adjoint of a linear transformation
11. Self-adjoint and skew-adjoint transformations
12. Unitary and orthogonal transformations
13. The Jordan canonical form
14. Schur’s upper triangular representation
15. Polar decomposition and singular value decomposition
16. Dual spaces
17. Convex sets
18. Quotient spaces
19. Multilinear mappings
20. Tensor products
1
AMS Open Math Notes: Works in Progress; Reference # OMN:201704.110695; 2017-04-14 09:02:55
2 Linear Algebra Michael Taylor
21. Exterior algebra
22. Vector spaces over more general fields
23. Rings and modules
Modules over a PID
24. The Jordan canonical form revisited
25. The matrix exponential
A. The fundamental theorem of algebra
B. Further observations on row reduction and column reduction
C. Rational matrices and algebraic numbers
Algebraic integers
D. Isomorphism Skew(V) ≈ Λ2V and the Pfaffian
E. Groups
F. Quaternions and matrices of quaternions
G. Algebras
H. Clifford algebras
I. Octonions
J. Noetherian rings and Noetherian modules
K. Polynomial rings over UFDs
L. Finite fields and other algebraic field extensions
Introduction
Linear algebra is an important gateway connecting elementary mathematics
to more advanced subjects, such as multivariable calculus, systems of dif-
ferential equations, differential geometry, and group representations. The
purpose of this work is to provide a compact but efficient treatment of this
topic.
In §1 we define the class of vector spaces (real and complex) and discuss
some basic examples, including Rn and Cn, or, as we denote them, Fn, with
F = R or C. In §2 we consider linear transformations between such vector
spaces. In particular we look at an m × n matrix A as defining a linear
transformation A : Fn → Fm. We define the range R(T) and null space
N(T) of a linear transformation T : V → W. In §3 we define the notion
of basis of a vector space. Vector spaces with finite bases are called finite
dimensional. We establish the crucial property that any two bases of such
a vector space V have the same number of elements (denoted dim V). We
apply this to other results on bases of vector spaces, culminating in the
“fundamental theorem of linear algebra,” that if T : V → W is linear and
V is finite dimensional, then dimN(T)+dimR(T) = dimV, and discuss
some of its important consequences.
A linear transformation T : V → V is said to be invertible provided it is
one-to-oneandonto,i.e.,providedN(T) = 0andR(T) = V. In§5wedefine
AMS Open Math Notes: Works in Progress; Reference # OMN:201704.110695; 2017-04-14 09:02:55
Introduction 3
the determinant of such T, detT (when V is finite dimensional), and show
thatT isinvertibleifandonlyifdetT (cid:54)= 0. In§6westudyeigenvaluesλ and
j
eigenvectors v of such a transformation, defined by Tv = λ v . Results
j j j j
of §5 imply λ is a root of the “characteristic polynomial” det(λI − T).
j
Section 7 extends the scope of §6 to a treatment of generalized eigenvectors.
This topic is connected to properties of nilpotent matrices and triangular
matrices, studied in §8.
In §9 we treat inner products on vector spaces, which endow them with
a Euclidean geometry, in particular with a distance and a norm. In §10 we
discuss two types of norms on linear transformations, the “operator norm”
and the “Hilbert-Schmidt norm.” Then, in§§11–12, we discuss some special
classes of linear transformations on inner product spaces: self-adjoint, skew-
adjoint, unitary, and orthogonal transformations.
Section13dealswiththeJordannormalformofacomplexn×nmatrix,
and §14 establishes a theorem of Schur that for eachn×n matrix A, there is
anorthonormalbasisofCnwithrespecttowhichAtakesanuppertriangular
form. Section 15 establishes a polar decomposition result, that each n×n
complex matrix can be written as KP, with K unitary and P positive
semidefinite, and a related result known as the singular value decomposition
of a complex matrix (square or rectangular).
In §16 we define the dual space V(cid:48) to a vector space V. We associate
to a linear map A : V → W its transpose At : W(cid:48) → V(cid:48) and establish
a natural isomorphism V ≈ (V(cid:48))(cid:48) when dimV < ∞. Section 17 looks at
convex subsets of a finite dimensional vector space. Section 18 deals with
quotient spaces V/W when W is a linear subspace of V.
Sections 19–21 deal with multilinear maps and related constructions,
including tensor products in §20 and exterior algebra in §21, which we ap-
proach as a further development of the theory of the determinant, initiated
in §5. Results of these sections are particularly useful in the development of
differential geometry and manifold theory, involving studies of tensor fields
and differential forms.
In §22, we extend the scope of our study of vector spaces, adding to R
and C more general fields F. We define the notion of a field, give a number
of additional examples, and describe how results of §§1–8, 13, and 16–21
extend to vector spaces over a general field F. Specific fields considered
include both finite fields Z/(p) and fields of algebraic numbers. In §23 we
extendthescopefurther,fromvectorspacesoverfieldstomodulesoverrings.
Specific rings considered include the ring Z of integers, rings of polynomials,
and matrix rings. We discuss R-linear maps between two such R-modules,
for such rings R. We compare and contrast the theories of modules and of
vector spaces. We pay particular attention to modules over principal ideal
AMS Open Math Notes: Works in Progress; Reference # OMN:201704.110695; 2017-04-14 09:02:55
4 Linear Algebra Michael Taylor
domains (PIDs). Examples of PIDs include both Z and polynomial rings
F[t].
In §24 we revisit the results obtained in §7 and §13 on generalized
eigenspacesandtheJordancanonicalformforA ∈ L(V),andshowhowthey
follow from results on the structure of R-modules in §23, when R = F[t].
In §25, we return to the setting of real and complex n×n matrices and
define the matrix exponential, etA, so that x(t) = etAv solves the differen-
tial equation dx/dt = Ax, x(0) = v. We produce a power series for etA
and establish some basic properties. The matrix exponential is fundamen-
tal to applications of linear algebra to ODE. Here, we use this connection
to produce another proof that if A is an n × n complex matrix, then Cn
has a basis consisting of generalized eigenvectors of A. The proof here is
completely different from that given in §7.
We end with some appendices. The first appendix gives a proof of the
fundamental theorem of algebra, that every nonconstant polynomial has
complex roots. This result has several applications in §§6–7.
Appendix B revisits a theme from §5, and shows how applying row
reduction to an m×n matrix A works to display a basis of its null space,
while applying column reduction to A works to display a basis of its range.
We also apply row reduction to LU-factorization.
In Appendix C we take a second look at the set A of algebraic numbers,
whicharerootsofpolynomialswithrationalcoefficients. Weshowthatthey
are precisely the eigenvalues of square matrices with rational entries, and
use this, together with some results of §20, to show that sums, products,
and reciprocals of (nonzero) algebraic numbers are also algebraic. That
is to say, A is a field. A different proof of this is given in §22. We also
look at the set O of algebraic integers, which are roots of polynomials with
integer coefficients, with leading coefficient 1. We show these are precisely
the eigenvalues of square matrices with integer entries, and use this to prove
that O is a ring. We discuss a test for when an element of A belongs to O.
Appendix D introduces a subtle cousin to the determinant, known as
the Pfaffian, Pf(X), defined when X is a real n×n skew-symmetric matrix,
and n = 2k is even.
Appendix E brings up another algebraic structure, that of a group. It
describeshowvariousgroupshaveariseninthetext, andpresentsafewgen-
eral observations on these objects, with emphasis on two classes of groups:
infinite matrix groups like G(cid:96)(n,R), which are Lie groups, on the one hand,
and groups like the permutation groups S , which are finite groups, on the
n
other. We cap our treatment of basic results on groups with a discussion
AMS Open Math Notes: Works in Progress; Reference # OMN:201704.110695; 2017-04-14 09:02:55
Introduction 5
of an application to a popular encryption scheme, based on a choice of two
large prime numbers.
InAppendixFwediscussquaternions,objectsoftheforma+bi+cj+dk
with a,b,c,d ∈ R, which form a noncommutative ring H, with a number of
interesting properties. In particular, the quaternion product captures both
the dot product and the cross product of vectors in R3. We also discuss
matrices with entries in H, with special attention to a family of groups
Sp(n) ⊂ M(n,H).
AppendixGdiscussesthegeneralconceptofanalgebra,anobjectthatis
simultaneouslyavectorspaceoverafieldFandaring,suchthattheproduct
is F-bilinear. Many of the rings introduced earlier, such as L(V) and H, are
algebras, but some, such as Z and Z[t], are not. We introduce some new
ones, such as the tensor algebra ⊗∗V associated to a vector space, and the
tensor product A⊗B of two algebras. Properly speaking, these algebras are
associative algebras. We briefly mention a class of nonassociative algebras
known as Lie algebras, and another class, known as Jordan algebras.
AppendixHtreatsanimportantclassofalgebrascalledCliffordalgebras.
These are intimately related to the construction of a class of differential
operators known as Dirac operators.
Appendix I treats an intriguing nonassociative algebra called the set of
octonions (or Cayley numbers). We discuss similarities and differences with
the algebra of quaternions.
Appendix J discusses the class of Noetherian rings and the associated
class of Noetherian modules. This class of rings, defined by a certain finite-
ness condition, contains the class of PIDs. It also contains other important
classes of rings, in particular polynomial rings in several variables, a re-
sult known as the Hilbert basis theorem. Even as the class of Noetherian
rings is preserved under passing from R to R[x], so is the class of unique
factorization domains. We prove this in Appendix K.
Appendix L produces new fields F(cid:101) from old fields, constructed so that a
polynomial P ∈ F[x] without roots in F will have roots in F(cid:101). In particular,
we obtain all finite fields in this fashion, proceeding from the fields Z/(p).
Material in this appendix puts the reader in a position to tackle treatments
of Galois theory.
The material presented here could serve for a two semester course in
linear algebra. For a one semester course, I recommend a straight shot
through Sections 1–12, with attention to Appendices A and B. Material in
Sections 13–25 and a selection from the appendices could work well in a
second semester course. To be sure, there is considerable flexibility in the
presentation of this material, and one might try some different orderings.
AMS Open Math Notes: Works in Progress; Reference # OMN:201704.110695; 2017-04-14 09:02:55
6 Linear Algebra Michael Taylor
For example, one might want to present §25, on the matrix exponential,
much earlier, anywhere after §10. As another example, one could move
§22, on vector spaces over general fields, way up, maybe right after §8 (and
maybe similarly move §13 up). In any case, I encourage the student/reader
to sample all the sections, as an encounter with the wonderful mathematical
topic that is linear algebra.
Acknowledgments. Thanks to Robert Bryant for useful conversations
related to various topics treated here, particularly regarding octonions.
Material in Sections 1–14 follows closely the presentation of basic linear
algebra in Chapter 2 of my text [T4], Introduction to Differential Equations,
publishedbytheAmericanMathematicalSociety. IamgratefultotheAMS
for permission to use this material here.
AMS Open Math Notes: Works in Progress; Reference # OMN:201704.110695; 2017-04-14 09:02:55
1. Vector spaces 7
1. Vector spaces
We expect the reader is familiar with vectors in the plane R2 and 3-space
R3. More generally we have n-space Rn, whose elements consist of n-tuples
of real numbers:
(1.1) v = (v ,...,v ).
1 n
There is vector addition; if also w = (w ,...,w ) ∈ Rn,
1 n
(1.2) v+w = (v +w ,...,v +w ).
1 1 n n
There is also multiplication by scalars; if a is a real number (a scalar),
(1.3) av = (av ,...,av ).
1 n
Wecouldalsousecomplexnumbers,replacingRn byCn,andallowinga ∈ C
in (1.3). We will use F to denote R or C.
Many other vector spaces arise naturally. We define this general notion
now. A vector space over F is a set V, endowed with two operations, that
of vector addition and multiplication by scalars. That is, given v,w ∈ V
and a ∈ F, then v+w and av are defined in V. Furthermore, the following
properties are to hold, for all u,v,w ∈ V, a,b ∈ F. First there are laws for
vector addition:
(1.4) Commutative law : u+v = v+u,
(1.5) Associative law : (u+v)+w = u+(v+w),
(1.6) Zero vector : ∃ 0 ∈ V, v+0 = v,
(1.7) Negative : ∃−v, v+(−v) = 0.
Next there are laws for multiplication by scalars:
(1.8) Associative law : a(bv) = (ab)v,
(1.9) Unit : 1·v = v.
Finally there are two distributive laws:
(1.10) a(u+v) = au+av,
(1.11) (a+b)u = au+bu.
It is easy to see that Rn and Cn satisfy all these rules. We will present
a number of other examples below. Let us also note that a number of other
AMS Open Math Notes: Works in Progress; Reference # OMN:201704.110695; 2017-04-14 09:02:55
8 Linear Algebra Michael Taylor
simple identities are automatic consequences of the rules given above. Here
are some, which the reader is invited to verify:
v+w = v ⇒ w = 0,
v+0·v = (1+0)v = v,
0·v = 0,
(1.12)
v+w = 0 ⇒ w = −v,
v+(−1)v = 0·v = 0,
(−1)v = −v.
Above we represented elements of Fn as row vectors. Often we represent
elements of Fn as column vectors. We write
   
v av +w
1 1 1
. .
(1.13) v =  .. , av+w =  .. .
v av +w
n n n
We give some other examples of vector spaces. Let I = [a,b] denote an
intervalinR, andtakeanon-negativeintegerk. ThenCk(I)denotesthe set
of functions f : I → F whose derivatives up to order k are continuous. We
denote by P the set of polynomials in x, with coefficients in F. We denote
by P the set of polynomials in x of degree ≤ k. In these various cases,
k
(1.14) (f +g)(x) = f(x)+g(x), (af)(x) = af(x).
Such vector spaces and certain of their linear subspaces play a major role in
the material developed in these notes.
Regarding the notion just mentioned, we say a subset W of a vector
space V is a linear subspace provided
(1.15) w ∈ W, a ∈ F =⇒ a w +a w ∈ W.
j j 1 1 2 2
Then W inherits the structure of a vector space.
Exercises
1. Specify which of the following subsets of R3 are linear subspaces:
(a) {(x,y,z) : xy = 0},
(b) {(x,y,z) : x+y = 0},
(c) {(x,y,z) : x ≥ 0, y = 0, z = 0},
(d) {(x,y,z) : x is an integer},
(e) {(x,y,z) : x = 2z, y = −z}.
AMS Open Math Notes: Works in Progress; Reference # OMN:201704.110695; 2017-04-14 09:02:55
1. Vector spaces 9
2. Show that the results in (1.12) follow from the basic rules (1.4)–(1.11).
Hint. To start, add −v to both sides of the identity v + w = v, and
take account first of the associative law (1.5), and then of the rest of
(1.4)–(1.7). For the second line of (1.12), use the rules (1.9) and (1.11).
Then use the first two lines of (1.12) to justify the third line...
3. Demonstrate that the following results for any vector space. Take a ∈
F, v ∈ V.
a·0 = 0 ∈ V,
a(−v) = −av.
Hint. Feel free to use the results of (1.12).
Let V be a vector space (over F) and W,X ⊂ V linear subspaces. We
say
(1.16) V = W +X
provided each v ∈ V can be written
(1.17) v = w+x, w ∈ W, x ∈ X.
We say
(1.18) V = W ⊕X
provided each v ∈ V has a unique representation (1.17).
4. Show that
V = W ⊕X ⇐⇒ V = W +X and W ∩X = 0.
5. Take V = R3. Specify in each case (a)–(c) whether V = W +X and
whether V = W ⊕X.
(a) W = {(x,y,z) : z = 0}, X = {(x,y,z) : x = 0},
(b) W = {(x,y,z) : z = 0}, X = {(x,y,z) : x = y = 0},
(c) W = {(x,y,z) : z = 0}, X = {(x,y,z) : y = z = 0}.
AMS Open Math Notes: Works in Progress; Reference # OMN:201704.110695; 2017-04-14 09:02:55
10 Linear Algebra Michael Taylor
6. If W ,...,W are linear subspaces of V, extend (1.16) to the notion
1 m
(1.19) V = W +···+W ,
1 m
and extend (1.18) to the notion that
(1.20) V = W ⊕···⊕W .
1 m
AMS Open Math Notes: Works in Progress; Reference # OMN:201704.110695; 2017-04-14 09:02:55