An Introduction to Linear Algebra

136 Pages·2015·1.023 MB·English

by  Andrew D. Hwang

An Introduction to Linear Algebra
Andrew D. Hwang
February 2015
AMS Open Math Notes: Works in Progress; Reference # OMN:201801.110759; Last Revised: 2018-01-20 09:27:31
AMS Open Math Notes: Works in Progress; Reference # OMN:201801.110759; Last Revised: 2018-01-20 09:27:31
To the Student
Linear algebra comprises a variety of topics and viewpoints, including
computational machinery (matrices), abstract objects (vector spaces),
mappings (linear transformations), and the “fine structure” of linear
transformations (diagonalizability).
Like all mathematical subjects at an introductory level, linear al-
gebra is driven by examples and comprehended by unfamiliar theory.
Each of the preceding topics can be difficult to assimilate until the
others are understood. You, the reader, consequently face a chicken-
and-egg problem: Examples appear unconnected without a theoretical
framework, but theory without examples tends to be dry and unmoti-
vated.
This preface sketches an overview of the entire book by looking at
a family of representative examples. The “universe” is the Cartesian
plane R2, whose coordinates we denote (x1,x2). (The use of indices
instead of the more familiar (x,y) will economize our use of letters,
particularly when we begin to study functions of arbitrarily many vari-
ables. The use of superscripts as indices rather than as exponents
highlights important, subtle structure in formulas.) We view ordered
pairs as individual entities, and write x = (x1,x2).
Vectors and Vector Spaces
We view an ordered pair x as a vector, a type of object that can be
added to another vector, or multiplied by a real constant (called a
scalar) to obtain another vector. If x = (x1,x2) and y = (y1,y2), and
if c is a scalar, we define
x+y = (x1 +y1,x2 +y2), cx = (cx1,cx2).
The set R2 equipped with these operations is said to be a vector space.
The vector x = (x1,x2) in the plane may be viewed geometrically
as the arrow with its tail at the origin 0 = (0,0) and its tip at the
iii
AMS Open Math Notes: Works in Progress; Reference # OMN:201801.110759; Last Revised: 2018-01-20 09:27:31
iv LINEAR ALGEBRA
point x. Vector addition corresponds to forming the parallelogram
with sides x and y, and taking x + y to be the far corner. Scalar
multiplication cx corresponds to “stretching” x by a factor of c if c > 0,
or to stretching by a factor of c and reversing the direction if c < 0.
| |
y = (y1,y2)
c2x −x c1x+y
x+y
(0,0)
c x
1
x = (x1,x2)
Linear Transformations
In linear algebra, most mappings send vector spaces to vector spaces.
The special properties dictated by linear algebra may be written
S(x+y) = S(x)+S(y),
for all vectors x, y, all scalars c.
S(cx) = cS(x)
)
For technical convenience, these conditions are often expressed as a
single condition
S(cx+y) = cS(x)+S(y) for all vectors x, y, all scalars c.
AmappingS satisfyingthisconditionsiscalledalinear transformation.
Geometrically, if x and y are arbitrary vectors and c is a scalar, so
that cx+y is the far corner of the parallelogram with sides cx and y,
thenS(cx+y) = cS(x)+S(y)isthefarcorneroftheparallelogramwith
sides S(cx) = cS(x) and S(y). A linear transformation S therefore
maps an arbitrary parallelogram to a parallelogram in an obvious (and
restrictive) sense.
y S(y) S(cx+y) S(x+y)
cx+y
x+y
0
cx 0 S(cx) S(x)
x
AMS Open Math Notes: Works in Progress; Reference # OMN:201801.110759; Last Revised: 2018-01-20 09:27:31
TO THE STUDENT v
The special vectors e = (1,0) and e = (0,1) are the standard
1 2
basis of R2. Every vector in R2 can be expressed uniquely as a linear
combination:
x = (x1,x2) = (x1,0)+(0,x2)
= x1(1,0)+x2(0,1)
= x1e +x2e .
1 2
Formally, a linear transformation “distributes” over an arbitrary linear
combination. In detail, if S : R2 R2 is a linear transformation,
→
repeated application of the defining properties gives
S(x) = S(x1e +x2e )
1 2
= S(x1e )+S(x2e )
1 2
= x1S(e )+x2S(e ).
1 2
This innocuous equation expresses a remarkable conclusion: A linear
transformation S : R2 R2 is completely determined by its values
→
S(e ), S(e ) at two vectors.
1 2
Matrix Representation
To study vector spaces and linear transformations in greater detail, we
will represent vectors and linear transformations as rectangular arrays
of numbers, called matrices. The first chapter of the book introduces
matrix notation, a central piece of computational machinery in linear
algebra. Here we focus on motivation and geometric intuition, using
the special case of linear transformations from the plane to the plane.
We use the notational convention in which vectors are written as
columns:
x1
x = (x1,x2) = .
x2
(cid:20) (cid:21)
With this notation, a linear transformation S : R2 R2 is completely
j →
and uniquely specified by scalars A satisfying
i
A1 A1
S(e ) = A1e +A2e = 1 , S(e ) = A1e +A2e = 2 .
1 1 1 1 2 A2 2 2 1 2 2 A2
(cid:20) 1(cid:21) (cid:20) 2(cid:21)
The (standard) matrix of S assembles these into a rectangular array A:
A1 A1
A = S(e ) S(e ) = 1 2 .
1 2 A2 A2
(cid:20) 1 2(cid:21)
(cid:2) (cid:3)
AMS Open Math Notes: Works in Progress; Reference # OMN:201801.110759; Last Revised: 2018-01-20 09:27:31
vi LINEAR ALGEBRA
The real number Ai in the ith row and jth column of A is called the
j
(i,j)-entry, and encodes the dependence of the ith output variable on
the jth input variable.
Matrix Multiplication
For all x, we have
S(x) = x1S(e )+x2S(e )
1 2
A1 A1 A1x1 +A1x2
= x1 1 +x2 2 = 1 2 .
A2 A2 A2x1 +A2x2
(cid:20) 1(cid:21) (cid:20) 2(cid:21) (cid:20) 1 2 (cid:21)
The expression on the right may be interpreted as a type of “product”
of the matrix of S and the column vector of x:
y1 A1x1 +A1x2 A1 A1 x1
= 1 2 = 1 2 ,
y2 A2x1 +A2x2 A2 A2 x2
(cid:20) (cid:21) (cid:20) 1 2 (cid:21) (cid:20) 1 2(cid:21)(cid:20) (cid:21)
or simply y = Ax. The second equality defines the product of the
“2 2 square matrix A” and the “2 1 column matrix x”.
× ×
Particularly when the number of variables is large, sigma (summa-
tion) notation comes into its own, both condensing common expres-
sions and highlighting their structure. The relationship between the
inputs xj and the outputs yi of a linear transformation may be written
yi = Aixj.
j j ∗
P
Composition of Linear Transformations
If T : R2 R2 is a linear transformation with matrix
→
B1 B1
B = 1 2 ,
B2 B2
(cid:20) 1 2(cid:21)
the composition T S : R2 R2, defined by (T S)(x) = T S(x) , is
◦ → ◦
easily shown to be a linear transformation (check this). The associated
(cid:0) (cid:1)
∗Physicists sometimes go further, omitting the summation sign and implicitly
summing over every index that appears both as a subscript and as a superscript:
yi = Aixj. This book does not use this “Einstein summation convention”, but
j
the possibility of doing so explains our use of superscripts as indices, despite the
potential risk of reading superscripts as exponents. Exponents appear so rarely in
linear algebra that mentioning them at each occurrence is feasible.
AMS Open Math Notes: Works in Progress; Reference # OMN:201801.110759; Last Revised: 2018-01-20 09:27:31
TO THE STUDENT vii
matrix is a “product” of the matrices B and A of the transformations T
andS. Todeterminetheentriesofthisproduct, notethatbydefinition,
T(e ) = B1e +B2e , S(e ) = A1e +A2e ,
1 1 1 1 2 1 1 1 1 2
T(e ) = B1e +B2e , S(e ) = A1e +A2e .
2 2 1 2 2 2 2 1 2 2
Consequently,
TS(e ) = T(A1e +A2e )
1 1 1 1 2
= A1T(e )+A2T(e )
1 1 1 2
= A1(B1e +B2e )+A2(B1e +B2e )
1 1 1 1 2 1 2 1 2 2
= (B1A1 +B1A2)e +(B2A1 +B2A2)e ;
1 1 2 1 1 1 1 2 1 2
similarly (check this),
TS(e ) = (B1A1 +B1A2)e +(B2A1 +B2A2)e .
2 1 2 2 2 1 1 2 2 2 2
Since the coefficients of T(e ) give the first column of the matrix of TS
1
and the coefficients of T(e ) give the second column of the matrix, we
2
are led to define the matrix product by
B1 B1 A1 A1 B1A1 +B1A2 B1A1 +B1A2
BA = 1 2 1 2 = 1 1 2 1 1 2 2 2 .
B2 B2 A2 A2 B2A1 +B2A2 B2A1 +B2A2
(cid:20) 1 2(cid:21)(cid:20) 1 2(cid:21) (cid:20) 1 1 2 1 1 2 2 2(cid:21)
This forbidding collection of formulas is clarified by summation nota-
tion:
(BA)i = BiAk.
j k j
k
TheprecedingequationhaspreciselXythesameformformatricesofarbi-
trary size, and furnishes our general definition of matrix multiplication.
(Convince yourself that the entries of BA are given by the preceding
summation formula.)
When working computationally with specific matrices, the formula
is generally less important than the procedure encoded by the formula.
First, define the “product” of a “row” and a “column” by
a
b b = ba+b a .
0 a 0 0
0
(cid:20) (cid:21)
(cid:2) (cid:3) (cid:2) (cid:3)
Now, to find the (i,j)-entry of the product BA, multiply the ith row
of B by the jth column of A. For example, to find the entry in the
first row and second column of BA, multiply the first row of B by the
second column of A.
AMS Open Math Notes: Works in Progress; Reference # OMN:201801.110759; Last Revised: 2018-01-20 09:27:31
viii LINEAR ALGEBRA
Geometry of Linear Transformations
Consider the linear transformation S that rotates R2 about the origin
by π, and T that shears the plane horizontally by one unit:
6
S(e2) T(e2)
e2 S(e1)
0 e 0 0 T(e1)
1
The matrix of each may be read off the images of the standard basis
vectors. Thus,
cos π cos 2π √3 1
A = S(e1) S(e2) = sin π6 sin 23π = 21 1 √−3 ,
(cid:20) 6 3 (cid:21) (cid:20) (cid:21)
(cid:2) (cid:3)
1 1
B = T(e ) T(e ) = .
1 2
0 1
(cid:20) (cid:21)
(cid:2) (cid:3)
The composite transformations TS (rotate, then shear) and ST
(shear, then rotate) are linear, and their matrices may be found by
matrix multiplication:
√3+1 √3 1 √3 √3 1
BA = 1 − , AB = 1 − .
2 1 √3 2 1 √3+1
(cid:20) (cid:21) (cid:20) (cid:21)
Note carefully that TS = ST. Composition of linear transformations,
6
and consequently multiplication of square matrices, is generally a non-
commutative operation.
ST(e2)
TS(e2)
TS(e1) ST(e1)
0
0
AMS Open Math Notes: Works in Progress; Reference # OMN:201801.110759; Last Revised: 2018-01-20 09:27:31
TO THE STUDENT ix
Diagonalization
The so-called identity matrix
1 0
0 1
(cid:20) (cid:21)
corresponds to the identity mapping I(x) = x for all x in R2. Gener-
ally, if λ1 and λ2 are real numbers, the diagonal matrix
λ1 0
0 λ2
(cid:20) (cid:21)
corresponds to axial scaling, (x1,x2) (λ1x1,λ2x2). Diagonal matri-
7→
ces are among the simplest matrices. In particular, if n is a positive
integer, the nth power of a diagonal matrix is trivially calculated:
λ1 0 n (λ1)n 0
= .
0 λ2 0 (λ2)n
(cid:20) (cid:21) (cid:20) (cid:21)
The solution to a variety of mathematical problems rests on our
ability to compute arbitrary powers of a matrix. We are naturally led
to ask: If S : R2 R2 is a linear transformation, does there exist
→
a coordinate system in which S acts by axial scaling? This question
turns out to reduce to existence of scalars λ1 and λ2, and of non-zero
vectors v and v , such that
1 2
S(v ) = λ1v , S(v ) = λ2v .
1 1 2 2
Each λi is an eigenvalue of S; each v is an eigenvector of S. A pair of
i
non-proportional eigenvectors in the plane is an eigenbasis for S.
A linear transformation may or may not admit an eigenbasis. The
rotation S of the preceding example has no real eigenvalues at all. The
shear T has one real eigenvalue, and admits an eigenvector, but has
no eigenbasis. The compositions TS and ST both turn out to admit
eigenbases.
Structural Summary
Basic linear algebra has three parallel “levels”. In increasing order of
abstraction, they are:
(i) “The level of entries”: Column vectors and matrices written out
as arrays of numbers. (yi = Aixj.)
j j
P
AMS Open Math Notes: Works in Progress; Reference # OMN:201801.110759; Last Revised: 2018-01-20 09:27:31
x LINEAR ALGEBRA
(ii) “The level of matrices”: Column vectors and matrices written as
single entities. (y = Ax.)
(iii) “The abstract level”: Vectors (defined axiomatically) and linear
transformations(mappingsthatdistributeoverlinearcombinations).
(y = S(x).)
Linear algebra is, at heart, the study of linear combinations and
mappings that “respect” them. Along your journey through the ma-
terial, strive to detect the levels’ respective viewpoints and idioms.
Amongthemostuniversalidiomsisthis: A linear combination of linear
combinations is a linear combination.
Matrices are designed expressly to handle the bookkeeping details.
In summation notation at level (i), if
m n
zi = Biyk and yk = Akxj,
k j
k=1 j=1
X X
then substitution of the second into the first gives
m n n m n
zi = Bi Akxj = BiAk xj = (BA)ixj.
k j k j j
!
k=1 j=1 j=1 k=1 j=1
X X X X X
Once we establish properties of matrix operations, the preceding can
be distilled down to an extremely simple computation at level (ii): If
z = By and y = Ax, then z = B(Ax) = (BA)x.
Organization of the Book
The first chapter introduces real matrices as formally and quickly as
feasible. The goal is to construct machinery for flexible computation.
The book proceeds to introduce vector spaces and their properties,
two auxiliary pieces of algebraic machinery (the dot product, and the
determinant function on square matrices, each of which has a useful
geometric interpretation), linear transformations and their properties,
and diagonalization.
Of necessity, the motivation for a particular definition may not be
immediately apparent. At each stage, we are merely generalizing and
systematizing the preceding summary. It may help to review this pref-
ace periodically as you proceed through the material.
AMS Open Math Notes: Works in Progress; Reference # OMN:201801.110759; Last Revised: 2018-01-20 09:27:31