Linear Algebra With Applications

303 Pages·1997·59.601 MB·English

by  Otto Bretscher

LINEAR
ALGEBRA
WITH APPLICATIONS
Otto Bretscher
Harvard University
PRENTICE HALL, Upper Saddle River, New Jersey 07458
CONTENTS
Library of Congress Cataloging-in-Publication Data
Bretscher. Otto.
Linear algebra with applications I Otto Bretscher.
Preface zx
p. cm.
lncludes index.
ISBN: 0-13-190729-8
1. Algebras. Linear. I. Title.
1
QAI84.B73 1996
94-36942 Linear Equations 1
512' .5-dc21
CJP
1.1 Introduction to Linear Systems
Acquisitions Editor: GEORGE LOBELL
1.2 Matrices and Gauss-Jordan Elimination 12
Editorial Assistant: GALE EPPS
Assistant Editor: AUDRA J, WALSH 1.3 On the Solutions of Linear Systems 33
Editorial Director: TIM BOZIK
Editor-in-Chief: JEROME GRANT RICCARDI .
Assistant Vice President of Production and Manufacturing: DAVID W. 2
Editoriai/Production Supervision: RICBARD DeLORENZO Linear Transformations 52
Managing Editor: LINDA MIHATOV BEHRENS
Executive Managing Editor: KATHLEEN SCHIAPARELLI 2.1 Introduction to Linear Transformations and Their
Manufacturing Buyer: ALAN FISCHER Inverses 52
Manufacturing Manager: TRUDY PISCIOTTI
2.2 Linear Transformations in Geometry 65
Director of Marketing: JOHN TWEEDDALE
Marketing Assistant: DIANA PENHA 2.3 The Inverse of a Linear Transformation 86
Creative Director: PAULA MAYLAHN
2.4 Matrix Products 97
ArtDirector: MAUREEN EIDE
Cover and Interior Design/Layout: MAUREEN EIDE
3
Cover Image: nromcrown Chapel. copyrighl by R. Greg Hurlsey; Eureka Springs, Arkansas. v Subspaces of IRn and Their Dimensions 128
© 1997 by Prentice-Hall, lnc. 3.1 Image and Kerne) of a Linear Transformation 128
Sirnon & Schuster I A Viacom Company
3.2 Subspaces of !Rn; Bases and Linear Independence 146
Upper Saddle River, NJ 07458
3.3 The Dimension of a Subspace of Rn 160
All rights reserved. No part of this book may be
reproduced, in any form or by any means,
without permission in writing from the publisher. 4
Orthogonality and Least Squares 177
Printed in the United States of America.
4.1 Orthonormal Bases and Orthogonal Projections 177
10987654321
4.2 Gram-Schrnidt Process and QR Factorization 198
4.3 Orthogonal Transformations and Orthogonal Matrices 207
ISBN 0-13-190729-8 4.4 Least Squaresand Data Fitting 218
s
Prentice-Hall International (UK) Limited, London Determinants 239
Prentice-Hall of Australia Pry. Limited, Sydney
Prentice-Hall Canada Inc., Toronto 5.1 Introduction to Deterrninants 239
Prentice-Hall Hispanoamericana. S.A., Mexico
Prentice-Hall of India Private Limited, New Delhi 5.2 Properties of the Deterrninant 250
Prentice-Hall of Japan, lnc., Tokyo 5.3 Geometrical Interpretations of the Deterrninant; Cramer's
Sirnon & Schuster Asia Pte. Ltd., Singapore
Ru1e 272
Editors Prentice-Hall do Brasil. Ltda., Rio de Janeiro
Vi •
Contents
/ 6
Eigenvalues and Eigenvectors 291
6.1 Dynamical Systems and Eigenvectors: An Introductory
Example 291
6.2 Finding t11e Eigenvalues of a Matrix 307
6.3 Findi ng the Eigenvectors of a Matrix 321 To my parents
6.4 Complex Eigenvalues 341
and Margrit Bretscher-Zwicky
O~to
6.5 Stability 357
wzth love and gratitude
1
Coordinate Systems 371
7 .l Coordinate Systems in IR" 371
7.2 Diagonalization and Similarity 387
7.3 Symmetrie Matrices 401
7.4 Quadratic Foro1S 4 J3
r 7.5 Singular Values 425
/ 8
Linear Systems of Differential Equations 438
8.1 An Introduction to Continuous Dynamical Systems 438
8.2 The Complex Case: Euler's Forrnula 458
/ 9
Linear Spaces 4 77
9.1 An Introduction to Linear Spaces 477
9.2 Coordinates in a Linear Space 496
9.3 Inner Product Spaces 511
9.4 Linear Differential Operators 528
APPENDIX A Vectors 546
Answers to Odd-Numbered Exercises A-1
Index I-1
PREFACE
Key Features
. d ·I on in the text to make the di cu.-
• Linear rransformario~IS are mtroduce. :~~ fand easier to visualize. Mappi ng,
ion of matrix operauon more meanll1", u ' . .
and transf rmati n be ome a theme in the text thereaftet. . . .
. I . are used as a untfytng them ' a
• Discrere and conrinuou dynanuca ·' .relll I" . f linear aJoebra.
m tiYati n ~ r eiQ.enve tors. and a maJor app tcatton o "' .
A police offleer on patrol at rnidnight, o runs an old joke, notice a man
.II 1:ind an abundan f rhoughr-provoking (and occa. tonally
• The rea d er Wl crawling about on his hand and knee under a treetlamp. He walk over to
deli2.htful) problem and exerci e · investigate, whereupon the man explain in a tired and omewhat lurred oice
- . d d d all throuohout the text. The major
that he has lost hi hou ekeys. The policeman offers to help, and for the n xt five
• Ab tracr concepr are mtro u e ~a Uc I f "' ·ality before the stuclent
idea are arefullv d Yeloped at \·anou le e o genet minutes he too i searching on bis hands and knees. At last he exclairn .. "Are
i intr duced t ab tra t ve tor pace · you absolutely certain that this is where you droppecl the keys?"
.I·cal internretation are emphasized exten ively "Here? Absolutely not. I dropped them a block down, in the middle of the
• \ isuali-aiion alld eollleri r
street."
throu.::hout.
"Then why the devil have you got me hunting around this lamppo tT
'Because thi i where the light is."
Hornepage
It is mathematics, and not just (as Bismarck claimed) politics, that con i in ·'the
art of the pos ible." Rather than search in the clarkness for so\ution to problem
of pressing interest, we contrive a realm of problems whose intere t Jie · above all
in the fact that solution can conceivably be found.
Perhaps the lm·gest patch of light surrounds the techniques of matrix arith
metic and algebra, ancl in particttlar matrix multiplication and row reduction. Here
we might begin with De ca11es, since it wa he who discovered the conceptual
meeting-point of geometry ancl algebra in the identification of Euclidean pace
with ]!(3; the technique and application proliferated ince hi day. To organize
and clarify tho e i the role of a modern linear algebra course.
+ Computersand Computation
An essential issue that needs to be addressed in establishing a mathematical
methodology is the roJe of computation and of computing technology. Are the
proper subjects of mathematics algorithm and calculation . or are they grand the
orie and abstractions which evade the need for computation? lf the former. i
it important that the studcnt leam to carry out the computations \ ith pencil and
paper, or hould the algorithm "press the calculator· x- 1 button'· be all ' ed to
substitute for the tnditional method of finding an in er. e? lf the lauer. . hould
the abstraction be taught through elaborate notational mechani m . or rhrough
computational examples and graphs?
We seek to take a con istenr approach to these question : algorithms and
computations cu·e primary, and precisely for thi · reason computer are not. Again
and again we examine the nitty-gritty of row recluction or matrix multipli ation in
order ro derive ne\ in ·ight . Most of the proofs. wh ther of rank-nullity the rem.
the volume-chang formula for determinant , r the pectral theorem for ·ymmetri
matrices. are in thi, way tied to hand -on procedure ..
ix
Preface • Xi
X • Preface
The aim is not just to know how to compute the solution to a prob lern, b~t t:o The examples make up a significant portion of the text; we have kept abstract
imagine the computations. The student needs to perform enoug.h,;ow reductl~ns exposition to a minimum. It i a matter of ta ·te whether general theorie . . hould
by band to be equipped to follow a Jine of argument of the form. .If w~ calcu ate give 1ise to specific examples, or be pasted tagether from them. In a text uch a
. ' and to appreciate m advance this one, attempting to keep an eye on applications, the latter i. clearly preferable:
the reduced row echelon form o f such a mat nx · · · , . .
the examples alway. precede the theorems in this book.
what the possible outcomes to a particular com.putatton are. . .
In applications the solution to a problern IS hardly ~ore.J~portant than .rec- Scattered throughout the mathematical exposition are quite a few name and
. · · f aJ'd'ty and appreciating how sensitive It IS to perturbatwns dates, some historical accounts, and anecdotes as weil. Students of mathematics
oomzmo Its range o v 1 1 , .
of the i~put. We emphasize the geometric and qualitative nature of ~e sol~ttons, are too rarely shown tbat the seemingly strange and arbitrary concept they tudy
notions. of approximation, stability, and "typicaJ" matrices. The dJscu~swn of are the results of long and hard struggles. It will encourage the readers to know
Cramer' s rule, for instance, underscores the value of closed-form solutJO~s. ~or that a mere two centmies ago some of the most brilliant mathematician were
wrestling with problems such as the meaning of dimension or the interpretation
visualizing a system's behavior, and understanding its dependence from Imtlal
of e", and to realize that the advance of time and understanding actually enables
conditions. .
them, with some effort of their own, to ee farther than those great mind .
The availability of computers is, however, neither to be Ignored n~r r~-
gretted. Each student and instructor will h.ave to dec~de how much p~actice IS
needed to be sufficiently farniliar with the mner workings of the a!gonthm .. As + Outline ofthe Text
the explicit computations are being replaced gradu~lly b~ a theorettcal overview
Chapter 1. This chapter provides a careful introduction to the olution
of how the algorithm works, the burden of calculatton will be t~en up by te~h­
of systems of linear equations by Gauss-Jordan elimination. Once the concrete
nology, particularly for those wishing to carry out the more numencal and applied
problern is solved, we restate it in terms of matrix formalism, and di cu the
exercises.
geometric properlies of the solutions.
It is possible to turn your Linear algebra course into a more computer orie~ted
Chapter 2. Here we raise the abstraction a notch and reinterpret matrice
or enhanced course by wrapping with this text either ATLAST Computer Exerctses
as linear transformations. The reader i introduced to the modern notion of a
for Linear Algebra (1997, edited by S. Leon, E. Herman, and ~· Faulkenb.erry) ?r
function, as an arbitrary a ociation between an input and an output, which Iead
Linear Algebra Labs with Matlab, 2nd edition ( 1996 by D. Hill and D. Zitar.elh).
into a discussion of inverses. The traditional method for finding the inverse of
Each of these supplements goes beyond just using the computer for computatwnal
a matrix is explained: it fits in naturally as a sort of automated algorithm for
matters. Each takes the standard topics in linear algebra and finds a method
Gauss-Jordan elimination.
of illuminating key ideas visually with the computer. Thus both have M-files
We define linear transformations primarily in tem1 of matlice . ince that
available that can be delivered by the Internet.
is how they are used; the abstract concept of linearity is presented as an auxiliary
notion. Rotation in IR2 are empha ized, both as archetypal, easily-vi ualized
examples, and as preparation for future applications.
+ Examples, Exerdses, Applications, and History
Chapter 3. We introduce subspaces. images and kerne! , linear indepen
dence, and base , in the context of IR". This allows a thorough discu ion of the
The exercises and examples are the heart of this book. Our objective is not just to
central concepts, without requiring a digres ion at thi early stage into the Jippery
show our readers a "patch of light" where questions may be posed and solved, but
issue of coorclinate systems.
to convince them that there is indeed a great deal of useful, interesting material
to be found in this area, if they take the time to Iook around. Consequently, we Chapter 4. This chapter includes some of the mo t basic application •. We
introduce orthonormal ba e and the Gram-Schmidt proce s. along with the QR
have included genuine applications of the ideas and methods under discussion to a
factorization. The calculation of correlation coefficients i di cussed, and the
broad range of sciences: physics, chernistry, biology, econornics, and, of course,
important technique of lea t-squares approximations i explained. in a number of
mathematics itself. Often we have simplified them to sharpen the point, but they
different context .
use the methods and models of contemporary scientists.
With such a !arge and varied set of exercises in eacb section, instructors Chapter 5. Our di cussion of determinants is algorithmic. based on the
counting of '·patterns" (a transparent way to deal with pernmtation ). We derive
should have little difficulty in designing a course that is suited to their aims and
the properlies of the deterrninant from careful analysis of thi procedure. and tie
to the needs of their students. Quite a few Straightforward computation problems
it Iogether with Gauss-Jordan elimination. The goal i to prepare for the main
are offered, of course. Simple (and, in a few cases, not so simple) proofs and
application of determinant. : the computation of characteri tic polynomiaL.
derivations are required in some exercises. In many cases, theoretical principles
Chapter 6. This chapter introduces the central application of the latter half
that are discussed at Jength in more abstract linear algebra courses are here found
of the text: linear dynamical systems. We begin with di crete stems. and arc
broken up in bite-size exercises.
Xiii
Preface •
xii •
Preface
naturally led to seek eigenvectors, which characterize the long-t~r~ behav_i~r of I have received valuable feedback from the book's revt.e wers:
the system. Qualitative behavior is emphasized, particularl_y s~ab!ltty ~ondi_t.Ion .
Camplex eigenvalues are explained, without apology, and ued mto earller dLscus- Frank B~atro~s, University of Pittsburgh
Tracy Bibelnieks, University of Minnesota
sions of two-dimensional rotation matrices.
Cbapter 7. Having introduced the change of coordinates to an eigenbasis Jeff 0· Farmer, University of Nortbern Colorado
in Chapter 6, we now explore the "general" theory of coordin~te s~stem~ (still K~nrad J. Heuvers, Michigan Technological University
finnly fixed in ~~~, however). This Ieads to a more com~rehens!ve d1scusst~n of Mic?ael ~allaber, Washington State University
diagonalization, and to the spectral theorem for symmetrtc matnces. These tdeas Dame! King, Oberlin College
are applied to quadratic forms, and conic sections are explained. Richard Kubelka, San Jose State University
Chapter 8. Here we apply the methods developed for discrete dynamical Peter C. Patton: University of Pennsylvania
systems to continuous ones, that is, to systems of first-order linear differential Jeff~ey M. Rab~n, Uni:ersity of California, San Diego
equations. Again, the cases of real and complex eigenvalues are discussed. Dan~el B. _Shapiro, Ohio State University
Chapter 9. The book ends where many texts begin: with the theory of David Stemsaltz, Technische Universität Berlin
abstract vector spaces (which are here called "linear spaces," to prevent the con
fusion that some students experience with the term "vector"). The chapter begins and RIi eabl sod tDha nLk my editor ' Ge o.rge. L o b e 11 ' •.o r hi.s encouragement and advice
with assembling the underlying principles of the many examples that have come ar e orenzo for coordmat.Ing book production. ,
before; from this lush growth the general theory then falls like ripe fruit. Once the struct~~:l d~:~~~t:,:n~~!dth~; :!~~ ~~~g~~pported
by a grant from the In-
reader has been convinced that there is nothing new here, the book concludes with
important applications tbat are fundamentally new, in particular Fourier analysis.
(with David Steinsaltz) Otto Bretscher
Department of Matbematics
Harvard University
Cambridge, MA 02138 USA
e-mail: [email protected]
+ Acknowledgments
I first thank my students, colleagues, and assistants at Harvard University for the
key role they have played in developing this text out of a series of rough lecture
notes. The following professors, who have taught the course with me, have made
invaluable contributions:
Persi Diaconis
Edward Frenkel
David Kazhdan
Barry Mazur
David Murnford
Shlomo Sternberg
I owe special thanks to William Calder and Robert Kaplan for their thoughtful
review of the manuscript.
I wish to thank Sylvie Bessette for the careful preparation of the manuscript
and Paul Nguyen for bis well-drawn figures.
I am gratefu~ to t~ose who _have contributed to the book in many ways:
Me_noo Cung,. SrdJan I?tvac, Robm Gottlieb, Luke Hunsberger, Bridge! Neale,
Akilesh Palantsamy, Rita Pang, Esther Silberstein, Radhika de Silva, Jonathan
Tannenhauser, Selina Tinsley, and Larry Wilson.
. ---- ·- - - - -.
LINEAR EQUATIONS
INTRODUCTION TO LINEAR SYSTEMS
Traditionally algebra was the art of solving equations or systems of equation
The word algebra comes from tbe Arabic al-jabr, wbich means reduction. Tbe
term was first used in a mathematicaJ sense by Mohammed al-Khowarizmi, who
lived about A.D. 800 in Baghdad. Linear algebra, tben, is tbe art of solving systems
of linear equations.
The need to solve systems of linear equations frequently arises in mathemat
ics, statistics, physics, astronomy, engineering, computer science, and economics.
Solving systems of linear equations is not conceptually difficult. For small
systems ad hoc methods certainly suffice. Larger systems, however, require more
systematic methods. The approach generally used today was beautifully explained
2000 years ago in a Chinese text, the "Nine Chapters on the MathematicaJ Art"
(Chiu-chang Suan-shu), which contains the following example.1
Tbe yield of one sheaf of inferior grain, two sheaves of medium grain,
and three sheaves of superior grain is 39 tou.2 The yield of one sheaf of
inferior grain, three sheaves of medium grain, and two sheave of superior
grain is 34 tou. The yield of three sheaves of inferior grain, two sbeaves of
medium grain, and one sbeaf of superior grain is 26 tou. What i the yield
of inferior medium, and superior grain?
In this problem the unknown quantitie are the yield of one sheaf of infe
rior, one sheaf of medium, and one sheaf of superior grain. Let us denote these
18. L. v.d.Waerden: Geometry and Algebra in Ancient Civilizariom. Springer-Verlag. Berlin, 1983.
1 A 1011 is a bronze bowl used as a food container during the middle and late Chou dynasty (c. 900-
255 B.C.).
1
3
Sec. 1.1 In troduction to Linear y tems •
2 •
hap. 1 Linear Equations
. Tl e probl m can th 11 be repre ented by Finally, we eliminate the variable z off the diagonal.
1
quantitie by x, v. and ;::,_ re pecuve_y . . 1
the following system of linear equauons.
x +5z = 49 - 5 x Ia t equation X = 2.75
r ++2 r ++ 3; :: = 39 y - z = - 5 + last equation y = 4.25
=
:r 3:r 2;:: 34 z = 9.25 - = 9.25
~
3x + 2y + .:: = 26
The yields of inferior, medium, and superior grain are 2.75, 4.25, and 9.25 tou
To olve for .r. y. and -, we need to transform this sy t m from the form
per sheaf, re pectively.
X+ 2_,, + 3;:: = 39 X -= ...... . By ubstituting these value , we can check that x = 2.75, y = 4.25, z = 9.25
y
x + 3v + -?. = 34 into the form is indeed t11e solution of the ystem.
3x + -·V + z = 26 z = ...
· . . 1 t rm that are off the diagonal. those 2.75 + 2 X 4.25 + 3 X 9.25 = 39
ln othe~ words. we. need to eltnundate ~~~ t1~e coefficient of the variables along
2.75 + 3 X 4.25 + 2 X 9.25 = 34
circled ll1 the equatwn below. an ma
the diagonal equal to l. 3 X 2.75 +2 X 4.25 + 9.25 = 26
X+ @+®
=39
Happily. in linear algebra you are almost always able to check your olutions.
G + 3y + ® = 34 It will help you if you get into the habit of checking now.
® +CB +
z = 26 .
We can accompli h the e goals step by step, one varia_ble at a t~me. ln the + Geometrie Interpretation
. l'fied y tems of equations by addtng equauons to one
past you may I1 ave nnp t . . bl f
another or subtracting them. In thi y tem we c~n eliminate the vana e .r rom How can we interpret this result geometrically? Each of the t11ree equations
the econd equati.on by ubtracting the first equauon from the second. of the system defines a plane in x-y-z space. The sohttion et of the ys
tem consists of those points (x. y, z) which lie in all three plane , that is, the
x + ?- Y + 3 Z-- 39 ~ x + 2y + 3z = 39 intersection of the three planes. Algebraically speaking, the solution set con
x + 3y + 2- = 34 - 1 t equation Y- 7 = - 5
sists of those ordered triples of numbers (x, y, 7) which ati fy all three equa
3x + 2y + - = 26 3x + 2y + z = 26
tion simultaneously. Our computations show that the ystem ha only one so
To eliminate the variable x from the third eq~ation, we subtract_t he first eql~ation lution. (x, y, z) = (2.75. 4.25, 9.25). This means that the plane defined by the
from the tbird equation three times. We multtply the first equatwn by 3 to oet three equations intersect at the point (x, y. -) = (2.75. 4.25. 9.25), a hown in
= Figure 1.
3x + 6y + 9- 117 (3 x Ist equation).
and then ·ubrract thi result from the third equation.
x + 2y + 3z = 39 ~ x + 2yy +- 3zz == -395 Figure 1 Three planes in space,
V- Z = - 5 intersecfing at a point.
3x + 2y + z = 26 - 3 x I t equation - 4y - 8z = - 91
Similarly, we eliminate the variable v off the diagonal.
x + 2y + 3z = 39 -2 x 2nd equation .. r + 5z = 49
y - z = - 5 y - z = - 5
- 4y - 8z = -91 +4 x 2nd equation - 12z = - I II
ßefore we eliminate the variable z off the diagonal, we makc the coefficient of ;::
the diagonal equal to I, by dividing the last equation by - 12.
011
x + 5z = 49 x + 5z = 49
y - z = - 5 ~ y- ;:: = - 5
- 12z= - 111 -:- (-12) z = 9.25
5
Sec. 1.1 Introduction to Linear Systems •
4 •
Chap. 1 Linear Equations
2x+4y+6z= 0 -:-2 x + 2y + 3z = 0 ---;
4x + Sy + 6z = 3 4x+ 5y+ 6z=3 - 4(I)
7x+8y+ 9z = 6 7x +8y +9z=6 - 7 I)
x + 2y + 3z = 0 x +2y + 3z = 0 -2(II)
- 3y- 6z = 3 -:- ( -3) y+ 2z = - 1 ~
- 6y - 12z = 6 - 6 y - 12z = 6 +6(U)
X - Z = 2 21
y + 2z = - L x - z=
0= 0 l y + 2z = - 1
After omitting the trivial equation 0 = 0, we have only two equations with
three unlmowns. The solution set is the intersection of two nonparallel planes in
Figure 2a Three planes having a space, that is, a line. This sy tem ha infinitely many solutions.
line in common. The two equations above can be written as follows:
x = z+ 21
y = - 2z- 1 ·
l
We see that both x and y are determined by ~- We can freely choose a value of
z, an arbitrary real number· then the two equations above give us the values of x
and y for this choice of z. For example:
• Choose z = l. Then x = z + 2 = 3 and y = -2~ - l = -3. The solution i
(x y '") = (3, - 3, l).
• Choose z = 7. Then x = z + 2 = 9 and y = - 2z- 1 = -15. The solution
Flglll'e 2b Three planes with no i (x. y, z) = (9 - 15. 7).
common intersedion.
More generally, if we choose z = t. an arbitrary real number. we get x =
t + 2 and y = - 2t - 1. Therefore the general solution is
While three planes in space usually intersect at a poi?t, they _may have a
li in common (see Figure 2a) or may not have a common tntersectwn at all, as (x, y. z) = (t + 2. - 2t - I, t) = (2, - l. 0) + t(l, - 2, l).
s;~wn t~ree
in Figure 2b. Therefore, a system of three _equations with unknowns
may have a unique solution, infinitely many solutmns, or no solutwns at all. Tbi equation represent a line in space, a shown in Figure 3.
+
A System with Infinitely Many Solutions
Figure 3 The line (x, y, z) =
(t+ 2, - 2t- 1, t).
Next, let's consider a system of linear equations that has infinitely many solutions:
2x +4y + 6z = 0
4x + Sy + 6z = 3
7x + 8y + 9z = 6
We can solve this system using elimination as discussed above. For sim
plicity, we Iabel the equations with Roman numerals.