Introduction to Linear Algebra: Models, Methods, and Theory

556 Pages·068.861 MB·English

by  Alan Tucker

Stony Brook University
Academic Commons
Department of Applied Mathematics & Statistics
Department of Applied Mathematics & Statistics
Faculty Books
1995
Introduction to Linear Algebra: Models, Methods,
and Theory
Alan Tucker
SUNY Stony Brook, [email protected]
Follow this and additional works at:https://commons.library.stonybrook.edu/ams-books
Part of theAlgebra Commons
This work is licensed under aCreative Commons Attribution-Noncommercial-No Derivative Works
4.0 License.
Recommended Citation
Tucker, Alan, "Introduction to Linear Algebra: Models, Methods, and Theory" (1995).Department of Applied Mathematics & Statistics
Faculty Books. 1.
https://commons.library.stonybrook.edu/ams-books/1
This Book is brought to you for free and open access by the Department of Applied Mathematics & Statistics at Academic Commons. It has been
accepted for inclusion in Department of Applied Mathematics & Statistics Faculty Books by an authorized administrator of Academic Commons. For
more information, please [email protected].
•
.. 1 n e a r
Models, Methods, and Theory
•
•
•
A M S
•
,
2 1 0
.
-
Alan Tucker
University at Stony Brook
.
•
•
•
•
•
Introduction to Linear Algebra:
Models, Methods, and Theory
Alan Tucker
University at Stony Brook
Assisted by Donald Small
Colby College
Basix Custom Publishing
University at Stony Brook
t
•
•
Copyright © 1995, Alan Tucker
Printed in the United States of America
•
All rights reserved. No part of this book may be reproduced or transmitted in any form or by any
means, electronic or mechanical, including photocopying, recording, or any information storage
and retrieval system, without permission in writing from the publisher.
.
Alan Tucker
c/o Basix Custo1n Publishing
Rm. 046 Stony Brook Union
University at Stony Brook
Stony Brook, New York 11794
Phone: [phone]
Fax: [phone]
-
E-mail: [email protected]
•
•
It is a pl.easure and honor to
dedicate my work on this book to
my father.
A. C. T.
I would like to dedicate my efforts
on this book to my parents, hvo
great teacliers.
D. B. S.
•
•
•
•
Pre ace
•
This book attempts to give students a unified introduction to the models,
methods, and theory of modem linear algebra. Linear models are now used
at least as widely as calculus-based models. The world today is commonly
thought to consist of large, complex systems with many input and output
variables. Linear models are the primary tool for analyzing these systems.
A course based on this book ( or one like it) should prove to be the most
useful college mathematics course most students ever take. With this goal
in mind, the material is presented with an eye toward making it easy to
remember, not just for the next hour test but for a lifetime of di verse uses . .
Linear algebra is an ideal subject for a lower-level college course in
mathematics, because the theory, numerical techniques, and applications are
interwoven so beautifully. The theory of linear algebra is powerful, yet easily
accessible. Best of all, theory in linear algebra is likable. It simplifies and
clarifies the workings of linear models and related computations. This is
what mathematics is really about, making things simple and clear. It provides
important answers that go beyond results we could obtain by brute compu
tation. For too many students. mathematics is either a collection of tech
niques, as in calculus, or a collection of formal theory with limited appli
cations, as in most courses after calculus (including traditional linear algebra
courses). This book tries to rectify this artificial dichotomy.
Again, the applications of linear algebra are powerful, easily under
stood, and very diverse. This book introduces students to economic in-
. put-output models, population growth models, Markov chains', linear pro
gramming, computer graphics, regression and other statistical techniques,
••
Vil
•••
VIII Preface
•
numerical methods for approximate solutions to most calculus problen1s,
linear codes, and much more. These different applications reinforce each
other and associated theory. Indeed, without these motivating applications,
several of the more theoretical topics could not be covered in an introductory
textbook.
The field of linear numerical analysis is very young, having been de
pendent on digital computers for its development. This field has wrought
major changes in what linear algebra theory should be taught in an intro
ductory course. The standout example of such a modem linear algebra text
is G . Strang's Linear Algebra and Its Applicatiorzs. Once the theory was
needed as an alternative to numerical computation, which was hopelessly
difficult. Now theory helps direct and interpret the numerical computation,
which computers do for us.
Overview of tJ1e Text This book develops linear algebra around mat
rices. Vector spaces in the abstract are not considered, only vector spaces
associated with matrices. This book puts problem solving and an intuitive
treatment of theory first, with a proof-oriented approach intended to come
in a second course, the same way that calculus is taught.
The book's organization is straightforward: Chapter 1 has introductory
linear models; Chapter 2 has the basics of matrix algebra; Chapter 3 develops
different ways to solve a system of equations; Chapter 4 has applications,
and Chapter 5 has vector-space theory associated with matrices and related
topics such as pseudoinverses and orthogonalization. Many linear algebra
textbooks start immediately with Gaussian elimination, before any matrix
algebra. Here we first pose problems in Chapter 1, then develop a matl1e
matical language for representing and recasting the problems in Chal)ter 2,
and then look at ways to solve the problems in Chapter 3-four different
solution methods are presented with an analysis of strengths and weaknesses
of each.
In ·most applications of linear algebra, the most difficult aspect is un
u -
derstanding matrix expressions, such as Ue0 1 Students from a traditional
.
linear algebra course have little preparation for understanding such expres
sions . This book constantly forces students to interpret the meaning of matrix
expressions, not just perform rote computations. Matrix notation is used as
much as possible, rather than constantly writing out systems of equations.
The sections are generally too long to be covered completely in class; most
have several examples (based on familiar models) that are designed to be
read by students on their own without explanation by the instructor. The
goal is for students to be able to read and understand uses of matrix algebra
for themselves .
The material is unified pedagogically by the repeated use of a few
linear models to illustrate all new concepts and techniques. These models
give the student mental pictures to ' 'visualize'' new ideas during this course
and help remember the ideas after the course is over.
Although this book is often informal (''proving theorems'' by example)
and sticks mainly to matrices rather than general linear transformations, it
covers several topics normally left to a more advanced course, such as matrix
norms, matrix decompositions, and approximation by orthogonal polyno
mials. These advanced topics find immediate, concrete applications. In ad-
•
•
Preface IX
dition, they are finite-dimensional versions of important theory in functional
analysis; for example, the eigenvalue decomposition of a matrix into simple
matrices is a special case of the spectral representation of linear operators.
Discrete Versus Continuous Mathematics Today there is a major cur
riculum debate in the mathematics community between computer sci
ence-oriented discrete mathematics and classical calculus-based mathemat
ics. Linear algebra, especially as viewed in this book, is right in the middle
of this debate. (Linear algebra and matrices have always been in the middle
of such debates. Matrices were a core topic in the best-known first-year
college mathematics text before 1950, Hall and Knight's College Algebra;
and much of Kemeny, Snell, and Thompson's Introduction to Finite Math
ematics involved new applications of linear algebra: Markov chains and
linear programming.)
This book attempts to present a healthy interplay between mathematics
and computer science, that is, between continuous and discrete modes of
thinking. The complementary roles of continuous and discrete thinking are
typified by the different uses of the euclidean norm (1 -norm) and sum norm
2
(1 -norm) in this book. An important example of computer science thinking
1
in this book is matrix representations, such as the LU decomposition. They
are viewed as a way to preprocess the data in a matrix in order to be ready
to solve quickly certain types of matrix problems.
We note that computer science even gives insights into the teaching
of any linear algebra course. A computer scientist's distinction between high
level languages (such as PASCAL) and low-level languages (such as assem
bly language) applies to linear algebra proofs: A high-level proof involves
=
matrix notation, such as BTA T (AB)7 while a low-level proof involves
,
=
individual entries ay, such as cij 2JauPkj·
Suggested Course Syllabus This book contains more than can be cov
ered in the typical first-semester sophomore course for which it is intended.
Most of Chapters l, 2, and 3 and the first four sections of Chapter 5 should
normally be covered. A freshman course would skip Chapter 5. In addition,
selected sections of Chapter 4 can be chosen based on available time arid
the class's interests. For the student, the essence of any course should be
the homework. This book has a large number of exercises at all levels .of
difficulty: computational exercises, applications, and proofs of much of the
basic theory (with extensive hints for harder proofs). For more information
about course outlines, plus suggested homework sets, sample exams, and
additional solutions of exercises, see the accompanying Instructor's Manual .
At the end of the book is a list of various programming languages and
software packages available for performing matrix operations. It is recom
mended that students have access to computers with ready matrix software
in the first week.
Acknowledgments The first people to thank for help with this book are
relatives. My father, A . W. Tucker, ignited and nurtured my born-again
interest in linear algebra. Notes from the linear algebra course of my brother,
Tom Tucker at Colgate, formed the foundation of early work on this book.
My family- wife, Mandy, and daughters, Lisa and Katie-provided a sup-
•
. portive atmosphere that eased the long hours of writing; more concretely,
Lisa's calculus project on cubic splines became the appendix to Section 4. 7.
X Preface
After the first draft was finished and initial reviews indicated that much
•
rewriting was needed, the book might have died had not Don Small offered
10 help with the revision work. He added new examples, reworked several
sections, and generally made the text better.
Many people have read various versions of this book and offered help
•
ful suggestions, including Don Albers, Richard Alo, Tony Ralston, and
Margaret Wright. Brian Kohn, Tom Hagstrom, and Roger Lutzenheiser
class-tested preliminary versions of this book. Finally, thanks go to Gary
W . Ostedt, Robert F. Clark, Elaine W. Wetterau, and the staff of the edi
torial and production departments at Macmillan.
A. T .
•
•
•
•
Contents
•
Preface
Introductory Models
1
Sectio,z 1 .1 Mathematical Models
1
Section 1.2 Systems of Linear Equations 11
Section 1 .3 Markov Chains and Dynamic Models 21
Sectio,z 1 .4 Linear Programming and Models Without
Exact Solutions 37
Section 1.5 Arrays of Data and Linear Filtering 48
Matrices
61
Section 2 .1 Examples of Matrices
61
Section 2 .2 Matrix Multiplication 71
Section 2 .3 0-I Matrices 98
Section 2 .4 Matrix Algebra 112
Section 2 .5 Scalar Measures of a Matrix:
Norms and Eigenvalues 127
Section 2. 6 Efficient Matrix Computation 141
•
XI
•