Table Of ContentLINEAR ALGEBRA
Concepts and Techniques on Euclidean Spaces
Second Edition
Ma Siu Lun
Ng Kah Loon
Victor Tan
Mc
[\
R
[Ty
Education
Linear Algebra
Concepts and Techniques on Euclidean Spaces
Second Edition
Copyright © 2016 by McGraw-Hill Education (Asia). All rights reserved. No part of this
publication may be reproduced or distributed in any form or by any means, or stored in a database
or retrieval system without the prior written permission of the publisher, including, but not
limited to, in any network or other electronic storage or transmission, or broadcast for distance
learning.
Cover image © SPI
c ) The Astan Customized Edition is a speedy publishing service for lecturers and
%mw has not undergone the full McGraw-Hill Education (Asia) editorial process.
10987654321
20 18 17 16
When ordering this title, use ISBN 978-981-4923-08-8 or MHID 981-4923-08-7
Printed in Singapore
Preface
This publication is used as the course lecture notes for the undergraduate module MA1101R,
Linear Algebra I, offered by the Department of Mathematics at the National University of
Singapore. This module is the first course on linear algebra and it serves as an introduction
to the basic concepts of linear algebra that are routinely applied in diverse fields such
as science, engineering, statistics, economics and computing. Mindful that majority of the
students taking this module are new to the subject, we have chosen to introduce the concepts
of linear algebra in the context of Euclidean spaces rather than to jump straight into abstract
vector spaces, which will be covered in the second course. The set up in Euclidean spaces
also facilitates the connections between the algebraic and geometric viewpoints of linear
algebra.
Formal proofs of most of the basic theorems in linear algebra have been included to
enhance a proper understanding of the fundamental ideas and techniques. Several applica-
tions of linear algebra in some of the fields mentioned above are also highlighted. At the
end of every chapter is a good collection of problems, all of which are culled from tutorial
problems, test and examination questions from the same module taught by the authors in
the past. These problems range from the straightforward computational ones to some highly
challenging questions. In order to achieve a deeper understanding of the topic, students are
advised to work through these problems.
There are seven chapters in this book:
In Chapter 1, we introduce systems of linear equations and discuss how to solve them
systematically. One can regard this chapter as an introduction to some important tools
needed for us to build up the theory on our main topic: Euclidean spaces.
In Chapter 2, we introduce matrices and their operations. In this book, matrices are
mainly served as tools to simplify the formulation of problems and hence provide simpler
way to solve them both theoretically and computationally.
The main topic of linear algebra is to study the algebraic structure of vector spaces.
In Chapter 3, we introduce Euclidean spaces as a generalization of the two dimensional
plane and the three dimensional space. We also study subspaces of Euclidean spaces which
provide us a way to generalize the objects of lines and planes to higher dimensional spaces.
iv Linear Algebra: Concepts and Techniques on Euclidean Spaces
After we have introduced Euclidean spaces and their subspaces, the remaining part of the
chapter concentrates on developing the concept of bases which is used to build up coordinate
systems for Euclidean spaces as well as their subspaces.
In Chapter 4, we try to relate Chapter 2 and Chapter 3 and study three vector spaces
arising from matrices, i.e. row spaces, columns spaces and nullspaces.
By introducing the concept of lengths and angles to Euclidean spaces in Chapter 5, we
have enriched the structure of the vector spaces. With the idea of orthogonality, we can
build coordinate systems with axes that are analogous to the z, y-axes of the two dimensional
plane and the z,y, z-axes of the three dimensional spaces. Also we can solve problems of
finding best approximations by using orthogonal projections.
In Chapter 6, we study the problem of reducing square matrices into diagonal forms so
that they can be computed efficiently in various applications.
Chapter 7 is an introduction to an important class of mappings called linear transforma-
tions which in abstract linear algebra, provides us tools to compare different vector spaces.
Technically, linear transformations can be defined by using standard matrices and hence
some of the properties of matrices discussed in Chapter 2, Chapter 4 and Chapter 6 can be
applied to linear transformations.
Finally, the authors would like to thank their colleagues from the Mathematics Depart-
ment in NUS who have contributed to the very first version of the lecture notes in 1998,
especially Chan Onn, Tan Hwee Huat, Roger Tan Choon Ee and Tang Wai Shing. More
recently, Toh Pee Choon and Wang Fei has also given many useful comments on areas of
improvement from the previous editions.
Contents
Chapter 1 Linear Systems and Gaussian Elimination 1
Section 1.1 Linear Systems and Their Solutions 1
Section 1.2 Elementary Row Operations 6
Section 1.3 Row-Echelon Forms 8
Section 1.4 Gaussian Elimination 11
Section 1.5 Homogeneous Linear Systems 22
Exercise 1 24
Chapter 2 Matrices 33
Section 2.1 Introduction to Matrices 33
Section 2.2 Matrix Operations 36
Section 2.3 Inverses of Square Matrices 45
Section 2.4 Elementary Matrices 48
Section 2.5 Determinants 59
Exercise 2 70
Chapter 3 Vector Spaces 84
Section 3.1 Euclidean n-Spaces 84
Section 3.2 Linear Combinations and Linear Spans 89
Section 3.3 Subspaces 98
Section 3.4 Linear Independence 102
Section 3.5 Bases 106
Section 3.6 Dimensions 111
Section 3.7 Transition Matrices 116
Exercise 3 120
Chapter 4 Vector Spaces Associated with Matrices 130
Section 4.1 Row Spaces and Column Spaces 130
Section 4.2 Ranks 139
Section 4.3 Nullspaces and Nullities 141
Exercise 4 145
Chapter 5 Orthogonality 150
Section 5.1 The Dot Product 150
Section 5.2 Orthogonal and Orthonormal Bases 153
Section 5.3 Best Approximations 160
Section 5.4 Orthogonal Matrices 165
Exercise 5 170
Chapter 6 Diagonalization 178
Section 6.1 Eigenvalues and Eigenvectors 178
Section 6.2 Diagonalization 185
Section 6.3 Orthogonal Diagonalization 192
Section 6.4 Quadratic Forms and Conic Sections 196
Exercise 6 202
Chapter 7 Linear Transformations 210
Section 7.1 Linear Transformations from R^n to R^m 210
Section 7.2 Ranges and Kernels 215
Section 7.3 Geometric Linear Transformations 219
Exercise 7 230
Index 238
Chapter 1
Linear Systems and Gaussian
Elimination
Section 1.1 Linear Systems and Their Solutions
Discussion 1.1.1 A line in the zy-plane can be represented algebraically by an equation
of the form
ar+by=c
where a and b are not both zero. An equation of this kind is known as a linear equation in
the variables of x and y. In general, we have the following definition.
Definition 1.1.2 A linear equation in n variables xy, o, ..., z, has the form
a1x1 + agxa + -+ apxy, = b
where a1, a2, ..., a, and b are real constants. The variables in a linear equation are also
called the unknowns.
We do not need to assume that ay, as, ..., a, are not all zero. If all ay, as, ..., a, and b are
zero, the equation is called a zero equation. A linear equation is called a nonzero equation
if it is not a zero equation.
Example 1.1.3
1. The equations x4+ 3y = 7, ©1 + 225 + 223 + 24 = x5, Yy = & — %z + 4.5 and
r1+x9+---+x, =1 are linear.
2 Chapter 1. Linear Systems and Gaussian Elimination
2. The equations zy = 2, sin(0) + cos(¢) = 0.2, 22 + 22+ --- +22 =1 and z = ¢¥
are not linear.
3. The linear equation azx + by + cz = d, where a, b, ¢, d are constants and a, b, ¢ are not
all zero, represents a plane in the three dimensional space. For example, z = 0 (i.e.
0z + Oy + z = 0) is the xy-plane contained inside the zyz-space.
Definition 1.1.4 Given n real numbers sy, Sa, ..., Sp, we say that 1 = sy, T2 = s9, ...,
Ty = Sp 18 a solution to a linear equation aix1 + agx2 + - -+ + apx, = b if the equation is
satisfied when we substitute the values into the equation accordingly. The set of all solutions
to the equation is called the solution set of the equation and an expression that gives us all
these solutions is called a general solution for the equation.
Example 1.1.5
1. Consider the linear equation 4x — 2y = 1. It has a general solution
=1
v 1 where t is an arbitrary parameter.
The equation also has another general solution
_ 1 1
:L'—§S+Z
where s is an arbitrary parameter.
y=s
Though the two general solutions above look different, they gives us the same set of
solutions including
r=1 rz=1.5 r=—1
y = 1.5, y = 2.5, y=—2.5,
and infinitely many other solutions.
Section 1.1. Linear Systems and Their Solutions 3
2. Consider the equation x; — 4z + 7x3 = 5. It has a general solution
xr = 5+ 4s — Tt
To =S where s, t are arbitrary parameters.
T3 = t
3. (Geometrical Interpretation)
(a) In the zy-plane, solutions to the equa-
tion z +y = 1 are points
(x,y) = (1 -5 8)
where s is any real number. These
points form a line as shown in the dia-
gram on the right.
(b) In the zyz-space, solutions to the equa-
tionz+y=1 (le.z+y+0z=1) are
points
(z,y,2) =(1—s, s, t)
where s and ¢ are any real numbers.
These points form a plane as shown in
the diagram on the right.
\//x+y:1
T
4. Consider the zero equation 0xq + 0z2 + - -+ + Ox, = 0. Any values of z1, x2, ..., x,
give us a solution. Thus the general solution is z1 = ty, 3 = ta, ..., , = t, where
ty, ta, ..., t, are arbitrary parameters.
5. For an equation Oxy + Ozg + - - - + Ox,, = b, where b is nonzero, any values of x, o,
..., , does not satisfy the equation and hence the equation has no solution.
Definition 1.1.6 A finite set of linear equations in the variables x1, z2, ..., T, is called a
system of linear equations (or a linear system):
a11x1 + apro+ -0 + a1, =by
a21%1 + agex2+ -+ + ap®n, = by
Am1T1 + Am2Z2 + -+ + GmnTn = by,
4 Chapter 1. Lincar Systems and Gaussian Elimination
where a1t a1z, amn and by, by, ..., by, ave real constants. If all anr,ar..a. ,a.mn and
by, ba, ..., by are zero, the system is called a zero system. A linear system is called a
nonzero system if it is not a zero system.
Given n real numbers s, s, ..., su, we say that o1 = 1, 22 = Sa, ..., Tn = sn is &
solution to the system if z;, = sy, o5 = s3, ..., T, = s, is a solution to every equation in
the system. The set of all solutions to the system is called the solution set of the system and
an expression that gives us all these solutions is called & general solution for the system.
Example 1.1.7 Consider the system of linear equations
21 =122 =2 23 = —1 is a solution to the system and 1 = 1, 22 = 8, z3 = 1 is not a
solution to the system.
Remark 1.1.8 Not all systems of linear equations have solutions. For example, the follow-
ing system has no solution as it is impossible to have a solution that satisfies both equations
simultancously. { .
T+ y
20+2y 6.
Definition 1.1.9 A system of linear equations that has no solution is said to be inconsis-
tent. A system that has at least one solution is called consistent.
Remark 1.1.10 Every system of linear equations has cither no solution, only one solution,
or infinitely many solutions. (See Question 2.22.)
Discussion 1.1.11
1. In the zy-plane, the two equations in the system
{ wmziby=a (L)
axtby=cs, (L)
where ax, by are not both zero and az, by are not both zero, represent two straight
lines. A solution to the system is a point of intersection of the two lines.
(a) The system has no solution if and only if L; and Ly are different but parallel
lines.
(b) The system has only one solution if and only if Ly and Ly are not parallel lines.
(¢) The system has infinitely many solutions if and only if Ly and Ly are the same
line.
