APPENDIX A LINEAR MAPS AND MATRIX ANALYSIS We describe here the main concepts and ...

107 Pages·2011·5.78 MB·English

APPENDIX A
LINEAR MAPS AND MATRIX ANALYSIS
We describe here the main concepts and results useful in linear system theory.
We start with basic algebraic concepts, functions, rings, fields, linear spaces and linear
maps. Next we tackle the question of representation of linear maps, a crucial concept
in many applications. Normed linear spaces are next, with the key concept of conver
gence. The final section covers adjoints. It leads naturally to the singular value
decomposition.
AI. Preliminary Notions
We assume that the reader is familiar with the notion of sets, intersections of sets
and unions of sets; and with the symbols e",c .¢,U ,andrl; and with the logic sym
bols 'fI, 3, 3!,=>,<=, and <=>. (See, e.g. [Loo.1], [Rud.1].)
I Some sets are given standard labels: for example Z, N, R,R+, €, (J: + denote the
sets of integers, nonnegative integers, real numbers, nonnegative real numbers, com
plex numbers and complex numbers with nonnegative real part, resp.
2 Given two sets X and Y, by their cartesian product we mean the set of all ordered
pairs (x,y) where x e X and ye Y. The cartesian product of X and Y is denoted by
X x Y. Consequently the set of all ordered n-tuples of real (complex) numbers is
denoted by Rn ( €n).
3 Next the notion of function. Given two sets X and Y, by f: X -4 Y, we mean that
to every xe X, the function f assigns one and only one element f(x)e Y called value
of fat x. X and Y are resp. called the domain and codomain of f and we say that f
maps X into Y. f(X):= {f(x) I xe X} is called the range of f. The words "map,"
"operator," and "transformation" have the same meaning as function; a more complete
specification for a function is f: X -+ Y : x -+ f(x) where x -+ f(x) means that f sends
xe X into f(x)e Y. The latter can also specify f when the domain and codomain are
known; for example t -4 cost defines the function cosine: R -4 R.
4 A function f: X -4 Y : x -+ f(x) is said to be injective, or called an injection,
or (one-one), iff f(xl)=f(x2) => XI =x2, (or equivalently XI ~ x2 => f(xl) ~ f(x2».
n,
5 A function f: X -4 Y is said to be surjective, or called a surjection, (or onto
iff VYE Y, 3XE X s.t. Y = f(x), (or equivalent Y = f{X».
6 A function f: X -4 Y is said to be bijective, or called a bijection iff it is injective
and surjective, (or equivalently 'fI ye Y 3! xe X s.t. y = f (x». (Recall that 3!
means "there exists a unique.")
7 Composition. Consider functions g: X -+ Y : x -+ g(x) and f: Y -+ Z : y -+ f(y). We
404
call the function
h:=fog : X ~ Z: x ~ h(x):=f(g(x»
the composition of g and f (in that order); h is also said to be a composite function.
8 Composition may be visualized by a diagram: see Fig. A 1. To go from X to Z by
fog is the same as going through Y in two steps. first by g. then by f. We say that the
diagram commutes.
9 Composition is associative: with f and g as above and
h : W ~ X then fo(goh) = (fog)oh.
Composition leads also to the definition of inverses:
10 Consider maps f: X ~ Y with Ix resp. 1y the identity maps on X and Y. Let
gL.gR and g be maps of Y into X.
a) gL is called a left inverse of f iff gLof= Ix.
~) gR is called a right inverse of f iff fogR = 1y •
'Y) g is called a two-sided inverse or an inverse of f denoted by II iff
gof= Ix and fog = Iy .
For the latter case we say that f is invertible or II exists.
11 Fact [MacL.l p.8]. Let f: X ~ Y be invertible. Then a two-sided inverse g of f is
unique; furthermore any two inverses (left. right. two-sided) of f are equal.
12 Facts [MacL.l pp.7-8]: Let f: X ~ Y.
a) f has a left inverse iff f is injective;
~) f has a right inverse iff f is surjective;
'Y) f is invertible iff f is bijective.
[Hint: a) Since f is bijective onto f(X) c Y. one can pick gL: Y ~ X s.t .• on f(X).
gL(f(x»=x; hence gLof= Ix.
13) Because f is onto Y it is possible to choose gR: Y ~ X S.t. fogR = 1 y ....J
z
X fog ~
\1
y
Fig. AI. A commutative diagram.
405
A2. Rings and Fields
1 In engineering we frequently encounter the following
(a) Fields: R, (I: ,Q ; lR(s), (I: (s) , where in particular
2 Q := the field of rational numbers
3 lR(s), ( (I: (s» := the field of rational functions in s with coefficients in
cr,
R, ( resp.).
(b) Commutative rings:
cr
Z ,R[s], [s], ~(s)'~.o(s), R(O), R 0(0), Ru ; diagonal manices with elements in a
cr, cr
field (e.g., R, or R(s» or in a commutative ring (e.g., R[s], [s]);
where in particular
4 R[s], ( (I: [s]) := the ring of polynomials in s with coefficients in R,
( cr,
resp.).
:= the ring of propert (strictly proper) tt rational functions with
coefficients in R.
6 R(O),(R 0(0» := the subring of elements of Rp(s), (Rp.o(s», that are analytic in
(1:+ (Le. with no poles in (1:+).
7 RU := the subring of elements of Rp(s) that are analytic in U: a
closed subset of (I: symmetric w.r.t. the real axis and
which includes (I: +.
The elements of R(O) and RU are also called exponentially stable, (abbreviated expo
stable) and U-stable transfer functions, resp.
(c) Noncommutative rings:
Rnxn, (I:nxn; R[s]nxn, (I: [s]nxn; R(s)nxn, (I: (s)nxn;
•
which denote n x n manices with elements in R, (1:, etc.
8 Exercise. For each of the rings and fields above, verify that you know the opera
tions of addition and multiplication. Identify precisely the identity under addition
denoted by 0 and under multiplication denoted by I, (denoted by I in the matrix case).
In order to emphasize the similarity and differences between rings and fields we
define them jointly: thus the left column is assigned to axioms of rings and the right
column to those of fields; the axioms that are common to both are stated only once.
9 Definitions. We call ring, (field, resp.), the object consisting of a set of elements,
t Bounded at infinity.
tt (Zero at infinity).
406
two binary operations viz. addition + and multiplication • , an identity element
under addition denoted by 0, and an identity element under multiplication denoted by
1 obeying the following axioms:
Ring: (R, +, 0; 0, 1) Field: (F, +, 0; 0, 1)
(a) Addition is
Associative: (a+p}+y=a+(l3+y) Va,p,y
Commutative: a+p = l3+a V a,p
:3 identity 0: cH.()=a Va °
:3 inverse: Va, :3 element (-a) S.t. a+(-a.) =
(b) Multiplication is
Associative : (a 'P) . y= a . (P . y) V a,p,y;
Not necessarily commutative Commutative
a' ~=~'a
:3 identity l:a·l=l·a=a Va.
°
Va E R, a # 4- a-I exists VaEF,a#O
=>3 inverse a-I S.t.
a'(a-I)=(a-I)'a= 1
(c) Distributive laws: Va,p,y
a'(~+y)=a'~+a'Y
•
(p+Y)'cx=~'cx+Y'cx .
Note that we require our rings to have an identity element 1: some algebraists do not
require this and call our rings, "unitary rings," e.g. [Sig.l]. There is a standard pro
cedure to add a 1 to any ring (Jac.l].
From the axioms above four important facts follow.
10 Fact. In any ring and in any field, the identities 0 and 1 are unique.
(This is easily shown by contradiction.)
11 Fact. In a ring, the cancellation law does not necessarily hold; more precisely, in
a ring
a~=ay
and a"# 0 do not necessarily imply p=y.
Example. Consider the noncommutative ring R2x2 :
407
but clearly 13 '" y, even though a '" O.
12 Remark. The cancellation law holds in any field F because a '" 0
=> 3! a-I e F and a-I (all) = a-I (ay)
=> (a-1a)p = (a-1a)y (associativity)
=> /3=y (a-Ia= 1).
13 Remark. We know some rings for which the cancellation law holds: e.g.
Z, R[s], cr [s], lRv(s), Rv.o(s), R(O), R 0(0), RU' Such rings are called integral
domains, or better yet, entire rings.
14 Fact. 'iae R, a'O=O'a=O
Proof. a+O=a => a'(a+O)=a'a => a·a+a·O=a·a. Adding -(a'a) to both
sides gives a' 0= O. Repeat the proof but multiply by a on the right:
O+a=a => (o+a)' a=a' a ,etc., gives O' a=O. •
15 Fact. 'ia,l3eR, (-a)I3=-(a·/3)=a·(-I3).
Proof.
0= 0,/3= [a+(--a))·/3= a'!3+(-a)'/3 => -(a'/3) = (--a)'r~
•
16 Exercise. Show, from the axioms, that in any ring R,
(± (I: i: I:
ai ]. /3k ] = ai/3k . •
i=1 k=! i=! k=1
17 The ring K is called a commutative ring iff, in addition to the standard axioms
(9) we have
18 pq=qp 'ip,qe K.
19 Example. The commutative ring K might be
(1) any field: R, R(s), ... ;
(2) R[s], Rp(s), Rp,o(s), R (0), R 0(0), ... ;
(3) scalar convolution operators: p*q = q*p.
408
20 Addition and Multiplication of matrices with elements in K are defined as follows
(n denotes the sequence 1,2, ... ,n):
If Pe Krnxn and QeKrnxn then
(P+Q)ij := Pij + qij V ie ill, V je n
defines the matrix P+Q that is, in fact, in Krnxn. If Pe Krnxn and Qe KTlXp then
n
L
(PQ)ik := Pij qjk V ie 01, V ke I!
j=l
defines the matrix PQ, which is an element of KffiXP. I
21 Exercise. Show that for n > I, Knxn is a noncom mutative ring. [Hint: check that
the axioms are satisfied].
22 Fact. For matrices with elements in K, the definition and properties of deter
minants hold as in the case of elements in a field as long as one does not take
inverses! For example, if P,Q e Knxn, then det(PQ) e K and det(PQ) =det(P)·det(Q).
23 Fact [Cramer's rule]. Let PeKnxn, hence detPe K. Let Adj(P) denote as usual
the "classical adjoint", [Sigl. p.282], of P. By direct calculation we have:
24 (a) Adj(P) P=P Adj(P)=(det P)In
(b) PeKnxn has an inverse in Knxn
25 <;:;> det P has an inverse in K.
In that case,
26 p-l = Adj (P)[det(PW 1 e Knxn.
27 Comments. From (24) and (26) it follows that P has a right inverse iff it has a
left inverse; the common right and left inverse of P is called the inverse of P, (cf.
(AU 1).
Proof of (23): Outline:
(a) (24) is equivalent to, [Sig.l,p.287],
n n
L L
CkiPkj= Pikcjk=BijldetPj Vi,jell
k=l k=l
where (I) Vi=j.
409
(2) Cij is the cofactor of element Pij of P, i.e., cij=(-I)i+jmij with mij
denoting the determinant of the matrix obtained by crossing out row i
and column j of P.
(b) If P has an inverse p-i e Knxn, then by the axioms of K, det(p-i) e K; now
pp-I = ~ implies [det(P)]. [det(p-I)] = 1; hence (25) holds. Conversely if (25) holds,
then the RHS of (26) e Knxn and is the inverse of P according to (24). •
28 Note. A matrix Pe Knxn is said to be nonsingular iff detP *" 0, where 0 is the
additive identity of K. Hence if K is a field then condition (25) is equivalent to
*"
det P O. Therefore, we have the following coroJlary.
29 Corollary: Let Pe pnxn, then
Pe pnxn has an inverse in pnxn
•
~ P is nonsingular.
30 Comment: If the ring K is not a field then there may exist nonsingular matrices
P e Knxn having no inverses in Kn xn : however corollary (29) still holds for inverses
in Fnxn where the field F has K as a subring."
31 Example: Let K = R[s] be the ring of polynomials; R[s] is a subring of the field
F= R(s) of rational functions.
Hence, according to Corollary (29), P e R[s]nxn c R(s)nxn has an inverse in
R(s)nxn iff Pis nonsingular.
[~2 l~s3
To wit: let Pt(s) = ; ], PI(s)-t= [_:2 -; ].
However, according to Fact (23), Pe R[s]nxn has an inverse in R[s]nxn iff det Pis S.t.
(detp)-I e R[s], i.e. such that det P is a nonzero constant (i.e. a polynomial of order
zero): such polynomial matrices are called unimodular, (equiv. invertible in
R[s]n xn ).
[~ ~]. [~
To wit: let P2(s) = P2(s)-1 = -;].
It follows that unimodular polynomial matrices are nonsingular but the converse is not
true.
To wit: PI(s) e R[s]2x2 and P (s) e R[s]2x2 are both nonsingular but only P (s) is uni
2 2
modular: det P2(s) = I, det PI (s) = I-s3 (not a nonzero constant).
A3. Linear Spaces
Every engineer has encountered the linear spaces Rn, trn .. '. Linear spaces
are also called "vector spaces" or "linear vector spaces." Roughly speaking, a linear
space is a set of vectors say V, to which we add, a field of scalars, say F, with which
410
we multiply vectors. So we shall denote a linear space by (V,F) or by V for brevity.
Sometimes to emphasize the field F, we say the F-linear space V.
1 Definition. We call linear space (V,F) the object consisting of a set (of vectors)
Y, a field (of scalars) F and two binary operations viz. addition of vectors + and
multiplication of vectors' by scalars ., which obey the following axioms:
(a) Addition is given by
+:VxV -+ V:(x,y) -+ x+y;
Addition is
Assqciative: (x+y)+z = x+(y+z) V x,y,ze V
Commutative: x+y = y+x V x,ye V
3! identity e, (called the zero vector), S.t.
x+e=e+x=x VxeV
3! inverse: V xe Y, 3!(-x)eY S.t. x+(-x)=9;
(b) Multiplication by scalars is given by
.: FxY -+ V:(a,x) -+ ax
where V xe V V a,~e F
(ap)x = a(px)
lx=x Ox=9;
(c) Addition and multiplication by scalars are related by distributive laws viz.
V xe V, V a.~e F (a+~)x = ax+~x
•
Vx.yeV, VaeF a(x+y) = ax+ay .
There are two extremely important examples of linear spaces: for this reason we call
them canonical examples.
2 Canonical Example I. The linear space (P.F): the linear space of n-tuples in F
over the field F, with elements x = (Si );. Y= (lli); where each Si:'lie F for ie n·
Addition and scalar multiplication are defined by
x+Y:=(Si+lli)P and aX:=(uSi)P V ae F .
The most common examples are ( ern, er), (Rn,R), (R(s)n,JR(s» or ern,Rn,R(s)n for
short.
3 Exercise. Show that (P,F) is a linear space. [Hint: use the axioms of the field
411
to check the axioms in Definition (1).]
4 Canonical Example II. The function space F(D,V):
Let (V,F) be a linear space. Let D be a set. Let M be the class of all functions:
D ~ V. On M define addition and scalar multiplication by
(f+g) (d) = f(d)+g(d) 'V f,ge M 'V de D
(aO(d)= af(d) 'VaeF,'VfeM 'VdeD
Then M with these operations is a linear space over F; it is denoted by F(D,V), or
F, when D and V are understood.
5 Exercise. Using the definitions of a function and of a linear space, show that
F(D,V) is a linear space. Describe precisely what the zero vector is in this case.
(Denote it by 9F, and that in V by 9v).
Comment. D stands for domain and is an arbitrary set, e.g. N, R, Rn or a function
space. Note also that V is an arbitrary linear space.
6 Example III. The function space PC ([ro,td,Rn): it is the set of all functions map
ping the bounded interval [to,ttl into Rn which are piecewise continuous, i.e. they are
continuous everywhere, except that they may have a finite number of discontinuity
points 'tk where the one-sided limits f('tk+) and f('tk-) are well defined and finite. An
Rn-valued function defined on an infinite interval is said to be piecewise continuous
iff it is piecewise continuous on every bounded subinterval.
The prototype of a function in PC ([O,oo),R) is the square wave.
7 Example IV. The space of continuous functions mapping [to,ttl ~ Rn denoted
by C([to,t)],Rn).
8 Example V. The space of k times continuously differentiable functions mapping
[to,t)] ~ Rn denoted by Ck([to,td,Rn) or Ck for short.
9 Example VI. The space of functions f: [to,td ~ Rn that are k times
differentiable S.t. the kth derivative is piecewise continuous (whence necessarily each
function and its derivatives up to the (k-l)th are continuous).
10 Exercise. Show that examples III-VI are linear spaces.
11 Example VII. Let F= R or C:. The space of 21t-periodic functions:
[O,21t] ~ F such that
412
L00
f(t)= Ckeikl where
k=-oo
Note that if F=1R then Ck=C_k=Ckr+jcki and
L00
f(t) = co+ 2 (Cia cos (kt)-cki sin (kt»,
k=l
then "each vector" of this space is specified by the sequence (ck );.
We shall next describe the concept of subspace and product space.
14 Definition. Let (V,F) be a linear space and W a subset of V. Then ( W,F) is
called a (linear) subspace of V iff ( W,F) is itself a linear space.
From this definition it follows that
15 Examples. The set of all vectors in Rn whose first component is zero. The set
of all functions fe F (D,V) that are equal to 8y (the zero vector in V) at some fixed
point doe D or on some fixed subset Dl of D. The set of all functions f: R+ --+ R,
integrable over ~, whose Laplace transform is zero at some fixed point Zo with
Re(zo) > O.
16 Exercises. Let I be an index set and (Wi)iEI be a family of subspaces of a linear
space (V,F). Show that n Wi is a subspace of V. Give an example to show that
ieI
WI uW2 is not necessarily a subspace. Show that
WI+W2:= {Wl+w2:wjeWj ie2,} is a subspace.
["Subspaces get smaller by intersecting and bigger by adding."]
17 Definition. Let (V,F) be a linear space. We call the subspace generated by a
subset S c V the intersection of all subspaces containing S, or equivalently the smallest
subspace containing S.
18 Fact [Sig.1. p.196]. Let (V,F) be a linear space. Then the smallest subspace
generated by a subset S cV is the span of S denoted by Sp(S) viz. the set of finite
linear combinations
(lie F, sjeS Vien.
19 Definition. Let (V,F) and (W,F) be linear spaces over the same field F. The
linear space (VxW,F) is called product space of V and W: it consists of vectors