APPENDIX A LINEAR MAPS AND MATRIX ANALYSIS We describe here the main concepts and results useful in linear system theory. We start with basic algebraic concepts, functions, rings, fields, linear spaces and linear maps. Next we tackle the question of representation of linear maps, a crucial concept in many applications. Normed linear spaces are next, with the key concept of conver gence. The final section covers adjoints. It leads naturally to the singular value decomposition. AI. Preliminary Notions We assume that the reader is familiar with the notion of sets, intersections of sets and unions of sets; and with the symbols e",c .¢,U ,andrl; and with the logic sym bols 'fI, 3, 3!,=>,<=, and <=>. (See, e.g. [Loo.1], [Rud.1].) I Some sets are given standard labels: for example Z, N, R,R+, €, (J: + denote the sets of integers, nonnegative integers, real numbers, nonnegative real numbers, com plex numbers and complex numbers with nonnegative real part, resp. 2 Given two sets X and Y, by their cartesian product we mean the set of all ordered pairs (x,y) where x e X and ye Y. The cartesian product of X and Y is denoted by X x Y. Consequently the set of all ordered n-tuples of real (complex) numbers is denoted by Rn ( €n). 3 Next the notion of function. Given two sets X and Y, by f: X -4 Y, we mean that to every xe X, the function f assigns one and only one element f(x)e Y called value of fat x. X and Y are resp. called the domain and codomain of f and we say that f maps X into Y. f(X):= {f(x) I xe X} is called the range of f. The words "map," "operator," and "transformation" have the same meaning as function; a more complete specification for a function is f: X -+ Y : x -+ f(x) where x -+ f(x) means that f sends xe X into f(x)e Y. The latter can also specify f when the domain and codomain are known; for example t -4 cost defines the function cosine: R -4 R. 4 A function f: X -4 Y : x -+ f(x) is said to be injective, or called an injection, or (one-one), iff f(xl)=f(x2) => XI =x2, (or equivalently XI ~ x2 => f(xl) ~ f(x2». n, 5 A function f: X -4 Y is said to be surjective, or called a surjection, (or onto iff VYE Y, 3XE X s.t. Y = f(x), (or equivalent Y = f{X». 6 A function f: X -4 Y is said to be bijective, or called a bijection iff it is injective and surjective, (or equivalently 'fI ye Y 3! xe X s.t. y = f (x». (Recall that 3! means "there exists a unique.") 7 Composition. Consider functions g: X -+ Y : x -+ g(x) and f: Y -+ Z : y -+ f(y). We 404 call the function h:=fog : X ~ Z: x ~ h(x):=f(g(x» the composition of g and f (in that order); h is also said to be a composite function. 8 Composition may be visualized by a diagram: see Fig. A 1. To go from X to Z by fog is the same as going through Y in two steps. first by g. then by f. We say that the diagram commutes. 9 Composition is associative: with f and g as above and h : W ~ X then fo(goh) = (fog)oh. Composition leads also to the definition of inverses: 10 Consider maps f: X ~ Y with Ix resp. 1y the identity maps on X and Y. Let gL.gR and g be maps of Y into X. a) gL is called a left inverse of f iff gLof= Ix. ~) gR is called a right inverse of f iff fogR = 1y • 'Y) g is called a two-sided inverse or an inverse of f denoted by II iff gof= Ix and fog = Iy . For the latter case we say that f is invertible or II exists. 11 Fact [MacL.l p.8]. Let f: X ~ Y be invertible. Then a two-sided inverse g of f is unique; furthermore any two inverses (left. right. two-sided) of f are equal. 12 Facts [MacL.l pp.7-8]: Let f: X ~ Y. a) f has a left inverse iff f is injective; ~) f has a right inverse iff f is surjective; 'Y) f is invertible iff f is bijective. [Hint: a) Since f is bijective onto f(X) c Y. one can pick gL: Y ~ X s.t .• on f(X). gL(f(x»=x; hence gLof= Ix. 13) Because f is onto Y it is possible to choose gR: Y ~ X S.t. fogR = 1 y ....J z X fog ~ \1 y Fig. AI. A commutative diagram. 405 A2. Rings and Fields 1 In engineering we frequently encounter the following (a) Fields: R, (I: ,Q ; lR(s), (I: (s) , where in particular 2 Q := the field of rational numbers 3 lR(s), ( (I: (s» := the field of rational functions in s with coefficients in cr, R, ( resp.). (b) Commutative rings: cr Z ,R[s], [s], ~(s)'~.o(s), R(O), R 0(0), Ru ; diagonal manices with elements in a cr, cr field (e.g., R, or R(s» or in a commutative ring (e.g., R[s], [s]); where in particular 4 R[s], ( (I: [s]) := the ring of polynomials in s with coefficients in R, ( cr, resp.). := the ring of propert (strictly proper) tt rational functions with coefficients in R. 6 R(O),(R 0(0» := the subring of elements of Rp(s), (Rp.o(s», that are analytic in (1:+ (Le. with no poles in (1:+). 7 RU := the subring of elements of Rp(s) that are analytic in U: a closed subset of (I: symmetric w.r.t. the real axis and which includes (I: +. The elements of R(O) and RU are also called exponentially stable, (abbreviated expo stable) and U-stable transfer functions, resp. (c) Noncommutative rings: Rnxn, (I:nxn; R[s]nxn, (I: [s]nxn; R(s)nxn, (I: (s)nxn; • which denote n x n manices with elements in R, (1:, etc. 8 Exercise. For each of the rings and fields above, verify that you know the opera tions of addition and multiplication. Identify precisely the identity under addition denoted by 0 and under multiplication denoted by I, (denoted by I in the matrix case). In order to emphasize the similarity and differences between rings and fields we define them jointly: thus the left column is assigned to axioms of rings and the right column to those of fields; the axioms that are common to both are stated only once. 9 Definitions. We call ring, (field, resp.), the object consisting of a set of elements, t Bounded at infinity. tt (Zero at infinity). 406 two binary operations viz. addition + and multiplication • , an identity element under addition denoted by 0, and an identity element under multiplication denoted by 1 obeying the following axioms: Ring: (R, +, 0; 0, 1) Field: (F, +, 0; 0, 1) (a) Addition is Associative: (a+p}+y=a+(l3+y) Va,p,y Commutative: a+p = l3+a V a,p :3 identity 0: cH.()=a Va ° :3 inverse: Va, :3 element (-a) S.t. a+(-a.) = (b) Multiplication is Associative : (a 'P) . y= a . (P . y) V a,p,y; Not necessarily commutative Commutative a' ~=~'a :3 identity l:a·l=l·a=a Va. ° Va E R, a # 4- a-I exists VaEF,a#O =>3 inverse a-I S.t. a'(a-I)=(a-I)'a= 1 (c) Distributive laws: Va,p,y a'(~+y)=a'~+a'Y • (p+Y)'cx=~'cx+Y'cx . Note that we require our rings to have an identity element 1: some algebraists do not require this and call our rings, "unitary rings," e.g. [Sig.l]. There is a standard pro cedure to add a 1 to any ring (Jac.l]. From the axioms above four important facts follow. 10 Fact. In any ring and in any field, the identities 0 and 1 are unique. (This is easily shown by contradiction.) 11 Fact. In a ring, the cancellation law does not necessarily hold; more precisely, in a ring a~=ay and a"# 0 do not necessarily imply p=y. Example. Consider the noncommutative ring R2x2 : 407 but clearly 13 '" y, even though a '" O. 12 Remark. The cancellation law holds in any field F because a '" 0 => 3! a-I e F and a-I (all) = a-I (ay) => (a-1a)p = (a-1a)y (associativity) => /3=y (a-Ia= 1). 13 Remark. We know some rings for which the cancellation law holds: e.g. Z, R[s], cr [s], lRv(s), Rv.o(s), R(O), R 0(0), RU' Such rings are called integral domains, or better yet, entire rings. 14 Fact. 'iae R, a'O=O'a=O Proof. a+O=a => a'(a+O)=a'a => a·a+a·O=a·a. Adding -(a'a) to both sides gives a' 0= O. Repeat the proof but multiply by a on the right: O+a=a => (o+a)' a=a' a ,etc., gives O' a=O. • 15 Fact. 'ia,l3eR, (-a)I3=-(a·/3)=a·(-I3). Proof. 0= 0,/3= [a+(--a))·/3= a'!3+(-a)'/3 => -(a'/3) = (--a)'r~ • 16 Exercise. Show, from the axioms, that in any ring R, (± (I: i: I: ai ]. /3k ] = ai/3k . • i=1 k=! i=! k=1 17 The ring K is called a commutative ring iff, in addition to the standard axioms (9) we have 18 pq=qp 'ip,qe K. 19 Example. The commutative ring K might be (1) any field: R, R(s), ... ; (2) R[s], Rp(s), Rp,o(s), R (0), R 0(0), ... ; (3) scalar convolution operators: p*q = q*p. 408 20 Addition and Multiplication of matrices with elements in K are defined as follows (n denotes the sequence 1,2, ... ,n): If Pe Krnxn and QeKrnxn then (P+Q)ij := Pij + qij V ie ill, V je n defines the matrix P+Q that is, in fact, in Krnxn. If Pe Krnxn and Qe KTlXp then n L (PQ)ik := Pij qjk V ie 01, V ke I! j=l defines the matrix PQ, which is an element of KffiXP. I 21 Exercise. Show that for n > I, Knxn is a noncom mutative ring. [Hint: check that the axioms are satisfied]. 22 Fact. For matrices with elements in K, the definition and properties of deter minants hold as in the case of elements in a field as long as one does not take inverses! For example, if P,Q e Knxn, then det(PQ) e K and det(PQ) =det(P)·det(Q). 23 Fact [Cramer's rule]. Let PeKnxn, hence detPe K. Let Adj(P) denote as usual the "classical adjoint", [Sigl. p.282], of P. By direct calculation we have: 24 (a) Adj(P) P=P Adj(P)=(det P)In (b) PeKnxn has an inverse in Knxn 25 <;:;> det P has an inverse in K. In that case, 26 p-l = Adj (P)[det(PW 1 e Knxn. 27 Comments. From (24) and (26) it follows that P has a right inverse iff it has a left inverse; the common right and left inverse of P is called the inverse of P, (cf. (AU 1). Proof of (23): Outline: (a) (24) is equivalent to, [Sig.l,p.287], n n L L CkiPkj= Pikcjk=BijldetPj Vi,jell k=l k=l where (I) Vi=j. 409 (2) Cij is the cofactor of element Pij of P, i.e., cij=(-I)i+jmij with mij denoting the determinant of the matrix obtained by crossing out row i and column j of P. (b) If P has an inverse p-i e Knxn, then by the axioms of K, det(p-i) e K; now pp-I = ~ implies [det(P)]. [det(p-I)] = 1; hence (25) holds. Conversely if (25) holds, then the RHS of (26) e Knxn and is the inverse of P according to (24). • 28 Note. A matrix Pe Knxn is said to be nonsingular iff detP *" 0, where 0 is the additive identity of K. Hence if K is a field then condition (25) is equivalent to *" det P O. Therefore, we have the following coroJlary. 29 Corollary: Let Pe pnxn, then Pe pnxn has an inverse in pnxn • ~ P is nonsingular. 30 Comment: If the ring K is not a field then there may exist nonsingular matrices P e Knxn having no inverses in Kn xn : however corollary (29) still holds for inverses in Fnxn where the field F has K as a subring." 31 Example: Let K = R[s] be the ring of polynomials; R[s] is a subring of the field F= R(s) of rational functions. Hence, according to Corollary (29), P e R[s]nxn c R(s)nxn has an inverse in R(s)nxn iff Pis nonsingular. [~2 l~s3 To wit: let Pt(s) = ; ], PI(s)-t= [_:2 -; ]. However, according to Fact (23), Pe R[s]nxn has an inverse in R[s]nxn iff det Pis S.t. (detp)-I e R[s], i.e. such that det P is a nonzero constant (i.e. a polynomial of order zero): such polynomial matrices are called unimodular, (equiv. invertible in R[s]n xn ). [~ ~]. [~ To wit: let P2(s) = P2(s)-1 = -;]. It follows that unimodular polynomial matrices are nonsingular but the converse is not true. To wit: PI(s) e R[s]2x2 and P (s) e R[s]2x2 are both nonsingular but only P (s) is uni 2 2 modular: det P2(s) = I, det PI (s) = I-s3 (not a nonzero constant). A3. Linear Spaces Every engineer has encountered the linear spaces Rn, trn .. '. Linear spaces are also called "vector spaces" or "linear vector spaces." Roughly speaking, a linear space is a set of vectors say V, to which we add, a field of scalars, say F, with which 410 we multiply vectors. So we shall denote a linear space by (V,F) or by V for brevity. Sometimes to emphasize the field F, we say the F-linear space V. 1 Definition. We call linear space (V,F) the object consisting of a set (of vectors) Y, a field (of scalars) F and two binary operations viz. addition of vectors + and multiplication of vectors' by scalars ., which obey the following axioms: (a) Addition is given by +:VxV -+ V:(x,y) -+ x+y; Addition is Assqciative: (x+y)+z = x+(y+z) V x,y,ze V Commutative: x+y = y+x V x,ye V 3! identity e, (called the zero vector), S.t. x+e=e+x=x VxeV 3! inverse: V xe Y, 3!(-x)eY S.t. x+(-x)=9; (b) Multiplication by scalars is given by .: FxY -+ V:(a,x) -+ ax where V xe V V a,~e F (ap)x = a(px) lx=x Ox=9; (c) Addition and multiplication by scalars are related by distributive laws viz. V xe V, V a.~e F (a+~)x = ax+~x • Vx.yeV, VaeF a(x+y) = ax+ay . There are two extremely important examples of linear spaces: for this reason we call them canonical examples. 2 Canonical Example I. The linear space (P.F): the linear space of n-tuples in F over the field F, with elements x = (Si );. Y= (lli); where each Si:'lie F for ie n· Addition and scalar multiplication are defined by x+Y:=(Si+lli)P and aX:=(uSi)P V ae F . The most common examples are ( ern, er), (Rn,R), (R(s)n,JR(s» or ern,Rn,R(s)n for short. 3 Exercise. Show that (P,F) is a linear space. [Hint: use the axioms of the field 411 to check the axioms in Definition (1).] 4 Canonical Example II. The function space F(D,V): Let (V,F) be a linear space. Let D be a set. Let M be the class of all functions: D ~ V. On M define addition and scalar multiplication by (f+g) (d) = f(d)+g(d) 'V f,ge M 'V de D (aO(d)= af(d) 'VaeF,'VfeM 'VdeD Then M with these operations is a linear space over F; it is denoted by F(D,V), or F, when D and V are understood. 5 Exercise. Using the definitions of a function and of a linear space, show that F(D,V) is a linear space. Describe precisely what the zero vector is in this case. (Denote it by 9F, and that in V by 9v). Comment. D stands for domain and is an arbitrary set, e.g. N, R, Rn or a function space. Note also that V is an arbitrary linear space. 6 Example III. The function space PC ([ro,td,Rn): it is the set of all functions map ping the bounded interval [to,ttl into Rn which are piecewise continuous, i.e. they are continuous everywhere, except that they may have a finite number of discontinuity points 'tk where the one-sided limits f('tk+) and f('tk-) are well defined and finite. An Rn-valued function defined on an infinite interval is said to be piecewise continuous iff it is piecewise continuous on every bounded subinterval. The prototype of a function in PC ([O,oo),R) is the square wave. 7 Example IV. The space of continuous functions mapping [to,ttl ~ Rn denoted by C([to,t)],Rn). 8 Example V. The space of k times continuously differentiable functions mapping [to,t)] ~ Rn denoted by Ck([to,td,Rn) or Ck for short. 9 Example VI. The space of functions f: [to,td ~ Rn that are k times differentiable S.t. the kth derivative is piecewise continuous (whence necessarily each function and its derivatives up to the (k-l)th are continuous). 10 Exercise. Show that examples III-VI are linear spaces. 11 Example VII. Let F= R or C:. The space of 21t-periodic functions: [O,21t] ~ F such that 412 L00 f(t)= Ckeikl where k=-oo Note that if F=1R then Ck=C_k=Ckr+jcki and L00 f(t) = co+ 2 (Cia cos (kt)-cki sin (kt», k=l then "each vector" of this space is specified by the sequence (ck );. We shall next describe the concept of subspace and product space. 14 Definition. Let (V,F) be a linear space and W a subset of V. Then ( W,F) is called a (linear) subspace of V iff ( W,F) is itself a linear space. From this definition it follows that 15 Examples. The set of all vectors in Rn whose first component is zero. The set of all functions fe F (D,V) that are equal to 8y (the zero vector in V) at some fixed point doe D or on some fixed subset Dl of D. The set of all functions f: R+ --+ R, integrable over ~, whose Laplace transform is zero at some fixed point Zo with Re(zo) > O. 16 Exercises. Let I be an index set and (Wi)iEI be a family of subspaces of a linear space (V,F). Show that n Wi is a subspace of V. Give an example to show that ieI WI uW2 is not necessarily a subspace. Show that WI+W2:= {Wl+w2:wjeWj ie2,} is a subspace. ["Subspaces get smaller by intersecting and bigger by adding."] 17 Definition. Let (V,F) be a linear space. We call the subspace generated by a subset S c V the intersection of all subspaces containing S, or equivalently the smallest subspace containing S. 18 Fact [Sig.1. p.196]. Let (V,F) be a linear space. Then the smallest subspace generated by a subset S cV is the span of S denoted by Sp(S) viz. the set of finite linear combinations (lie F, sjeS Vien. 19 Definition. Let (V,F) and (W,F) be linear spaces over the same field F. The linear space (VxW,F) is called product space of V and W: it consists of vectors
Description: