Hilbert Spaces [expository notes]

35 Pages·2014·0.204 MB·English

by  Joseph Muscat

Hilbert Spaces
Joseph Muscat
2014-5-27
(A revised and expanded version of these notes are now published by
Springer.)
1 Inner Product Spaces
1.1 Introduction
Definition An inner-product on a vector space X is a map
, : X X C
h i × →
such that
x,y +z = x,y + x,z ,
h i h i h i
x,λy = λ x,y ,
h i h i
y,x = x,y ,
h i h i
x,x > 0; x,x = 0 x = 0.
h i h i ⇔
In words, it is said to be a positive-definite sesquilinear form.
The simplest examples are RN and CN with x,y = N x¯ y ; the
h i n=1 n n
square matrices of size N N also have an inner-product given by A,B =
N A¯ B . × P h i
i,j=1 ij ij
PProposition 1.1
1. x,x is real (and positive) and is denoted by x 2
h i k k
2. x,y = 0 x y = 0 (put x = y)
h i ∀ ⇒
¯
3. λx,y = λ x,y (anti-linear); x+y,z = x,z + y,z ;
h i h i h i h i h i
1
1.1 Introduction J Muscat 2
4. λx = λ x .
k k | |k k
Definition Two vectors x,y are orthogonal when x,y = 0, written
h i
as x y. The angle between two vectors is given by cosθ = x,y / x y .
⊥ h i k kk k
Proposition 1.2
1. x+y 2 = x 2 +2Re x,y + y 2.
k k k k h i k k
2. (Pythagoras) If z = x+y and x,y = 0 then z 2 = x 2+ y 2.
h i k k k k k k
3. For any orthogonal vectors x,y, x 6 x+y .
k k k k
4. For any non-zero vectors x,y, there is a unique vector z and a
unique scalar λ such that x = z +λy and z y.
⊥
Proof. The first three statements follow immediately from the axioms
and the first proposition. For the last statement, let z = x λy as required;
−
then z y holds λ = y,x / y,y (check).
⊥ ⇔ h i h i
(cid:3)
Proposition 1.3 Cauchy-Schwarz
x,y 6 x y
|h i| k kk k
Proof. Decompose x into orthogonal parts x = (x λy)+λy where λ = hy,xi
− y,y
h i
(assuming y = 0); nowusePythagoras’ theorem, anddeduce that λy 6 x .
6 | | k k
(cid:3)
Note that Pythagoras’ theorem and Cauchy-Schwarz’s inequality are still
valid even if the ‘inner-product’ is not positive definite but just semi-definite,
as long as y = 0.
k k 6
Corollary
x+y 6 x + y
k k k k k k
The proof is simply an application of the Cauchy-Schwarz inequality to
the expansion of x+y 2.
k k
(cid:3)
1.1 Introduction J Muscat 3
Hence x is a norm, and all the facts about normed vector spaces apply
k k
to inner-product spaces. In particular they are metric spaces with distance
d(x,y) = x y , convergence of sequences makes sense as x x
n
k − k → ⇔
x x 0, continuity and dual spaces also make sense. Inner-product
n
k − k →
spaces are special normed spaces which not only have a concept of length
but also of angle.
Definition A Hilbert space is an inner-product space which is com-
plete as a metric space.
Proposition 1.4 The closure of a linear subspace remains a linear
subspace.
¯
Proof. Ifx ,y A, thenlim x +lim y = lim (x +y ) A.
n n n n n n n n n
∈ →∞ →∞ →∞ ∈ (cid:3)
Recall that finite dimensional subspaces are always closed. (exercise)
From the theory of Banach spaces we know that addition and scalar
multiplication are continuous operations.
Proposition 1.5 +,λ , , are continuous in each variable.
· h i
Proof. If x x < δ and y y < δ then
′ ′
k − k k − k
(x+y) (x +y ) 6 x x + y y < 2δ.
′ ′ ′ ′
k − k k − k k − k
Similarly,
λ(x x) = λ x x < λ δ.
′ ′
k − k | |k − k | |
We can consider φ := y, to be a functional mapping x y,x .
y
h i → h i
This functional is continuous because φ (x) = y,x 6 y x . In fact
y
| | |h i| k kk k
φ = y .
y
k k k k
(cid:3)
It follows that taking limits commutes with the inner-product:
x, lim y = lim x,y .
n n
h n i n h i
→∞ →∞
Doallnormsonvectorspacescomefrominner-products, andifnot, which
normed vector spaces are in fact inner-product spaces? The answer is given
by
1.1 Introduction J Muscat 4
Proposition 1.6 (Parallelogram law)
A norm comes from an inner-product if, and only if, it satisfies
x+y 2+ x y 2 = 2( x 2 + y 2)
k k k − k k k k k
Proof. Expanding and adding x + y 2 and x y 2 gives the paral-
k k k − k
lelogram law. Conversely, subtracting the two gives 4Re x,y . Hence, by
h i
noticing that Im x,y = Rei x,y = Re ix,y , we get the polarization
h i − h i h i
identity
1
x,y = y +x 2 y x 2 +i y +ix 2 i y ix 2 .
h i 4 k k −k − k k k − k − k
(cid:0) (cid:1)
The converse is true but harder to prove.
(cid:3)
This law can be generalized further. If ωN = 1 (N > 2), then x,y =
h i
1 N ωn y + ωnx 2. Even more generally, x,y = 1 y + zx 2dz.
N n=1 k k h i 2πi S1k k
That is, the inner product x,y is a ‘complex’ average of lengths on a ball
P h i R
of radius x , centred at y.
k k
Proposition 1.7 L2(R) is a Hilbert space with inner-product f,g =
¯ h i
fg.
R Proof. We assume L2(R) is a complete normed space. Verifying the
parallelogram law,
f +g 2 + f g 2 = ( f 2+f¯g +fg¯+ g 2 + f 2 f¯g fg¯+ g 2.
| | | − | | | | | | | − − | |
Z Z Z
Hence find the associated inner-product from the previous proposition.
(cid:3)
1.1.1 Exercises
1. Find the angle between sin and cos in the space L2[ π,π].
−
2. Show that ℓ2 is a Hilbert space with inner-product hx,yi = ∞n=1x¯nyn.
Show further that l1 and l are not inner-product spaces by finding
∞
P
two sequences which do not satisfy the parallelogram law.
3. Similarly show that L and L1 are not inner-product spaces.
∞
1.1 Introduction J Muscat 5
4. Show that the Cauchy-Schwarz inequality becomes an equality if, and
only if, x = αy α.
∃
5. Let d = inf x+λy ; show that
λ
k k
x,y 2 6 ( x 2 d2) y 2.
|h i| k k − k k
6. Show that any finite-dimensional subspace of a Hilbert space is closed.
Deduce that the set of polynomials of degree at most m forms a closed
linear subspace of L2[a,b] with dimension m+ 1; find a basis for this
space.
7. Show that the set of continuous functions C[a,b] is a non-closed linear
subspace of L2[a,b].
8. Starting from a norm . that satisfies the parallelogram law, we can
k k
create an inner-product using Prop. 6, from which we can define a
norm . := .,. . Show that the two norms are identical.
1
k k h i
9. Show that twopinner-products, on the same vector space, are conformal
i.e. give the same angles between vectors, if, and only if, x,y =
h i1
λ x,y for some positive real λ.
h i2
10. We have shown that an inner-product is determined by a norm that
satisfies the parallelogram law. More generally, show that any bilinear
form A(x,y) that satisfies A(y,x) = A(x,y), is determined by its real
quadratic form q(x) = A(x,x), as A(x,y) = 1(q(x + y) + q(x y) +
4 −
iq(x+iy) iq(x iy)).
− −
11. Showthattherealandimaginarypartsofaninner-productgivesymmetric/anti-
symmetric real-valued inner-products. Thus a complex inner-product
has more structure than a real (symmetric) inner-product e.g. C2 has
more structure than R4.
12. Let H H be the product of two inner-product spaces. Show that
1 2
×
x ,x , y ,y := x ,y + x ,y
h 1 2 1 2 i h 1 1iH1 h 2 2iH2
(cid:0) (cid:1) (cid:0) (cid:1)
defines an inner-product on H H .
1 2
×
13. Show that the formula f,g := f(x)g(x)w(x)dx, where w(x) is a
h i
positive real function, defines an inner-product. The resulting space of
R
functions is called a weighted L2 space.
1.2 Orthogonality J Muscat 6
14. * Consider the vector space of holomorphic functions f : R R+ C
× →
which satisfy sup f 2dx < . Show that the formula f,g :=
y>0 | | ∞ h i
lim f(x+iǫ)g(x+iǫ)dx gives an inner-product.
ǫ 0+ R
→
R
1.2 Orthogonality
Definition The orthogonal space of a set A is the set
A = x : y,x = 0 y A .
⊥
{ h i ∀ ∈ }
1.2.1 Exercises
1. Show that 0 = X, X = 0.
⊥ ⊥
2. Show that if x = X then x = 0.
⊥
{ }
3. Show that if x = 0 then X is one-dimensional.
⊥
{ }
Proposition 1.8 A is a closed linear subspace.
⊥
Proof. ThatA isalinearsubspace followsfromthelinearityoftheinner-
⊥
product. Let x A . That is, there is a sequence of vectors x A such
⊥ n ⊥
∈ ∈
that x x. Now, for any y A, x,y = lim x ,y = lim x ,y = 0.
n n n n n
→ ∈ h i h i h i
Hence x A .
⊥
∈
(cid:3)
Proposition 1.9
1. A A 0;
⊥
∩ ⊆
2. A B B is a closed subspace of A ;
⊥ ⊥
⊆ ⇒
3. A A .
⊥⊥
⊆
Proof. (i) is left as an easy exercise. For (ii), let x B i.e. b,x =
⊥
∈ h i
0 b B. In particular this is true for b A so that x A . For (iii), let
⊥
∀ ∈ ∈ ∈
x A, then y,x = 0, y A . Hence x,y = 0 y A . Hence x A .
⊥ ⊥ ⊥⊥
∈ h i ∀ ∈ h i ∀ ∈ ∈
(cid:3)
NotethatA isalwaysaclosedlinearsubspaceevenifAisn’t. Question:
⊥⊥
if M is a closed linear subspace is it necessarily true that M = M ?
⊥⊥
1.3 Least Distance in Hilbert spaces J Muscat 7
1.3 Least Distance in Hilbert spaces
Definition Aconvexset is aset Athat includes all linesegments between
its points:
x,y A, t [0,1], tx+(1 t)y A.
∀ ∈ ∀ ∈ − ∈
Exercise: Show that linear subspaces and balls are convex.
Theorem 1.10 If M is a closed convex set and x is any point of
a Hilbert space H, then there is a unique point x M which is
0
∈
closest to x.
x H !x M : d(x,x ) 6 d(x,y) y M.
0 0
∀ ∈ ∃ ∈ ∀ ∈
Proof. Let d be the smallest distance from M to x i.e. d = inf d(x,y)
y M
∈
(how do we know that d exists?) Then there is a sequence of vectors y M
n
∈
such that x y = d(x,y ) d. Now, by the parallelogram law,
n n
k − k →
y y 2 = 2 y x 2 +2 y x 2 (y +y ) 2x 2
n m n m n m
k − k k − k k − k −k − k
6 2 y x 2 +2 y x 2 4d2
n m
k − k k − k −
0
→
as n . Hence (y ) is a Cauchy sequence. But M is closed (hence
n
→ ∞
complete) and so y x M. It follows, by continuity of the norm, that
n 0
→ ∈
x x = d.
0
k − k
Suppose y M is another closest point to x, i.e. d(x,y) = d. Then
∈
x y 2 = 2 x x 2 +2 y x 2 (x +y) 2x 2
0 0 0
k − k k − k k − k −k − k
6 2 x x 2 +2 y x 2 4d2
0
k − k k − k −
= 0
(cid:3)
Note that this theorem, which does not refer to inner-products, is not
true in Banach spaces in general. For example l has a closed subspace c ,
∞ 0
but there is no closest sequence in c to the sequence (1,1,1,...). In fact,
0
the smallest distance is d = 1 and this is achieved by any bounded sequence
(a ) with 0 6 a 6 2.
n n
Exercise: prove these assertions.
1.3 Least Distance in Hilbert spaces J Muscat 8
Let us concentrate on the case when M is a closed subspace of H.
Theorem 1.11 For a closed linear subspace M of a Hilbert space
H, x M is the closest point to x H if, and only if, x x M .
0 0 ⊥
∈ ∈ − ∈
Proof. Let a be any point of M. Then x x = λa+z with a z, where
0
− ⊥
λ = a,x x / a 2 and z = x (x +λa). By Pythagoras’ theorem, we get
0 0
h − i k k −
z 6 x x making x +λa even closer to x than the closest point x .
0 0 0
k k k − k
This is only possible if λ = 0 i.e. a,x x = 0. Since a is arbitrary we get
0
h − i
x x M.
0
− ⊥
Conversely, if x x a for any a M, then by Pythagoras’ theorem,
0
− ⊥ ∈
x a 2 = x x 2 + x a 2, so that x x 6 x a , making x
0 0 0 0
k − k k − k k − k k − k k − k
the closest point to x.
(cid:3)
Corollary If M is a closed linear subspace of a Hilbert space
H, then H = M M .
⊥
⊕
Proof. Write x = x + (x x ). These two components are in M and
0 0
−
M as proved previously. Moreover M M = 0.
⊥ ⊥
∩
(cid:3)
Note that this is false for Banach spaces e.g. the space l = c M for
∞ 0
6 ⊕
any linear subspace M.
Corollary If M is a closed linear subspace of a Hilbert space
H, then M = M. More generally, for any set A, A = [[A]].
⊥⊥ ⊥⊥
Proof. Let x M . Then x = a + b where a M and b M .
⊥⊥ ⊥
∈ ∈ ∈
Then 0 = b,x = b,a + b,b = b 2, making b = 0 and x M. For the
h i h i h i k k ∈
secondpart, notethat[[A]]isthesmallest closedlinearsubspace containingA.
Therefore, since A [[A]], we get [[A]]⊥ A and hence A [[A]]⊥⊥ = [[A]].
⊥ ⊥⊥
⊆ ⊆ ⊆
(cid:3)
Corollary [[A]] is dense in H if, and only if, A = 0.
⊥
¯
Proof. Suppose [[A]] = H. Then A = A = H = 0, since A is a
⊥ ⊥⊥⊥ ⊥ ⊥
¯
closed linear subspace. Conversely, [[A]] = A = 0 = H.
⊥⊥ ⊥
(cid:3)
1.3 Least Distance in Hilbert spaces J Muscat 9
1.3.1 Exercises
1. Find the closest points x to a closed linear subspace M from (i) a
0
point x M, (ii) a point x M .
⊥
∈ ∈
2. In the decomposition x = a+b with a M and b M , show that a
⊥
∈ ∈
and b are unique.
3. Deduce that if H = M N where M is a closed linear subspace and
⊕
M N then N = M .
⊥
⊥
4. Show that, if M is a closed linear subspace of N, then M (M N) =
⊥
⊕ ∩
N.
1.3.2 Projections
Let us take a closer look at the map from x to x M.
0
∈
Definition A projection is a linear map P : H H such that
→
P2 = P. An orthogonal projection is one such that kerP imP.
⊥
Theorem 1.12 If M is a closed linear subspace of a Hilbert space
H then the map P : H H defined by Px = x is a continuous
0
→
orthogonal projection with imP = M and kerP = M .
⊥
Proof. By the definition of P, for any a H, (a Pa) M and this
⊥
∈ − ∈
property defines Pa M.
∈
P is linear since (x+y) (Px+Py) = (x Px)+(y Py) M and
⊥
− − − ∈
Px+Py M, hence P(x+y) = Px+Py. Similarly P(λx) = λPx.
∈
P is a projection since Px M implies that P2x = Px.
∈
P is onto M since for any x M, Px = x. Moreover x kerP Px =
∈ ∈ ⇔
0 x = x Px M .
⊥
⇔ − ∈
P is continuous since x 2 = x Px 2+ Px 2 by Pythagoras’ theorem
k k k − k k k
so that Px 6 x .
k k k k
(cid:3)
1.3.3 Exercises
1. Show that the map (x,y,z) (x,0,0) is an orthogonal projection in
7→
R3. Find a projection which is not orthogonal.
2. Let P,Q be two commuting projections. Show that PQ is also a pro-
jection.
1.3 Least Distance in Hilbert spaces J Muscat 10
3. Show that if P is a projection then H = kerP imP.
⊕
4. Show that any orthogonal projection is continuous, and imP kerP.
⊥
Hence note that there is a 1-1 correspondence between closed linear
subspaces and orthogonal projections.
5. Show that if P is an orthogonal projection onto M then I P is an
−
orthogonal projection onto M .
⊥
6. SupposethatH = M N. Showthat,ifP andQdenotetheorthogonal
⊕
projections onto M and N respectively, then I = P +Q and PQ = 0.
Extend this by induction to the case when H = M ... M to get
1 n
⊕ ⊕
I = P +...P with P P = P δ .
1 n i j i ij
7. Consider two closed linear subspaces M and M (with associated pro-
1 2
jections P and P ). Show that the iteration x = P P x starting
1 2 n+1 2 1 n
from x = y converges to the closest point y M M .
0 0 1 2
∈ ∩
8. Let M =< y > (a closed linear subspace). Find the orthogonal pro-
jection P which maps any point x to its closest point in M. (Ans:
Px = y,x y)
h i
9. Let M = y ,...,y . Find P again. (Ans: Px = α y where
h 1 Ni i i i
α = y ,x / y 2.
i i i
h i k k P
1.3.4 Examples: Least Squares Approximation
1. Find the closest point in the plane 2x+y 3z = 0 to the point x =
−
(5,2,0). Letting n = (2,1, 3), the equation of the plane is given by
−
n x = 0. Therefore we are looking for a point x in the plane such
0
·
that x x is perpendicular to the plane i.e. is parallel to n. Hence,
0
−
x = x+λn such that it satisfies n x = 0. Substituting gives λ.
0 0
·
2. Find the best-fitting (closest in the space L2) cubic polynomial to the
function sin in the region [0,2π].
The space of cubic polynomials, a+bx+cx2+dx3, is a four-dimensional
closedlinearsubspaceoftheHilbertspaceL2[0,2π],withbasis1,x,x2,x3.
It must therefore have an element p(x) = a+bx+cx2 +dx3 which is
closest to sinx. In fact, p is characterized by the condition p sin
− ⊥
q, q M. In particular we get four equations p(x) sin(x),xi = 0
∀ ∈ h − i
for i = 0,...,3, or, equivalently, p(x),xi = sin(x),xi . The right-
h i h i
hand sides can be worked out, giving four linear equations in the four
unknowns a,b,c,d, which can be extracted by solving.