An Introduction to Wavelets Through Linear Algebra

517 Pages·2000·19.086 MB·English

by  Michael W. Frazier (auth.)

Undergraduate Thxts in Mathematics
Editors
S. Axler
F. W. Gehring
K.A. Ribet
Springer Science+Business Media, LLC
Undergraduate Texts in Mathematics
Anglin: Mathematics: A Concise History Devlin: The Joy of Sets: Fundamentals
and Philosophy. of Contemporary Set Theory.
Readings in Mathematics. Second edition.
Anglin/Lambek: The Heritage of Dixmier: General Topology.
Thales. Driver: Why Math?
Readings in Mathematics. Ebbinghaus/Flum/Thomas:
Apostol: Introduction to Analytic Mathematical Logic. Second edition.
Number Theory. Second edition. Edgar: Measure, Topology, and Fractal
Armstrong: Basic Topology. Geometry.
Armstrong: Groups and Symmetry. Elaydi: Introduction to Difference
Axler: Linear Algebra Done Right. Equations.
Second edition. Exner: An Accompaniment to Higher
Beardon: Limits: A New Approach to Mathematics.
Real Analysis. Fine/Rosenberger: The Fundamental
Bak/Newman: Complex Analysis. Theory of Algebra.
Second edition. Fischer: Intermediate Real Analysis.
BanchofUWermer: Linear Algebra Flanigan/Kazdan: Calculus Two: Linear
Through Geometry. Second edition. and Nonlinear Functions. Second
Berberian: A First Course in Real edition.
Analysis. Fleming: Functions of Several Variables.
Bix: Conics and Cubics: A Second edition.
Concretem Introduction to Algebraic Foulds: Combinatorial Optimization for
Curves. Undergraduates.
Bremaud: An Introduction to Foulds: Optimization Techniques: An
Probabilistic Modeling. Introduction.
Bressoud: Factorization and Primality Franklin: Methods of Mathematical
Testing. Economics.
Bressoud: Second Year Calculus. Frazier: An Introduction to Wavelets
Readings in Mathematics. Through Linear Algebra.
Brickman: Mathematical Introduction Gordon: Discrete Probability.
to Linear Programming and Game Hairer/Wanner: Analysis by Its History.
Theory. Readings in Mathematics.
Browder: Mathematical Analysis: Halmos: Finite-Dimensional Vector
An Introduction. Spaces. Second edition.
Buskeslvan Rooij: Topological Spaces: Halmos: Naive Set Theory.
From Distance to Neighborhood. Hammerlin/Hoffmann: Numerical
Cederberg: A Course in Modem Mathematics.
Geometries. Readings in Mathematics.
Childs: A Concrete Introduction to Hijab: Introduction to Calculus and
Higher Algebra. Second edition. Classical Analysis.
Chung: Elementary Probability Theory Hilton/Holton/Pedersen: Mathematical
with Stochastic Processes. Third Reflections: In a Room with Many
edition. Mirrors.
Cox!Little/O'Shea: Ideals, Varieties, Iooss/Joseph: Elementary Stability
and Algorithms. Second edition. and Bifurcation Theory. Second
Croom: Basic Concepts of Algebraic edition.
Topology. Isaac: The Pleasures of Probability.
Curtis: Linear Algebra: An Introductory Readings in Mathematics.
Approach. Fourth edition.
I continued after index)
Michael W. Frazier
An Introduction
to Wavelets Through
Linear Algebra
With 46 Illustrations
~
Springer
Michael W. Frazier
Michigan State University
Department of Mathematics
East Lansing, MI 48824
USA
Editorial Board
S. Axler F.W. Gehring K.A. Ribet
Mathematics Department Mathematics Department Department of
San Francisco State East Hall Mathematics
University University of Michigan University of California
San Francisco, CA 94132 Ann Arbor, MI 48109 at Berkeley
USA USA Berkeley, CA 94720-3840
USA
Mathematics Subject Classification (1991): 42-01 46CXX 65F50
Library of Congress Cataloging-in-Publication Data
Frazier, Michael, 1956-
An introduction to wavelets through linear algebra / Michael W. Frazier
p. cm.-(Undergraduate texts in mathematics)
Inc1udes bibliographical references and index.
1. Wave1ets (Mathematics) 2. Aigebras, Linear. I. Tide.
II. Se ries.
QA403.3.F73 1999
515' .2433-dc21 98-43866
Printed on acid-free paper.
© Springer Science+Business Media New York 1999
Originally published by Springer-Verlag New York, Ine. in 1999
Softcover reprint of the hardcover 1s t edition 1999
All rights reserved. This work may not be translated or copied in whole or in part without
the written permission of the publisher except for brief excerpts in connection with reviews or
scholarly analysis. Use in connection with any form of information storage and retrieval, electronic
adaptation, computer software, or by similar or dissimilar methodology now known or
hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication,
even if the former are not especially identified, is not to be taken as a sign that such names,
as understood by the 'Ii"ade Marks and Merchandise Marks Act, may accordingly be used
freely by anyone.
Production managed by Jenny Wolkowickij manufacturing supervised by Joe Quatela.
1Ypeset by The Bartlett Press, Inc. from the author's TEX files.
9 8 7 6 5 4 3 2 1
ISBN 978-3-642-85572-6 ISBN 978-3-642-85570-2 (eBook)
DOI 10.1007/978-3-642-85570-2
Preface
Mathematics majors at Michigan State University take a "Capstone"
course near the end of their undergraduate careers. The content
of this course varies with each offering. Its purpose is to bring
together different topics from the undergraduate curriculum and
introduce students to a developing area in mathematics. This text
was originally written for a Capstone course.
Basic wavelet theory is a natural topic for such a course. By name,
wavelets date back only to the 1980s. On the boundary between
mathematics and engineering, wavelet theory shows students that
mathematics research is still thriving, with important applications
in areas such as image compression and the numerical solution
of differential equations. The author believes that the essentials of
wavelet theory are sufficiently elementary to be taught successfully
to advanced undergraduates.
This text is intended for undergraduates, so only a basic
background in linear algebra and analysis is assumed. We do not
require familiarity with complex numbers and the roots of unity.
These are introduced in the first two sections of chapter 1. In the
remainder of chapter 1 we review linear algebra. Students should be
familiar with the basic definitions in sections 1.3 and 1.4. From our
viewpoint, linear transformations are the primary object of study;
v
vi
Preface
a matrix arises as a realization of a linear transformation. Many
students may have been exposed to the material on change ofbasis
in section 1.4, but may benefit from seeing it again. In section 1.5,
we ask how to pick a basis to simplify the matrix representation of
a given linear transformation as much as possible. We then focus on
the simplest case, when the linear transformation is diagonalizable.
In section 1.6, we discuss inner products and orthonormal bases. We
end with a statement of the spectral theorem for matrices, whose
proof is outlined in the exercises. This is beyond the experience of
most undergraduates.
Chapter 1 is intended as reference material. Depending on
background, many readers and instructors will be able to skip or
quickly review much of this material. The treatment in chapter 1 is
relatively thorough, however, to make the text as self-contained as
possible, provide a logically ordered context for the subject matter,
and motivate later developments.
The author believes that students should be introduced to Fourier
analysis in the finite dimensional context, where everything can be
explained in terms of linear algebra. The key ideas can be exhibited
in this setting without the distraction of technicalities relating to
convergence. We start by introducing the Discrete Fourier Transform
(DFT) in section 2.1. The DFT of a vector consists of its components
with respect to a certain orthogonal basis of complex exponentials.
The key point, that all translation-invariant linear transformations
are diagonalized by this basis, is proved in section 2.2. We turn to
computational issues in section 2.3, where we see that the DFT can
be computed rapidly via the Fast Fourier Transform (FIT).
It is not so well known that the basics of wavelet theory can
also be introduced in the finite dimensional context. This is done
in chapter 3. The material here is not entirely standard; it is an
adaptation of wavelet theory to the finite dimensional setting. It has
the advantage that it requires only linear algebra as background. In
section 3.1, we search for orthonormal bases with both space and
frequency localization, which can be computed rapidly. We are led
to consider the even integer translates of two vectors, the mother and
father wavelets in this context. The filter bank arrangement for the
computation of wavelets arises naturally here. By iterating this filter
bank structure, we arrive in section 3.2 at a multilevel wavelet basis.
Preface Vll
Examples and applications are discussed in section 3.3. Daubechies's
wavelets are presented in this context, and elementary compression
examples are considered. A student familiar with MatLab, Maple, or
Mathematica should be able to carry out similar examples if desired.
In section 4.1 we change to the infinite dimensional but discrete
setting l 2(Z), the square summable sequences on the integers.
General properties of complete orthonormal sets in inner product
spaces are discussed in section 4.2. This is first point where analysis
enters our picture in a serious way. Square integrable functions
on the interval [- n, n) and their Fourier series are developed in
section 4.3. Here we have to cheat a little bit: we note that we
are using the Lebesgue integral but we don't define it, and we
ask students to accept certain of its properties. We arrive again at
the key principle that the Fourier system diagonalizes translation
invariant linear operators. The relevant version of the Fourier
transform in this setting is the map taking a sequence in l 2(Z)
to a function in L2([ -n, n)) whose Fourier coefficients make up
the original sequence. Its properties are presented in section 4.4.
Given this preparation, the construction of first stage wavelets on
the integers (section 4.5) and the iteration step yielding a multilevel
basis (section 4.6) are carried out in close analogy to the methods
in chapter 3. The computation of wavelets in the context of l 2(Z)
is discussed in section 4. 7, which includes the construction of
Daubechies's wavelets on Z. The generators u and v of a wavelet
system for l 2(Z) reappear in chapter 5 as the scaling sequence and
its companion.
The usual version of wavelet theory on the real line is presented
in chapter 5. The preliminaries regarding square integrable func
tions and the Fourier transform are discussed in sections 5.1 and 5.2.
The facts regarding Fourier inversion in L2(JR) are proved in detail,
although many instructors may prefer to assume these results. The
Fourier inversion formula is analogous to an orthonormal basis rep
resentation, using an integral rather than a sum. Again we see that
the Fourier system diagonalizes translation-invariant operators. Mal
lat's theorem that a multiresolution analysis yields an orthonormal
wavelet basis is proved in section 5.3. The aformentioned relation
between the scaling sequence and wavelets on l2(Z) allows us to
make direct use of the results of chapter 4. The conditions under
viii
Preface
which wavelets on i 2(Z) can be used to generate a multiresolution
analysis, and hence wavelets on R, are considered in section 5.4.
In section 5.5, we construct Daubechies's wavelets of compact sup
port, and show how the wavelet transform is implemented using
filter banks.
We briefly consider the application of these results to numerical
differential equations in chapter 6. We begin in section 6.1 with
a discussion of the condition number of a matrix. In section
6.2, we present a simple example of the numerical solution of a
constant coefficient ordinary differential equation on [0, 1] using
finite differences. We see that although the resulting matrix is
sparse, which is convenient, it has a condition number that grows
quadratically with the size of the matrix. By comparison, in section
6.3, we see that for a wavelet-Galerkin discretization of a uniformly
elliptic, possibly variable-coefficient, differential equation, the
matrix of the associated linear system can be preconditioned to be
sparse and to have bounded condition number. The boundedness
of the condition number comes from a norm equivalence property
of wavelets that we state without proof. The sparseness of the
associated matrix comes from the localization ofthe wavelet system.
A large proportion of the time, the orthogonality of wavelet basis
members comes from their supports not overlapping (using wavelets
of compact support, say). This is a much more robust property,
for example with respect to multiplying by a variable coefficient
function, than the delicate cancellation underlying the orthogonality
of the Fourier system. Thus, although the wavelet system may not
exactly diagonalize any natural operator, it nearly diagonalizes (in
the sense of the matrix being sparse) a much larger class of operators
than the Fourier basis.
Basic wavelet theory includes aspects of linear algebra, real
and complex analysis, numerical analysis, and engineering. In
this respect it mimics modern mathematics, which is becoming
increasingly interdisciplinary.
This text is relatively elementary at the start, but the level
of difficulty increases steadily. It can be used in different ways
depending on the preparation level of the class. If a long time is
required for chapter 1, then the more difficult proofs in the later
chapters may have to be only briefly outlined. For a more advanced
ix
Preface
group, most or all of chapter 1 could be skipped, which would leave
time for a relatively thorough treatment of the remainder. A shorter
course for a more sophisticated audience could start in chapter
4 because the main material in chapters 4 and 5 is technically,
although not conceptually, independent of the content of chapters
2 and 3. An individual with a solid background in Fourier analysis
could learn the basics of wavelet theory from sections 4.5, 4.7, 5.3,
5.4, 5.5, and 6.3 with only occasional references to the remainder of
the text.
This volume is intended as an introduction to wavelet theory
that is as elementary as possible. It is not designed to be a thorough
reference. We refer the reader interested in additional information
to the Bibliography at the end of the text.
Michigan State University M. Frazier
April1999