Table Of Content

Acta Numerica 1992 Managing Editor A. Iserles DAMTP, University of Cambridge, Silver Street Cambridge CB3 9EW, England Editorial Board C. de Booi; University of Wisconsin, Madison, USA F. Brezzi, Instituto di Analisi Numerica del CNR, Italy J.C. Butchei; University of Auckland, New Zealand P.G. Ciarlet, Universiti Paris VI, France G.H. Golub, Stanford University, USA H.B. Keller California Institute of Technology, USA H.-O. Kreiss, University of California, Los Ang/eks, USA K.W. Morton, University of Oxford, England MJ.D. Powell, University of Cambridge, England R. Temam, Universiti Paris Sud, France umerica 1992 CAMBRIDGE UNIVERSITY PRESS Published by the Press Syndicate of the University of Cambridge The Pitt Building, Trumpington Street, Cambridge CB2 1RP 40 West 20th Street, New York, NY 10011-4211, USA 10 Stamford Road, Oakleigh, Victoria 3166, Australia © Cambridge University Press 1992 First published 1992 Printed in the United States of America A catalog record for this book is available from the British Library ISBN 0-521-41026-6 hardback ISSN 0962-4929 Contents Wavelets 1 Ronald A. DeVore and Bradley J. Lucier Iterative solution of linear systems 57 R. W. Freund, G.H. Golub and N.M. Nachtigal Problems with different time scales 101 Heinz-Otto Kreiss Numerical methods for differential algebraic equations 141 Roswitha Mdrz Theory of algorithms for unconstrained optimization 199 J. Nocedal Symplectic integrators for Hamiltonian problems: an overview 243 J.M. Sanz-Serna Error analysis of boundary integral methods 287 Ian H. Sloan Interior methods for constrained optimization 341 Margaret H. Wright Preface In these days and age, when the sheer number of publications in numerical mathematics increases so rapidly, it is surely necessary to provide valid justification to a new publication. The reason for Ada Numerica is, para- doxically, to counteract the information explosion by presenting selected and important developments in numerical mathematics and scientific computa- tion on an annual basis. Each year, the Editorial Board of Ada Numerica poses itself the question 'what are recent significant developments in our subject, developments that are important enough to merit interest by the numerical community as a whole'. Having selected a shortlist of topics, we ask named individuals to write survey papers. The purpose of the exercise being to disseminate knowl- edge outside restricted professional boundaries, the authors are expected to pitch their exposition so that it can be understood and appreciated by all practitioners of the numerical art, and not just by workers in a narrow sub- discipline. We are guided in our choice of authors both by their contribution to the underlying topic and by their track record as expositors. Numerical analysts, like other professionals in a competitive world, are busy with their own research, academic and administrative duties. It is difficult (and sometimes impossible) to keep up with developments outside one's narrow experience. This, we believe, is an unhealthy and undesir- able situation, not only because of broader cultural considerations but also since developments in different parts of numerical mathematics frequently impinge upon each other. We hope that Ada Numerica will play a role in bridging gaps and presenting many new and exciting ideas - algorithms and mathematical analysis alike - to a wider audience. Ada Numerica (1991), pp. 1-56 Wavelets* Ronald A. DeVore Department of Mathematics University of South Carolina, Columbia, SC 29208 USA E-maU: [email protected] Bradley J. Lucier Department of Mathematics Purdue University, West Lafayette, IN 47907 USA E-mail: [email protected] CONTENTS 1 Introduction 1 2 The Haar wavelets 4 3 The construction of wavelets 13 4 Fast wavelet transforms 37 5 Smoothness spaces and wavelet coefficients 40 6 Applications 44 References 54 1. Introduction The subject of 'wavelets' is expanding at such a tremendous rate that it is impossible to give, within these few pages, a complete introduction to all aspects of its theory. We hope, however, to allow the reader to become suf- ficiently acquainted with the subject to understand, in part, the enthusiasm of its proponents toward its potential application to various numerical problems. Furthermore, we hope that our exposition can guide the reader who wishes to make more serious excursions into the subject. Our viewpoint is biased by our experience in approximation theory and data compression; we warn the reader that there are other viewpoints that are either not represented here or discussed only briefly. For example, orthogonal wavelets were developed primarily in the context of signal processing, an application upon * This work was supported in part by the National Science Foundation (grants DMS- 8922154 and DMS-9006219), the Air Force Office of Scientific Research (contract 89- 0455-DEF), the Office of Naval Research (contracts N00014-90-1343, N00014-91-J-1152, and N00014-91-J-1076), the Defense Advanced Research Projects Agency (AFOSR contract 90-0323), and the Army High Performance Computing Research Center. R. A. DEVORE AND B. J. LUCIER 4> -1 0 j - 1 j + 1 _ 2* 2* 2* Fig. 1. An example of functions <f> and ^(2*- — j). which we touch only indirectly. However, there are several good expositions (e.g. Daubechies (1990) and Rioul and Vetterli (1991)) of this application. A discussion of wavelet decompositions in the context of Littlewood-Paley theory can be found in the monograph of Frazier et al. (1991). We shall also not attempt to give a complete discussion of the history of wavelets. Histor- ical accounts can be found in the book of Meyer (1990) and the introduction of the article of Daubechies (1990). We shall try to give sufficient historical commentary in the course of our presentation to provide some feeling for the subject's development. The term 'wavelet' (originally called wavelet of constant shape) was intro- duced by J. Morlet. It denotes a uni-variate function ty (multi-variate wavelets exist as well and will be discussed subsequently), defined on R, which, when subjected to the fundamental operations of shifts (i.e. translation by integers) and dyadic dilation, yields an orthogonal basis of L2(R). That is, the functions V'j.fc := 2*/2V>(2*- — j), j,fc6Z, form a complete orthonormal system for L2OR). In this work, we shall call such a function an orthogonal wavelet, since there are many generalizations of wavelets that drop the requirement of orthogonality. The Haar function H := X[o,i/2) ~ X[i/2,i)> which will be discussed in more detail in the section that follows, is the sim- plest example of an orthogonal wavelet. Orthogonal wavelets with higher smoothness (and even compact support) can also be constructed. But before considering that and other questions, we wish first to motivate the desire for such wavelets. We view a wavelet ip as a 'bump' (and think of it as having compact support, though it need not). Dilation squeezes or expands the bump and translation shifts it (see Figure 1). Thus, tpj^ is a scaled version of xp centred at the dyadic integer j2~k. If k is large positive, then tpj^ is a bump with small support; if k is large negative, the support of ipj^ is large. WAVELETS 3 The requirement that the set {V"j,fc}j,fcez forms an orthonormal system means that any function / G L2O&) can be represented as a series / = with {/, g) := J fgdx the usual inner product of two L2W functions. We R view (1.1) as building up the function / from the bumps ipj,k- Bumps corresponding to small values of k contribute to the broad resolution of /; those corresponding to large values of k give finer detail. The decomposition (1.1) is analogous to the Fourier decomposition of a function / G Z<2(T) in terms of the exponential functions e^ :=eik', but there are important differences. The exponential functions e*; have global support. Thus, all terms in the Fourier decomposition contribute to the value of / at a point x. On the other hand, wavelets are usually either of compact support or fall off exponentially at infinity. Thus, only the terms in (1.1) corresponding to Vj,fc with j2~k near x make a large contribution at x. The representation (1.1) is in this sense local. Of course, exponential functions have other important properties; for example, they are eigenfunctions for differentiation. Many wavelets have a corresponding property captured in the 'refinement equation' for the function <j> from which the wavelet ijj is derived, as discussed in Section 3.1. Another important property of wavelet decompositions not present di- rectly in the Fourier decomposition is that the coefficients in wavelet decompositions usually encode all information needed to tell whether / is in a smoothness space, such as the Sobolev and Besov spaces. For example, if ip is smooth enough, then a function / is in the Lipschitz space Lip(o!, Loc(R)), 0 < a < 1, if and only if sup2*(a+2)|(/,t/',fc)| (1.2) : i is finite, and (1.2) is an equivalent semi-norm for Lip(a, Loo(R)). All this would be of little more than theoretical interest if it were not for the fact that one can efficiently compute wavelet coefficients and reconstruct functions from these coefficients. Such algorithms, known as 'fast wavelet transforms' are the analogue of the Fast Fourier Transform and follow simply from the refinement equation mentioned earlier. In many numerical applications, the orthogonality of the translated dilates tpjk is not vital. There are many variants of wavelets, such as the t pre-wavelets proposed by Battle (1987) and the ^-transform of Frazier and Jawerth (1990), that do not require orthogonality. Typically, for a given function ip, one wants the translated dilates V'j.fc) h & € Z, to form a stable basis (also called a Riesz basis) for Z,2(K). This means that each / € L2W has a unique series decomposition in terms of the Vj.fc, and that the £2