Sparse Approximation with Bases

271 Pages·2015·1.78 MB·English

by  Temlyakov

Advanced Courses in Mathematics CRM Barcelona
Vladimir Temlyakov
Sparse
Approximation
with Bases

Advanced Courses in Mathematics
CRM Barcelona
Centre de Recerca Matemàtica
Managing Editor:
Carles Casacuberta

More information about this series at http://www.springer.com/series/5038
Vladimir Temlyakov
Sparse Approximation
with Bases
Editor for this volume:
Sergey Tikhonov, ICREA and CRM, Barcelona
Vladimir Temlyakov
Department of Mathematics
University of South Carolina
Columbia, SC, USA
ISSN 2297-0304 ISSN 2297-0312 (electronic)
Advanced Courses in Mathematics - CRM Barcelona
ISBN 978-3-0348-0889-7 ISBN 978-3-0348-0890-3 (eBook)
DOI 10.1007/978-3-0348-0890-3

Library of Congress Control Number: 2015935236

Mathematics Subject Classification (2010): Primary: 41A65, 41A25, 42A10; Secondary: 46B20
Springer Basel Heidelberg New York Dordrecht London
© Springer Basel 2015
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the
editors give a warranty, express or implied, with respect to the material contained herein or for any errors
or omissions that may have been made.

Printed on acid-free paper

Springer Basel AG is part of Springer Science+Business Media (www.birkhauser-science.com)
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
1 Introduction
1.1 General setting of approximation problems. . . . . . . . . . . . . 1
1.2 Existence and uniqueness of best approximation. . . . . . . . . . 5
1.3 Schauder bases in Banach spaces . . . . . . . . . . . . . . . . . . 10
1.4 Unconditional bases . . . . . . . . . . . . . . . . . . . . . . . . . 16
2 Lebesgue-type Inequalities for Greedy Approximation
with Respect to Some Classical Bases
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2 The trigonometric system . . . . . . . . . . . . . . . . . . . . . . 25
2.3 Wavelet bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.4 Greedy bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.5 Some examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.5.1 Unconditionality does not imply democracy . . . . . . . . 42
2.5.2 Democracy does not imply unconditionality . . . . . . . . 43
2.5.3 Superdemocracy does not imply unconditionality . . . . . 43
2.5.4 A quasi-greedy basis is not necessarily
an unconditional basis . . . . . . . . . . . . . . . . . . . . 44
2.6 Further results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.6.1 Direct and inverse theorems . . . . . . . . . . . . . . . . . 46
2.6.2 Greedy approximation in L1 and L∞ . . . . . . . . . . . . 51
2.7 Some inequalities for the tensor product of greedy bases . . . . . 54
3 Quasi-greedy Bases and Lebesgue-type Inequalities
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.2 Properties of quasi-greedy bases . . . . . . . . . . . . . . . . . . . 64
3.3 Construction of quasi-greedy bases . . . . . . . . . . . . . . . . . 77
3.4 Uniformly bounded quasi-greedy systems . . . . . . . . . . . . . . 84
3.5 Lebesgue-type inequalities for quasi-greedy bases . . . . . . . . . 90
v
vi Contents
3.6 Lebesgue-type inequalities for uniformly bounded
quasi-greedy bases . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.7 Lebesgue-type inequalities for uniformly bounded
orthonormal quasi-greedy bases . . . . . . . . . . . . . . . . . . . 99
4 Almost Greedy Bases and Duality
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.2 Greedy conditions for bases . . . . . . . . . . . . . . . . . . . . . 105
4.3 Democratic and conservative bases . . . . . . . . . . . . . . . . . 108
4.4 Bidemocratic bases . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.5 Duality of almost greedy bases . . . . . . . . . . . . . . . . . . . 116
5 Greedy Approximation with Respect to the Trigonometric System
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5.2 Convergence. Conditions on Fourier coefficients . . . . . . . . . . 127
5.2.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.2.2 Sufficient conditions in terms of Fourier coefficients.
Proof of Theorem 5.2.1. . . . . . . . . . . . . . . . . . . . 130
5.2.3 Sufficient conditions in terms of the decreasing rearrange-
ment of Fourier coefficients. Proof of Theorem 5.2.2. . . . 137
5.2.4 Convergence in the uniform norm.
Proof of Theorems 5.2.3–5.2.5 . . . . . . . . . . . . . . . . 140
5.3 Convergence. Conditions on greedy approximants . . . . . . . . . 150
5.3.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . 150
5.3.2 Some inequalities . . . . . . . . . . . . . . . . . . . . . . . 152
5.3.3 Sufficient conditions in the case p∈(2,∞) . . . . . . . . . 158
5.3.4 Necessary conditions in the case p∈(2,∞) . . . . . . . . 161
5.3.5 Necessary and sufficient conditions in the case p=∞. . . 169
5.4 An application of WCGA . . . . . . . . . . . . . . . . . . . . . . 174
5.4.1 Convergence. . . . . . . . . . . . . . . . . . . . . . . . . . 175
5.4.2 Rate of approximation . . . . . . . . . . . . . . . . . . . . 175
5.4.3 Constructive approximation of function classes . . . . . . 177
5.5 Constructive nonlinear trigonometric m-term approximation . . . 179
6 Greedy Approximation with Respect to Dictionaries
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
6.2 The Weak Chebyshev Greedy Algorithm . . . . . . . . . . . . . . 193
6.3 Relaxation. Co-convex approximation. . . . . . . . . . . . . . . . 200
6.4 Free relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
6.5 Fixed relaxation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
6.6 Relaxation. X-greedy algorithms . . . . . . . . . . . . . . . . . . 212
6.7 Greedy expansions . . . . . . . . . . . . . . . . . . . . . . . . . . 214
Contents vii
6.7.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . 214
6.7.2 Convergence of the Dual-Based Expansion . . . . . . . . . 217
6.7.3 A modification of the Weak Dual Greedy Algorithm . . . 221
6.7.4 Convergence of WDGA . . . . . . . . . . . . . . . . . . . 226
7 Appendix
7.1 Lp-spaces and some inequalities . . . . . . . . . . . . . . . . . . . 229
7.1.1 Modulus of continuity . . . . . . . . . . . . . . . . . . . . 229
7.1.2 Some inequalities . . . . . . . . . . . . . . . . . . . . . . . 231
7.2 Duality in L spaces . . . . . . . . . . . . . . . . . . . . . . . . . 236
p
7.3 Fourier series of functions in L . . . . . . . . . . . . . . . . . . . 239
p
7.4 Trigonometric polynomials . . . . . . . . . . . . . . . . . . . . . . 243
7.5 Bernstein–Nikol’skii Inequalities.
The Marcinkiewicz Theorem . . . . . . . . . . . . . . . . . . . . . 249
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

Preface
The last decade has seen great progress in the study of nonlinear approxima-
tion, which was motivated by numerous applications. Nonlinear approximation
is important in applications because of its concise representations and increased
computationalefficiency.Twotypesofnonlinearapproximationarefrequentlyem-
ployed in applications. Adaptive methods are used in PDE solvers, while m-term
approximation, considered here, is used in image/signal/data processing, as well
as in the design of neural networks. Another name for m-term approximation is
sparse approximation. Sparse approximations (representations) of functions are
notonly a powerful analytictool, but they areutilized in many applicationareas,
such as image/signal processing and numerical computation.
The fundamental questionof nonlinear approximationis how to devise good
constructive methods (algorithms). This problem has two levels of nonlinearity.
The first level of nonlinearity is m-term approximation with respect to bases. In
this problem one can use the unique function expansion with respect to a given
basisto build anapproximant.Nonlinearityenters by lookingfor m-termapprox-
imants with terms (i.e., basis elements in approximant) allowedto depend on the
givenfunction. Since the elements of the basis used in the m-term approximation
areallowedtodependonthefunctionbeingapproximated,thistypeofapproxima-
tion is very efficient. This idea is utilized in the method termed the Thresholding
Greedy Algorithm, discussed in detail in this book.
We focus on the following fundamental question. Which bases are suitable
for the use of the Thresholding Greedy Algorithm (TGA)? By answering this
question we introduce several new concepts of bases of a Banach space X: greedy
bases, quasi-greedy bases, almost greedy bases. The greedy bases are the best for
applicationofTGAforsparseapproximation—foranyf ∈X TGAprovidesafter
m iterations approximation with the error of the same order as the best m-term
approximationoff.IfabasisΨisquasi-greedy,thenitmerelyguaranteesthatfor
anyf ∈X TGAprovidesapproximantsthatconvergetof,butdoesnotguarantee
the rate of convergence. This gives rise to the following question: Can we bound
the error of TGA’s m-th approximation by the best m-term approximation with
an extra multiplier, say, C(m)? If yes, what is the best C(m) for a given basis?
The above questions lead to the concept of Lebesgue-type inequalities. We discuss
them in detail.
ix