Table Of ContentFourier and Wavelet Signal Processing
Jelena Kovaˇcevi´c
Carnegie Mellon University
Vivek K Goyal
Massachusetts Institute of Technology
Martin Vetterli
E´cole Polytechnique F´ed´erale de Lausanne
January 17, 2013
Copyright (c) 2013 Jelena Kovaˇcevi´c, Vivek K Goyal, and Martin Vetterli.
These materials are protected by copyright under the
Attribution-NonCommercial-NoDerivs 3.0 Unported License
from Creative Commons.
Contents
Image Attribution ix
Quick Reference xi
Preface xvii
Acknowledgments xxi
1 Filter Banks: Building Blocks of Time-Frequency Expansions 1
1.1 Introduction 2
1.2 OrthogonalTwo-Channel Filter Banks 7
1.2.1 A Single Channel and Its Properties 7
1.2.2 Complementary Channels and Their Properties 10
1.2.3 Orthogonal Two-Channel Filter Bank 11
1.2.4 Polyphase View of Orthogonal Filter Banks 14
1.2.5 Polynomial Approximation by Filter Banks 18
1.3 Design of OrthogonalTwo-Channel Filter Banks 20
1.3.1 Lowpass Approximation Design 20
1.3.2 Polynomial Approximation Design 21
1.3.3 Lattice Factorization Design 25
1.4 BiorthogonalTwo-Channel Filter Banks 26
1.4.1 A Single Channel and Its Properties 29
1.4.2 Complementary Channels and Their Properties 32
1.4.3 BiorthogonalTwo-Channel Filter Bank 32
1.4.4 Polyphase View of Biorthogonal Filter Banks 34
1.4.5 Linear-Phase Two-Channel Filter Banks 35
1.5 Design of BiorthogonalTwo-Channel Filter Banks 37
1.5.1 Factorization Design 37
1.5.2 Complementary Filter Design 39
1.5.3 Lifting Design 40
1.6 Two-Channel Filter Banks with Stochastic Inputs 41
1.7 Computational Aspects 42
1.7.1 Two-Channel Filter Banks 42
1.7.2 Boundary Extensions 46
Chapter at a Glance 49
Historical Remarks 52
Further Reading 52
2 Local Fourier Bases on Sequences 55
2.1 Introduction 56
2.2 N-Channel Filter Banks 59
2.2.1 Orthogonal N-Channel Filter Banks 59
2.2.2 Polyphase View of N-Channel Filter Banks 61
2.3 Complex Exponential-Modulated Local Fourier Bases 66
2.3.1 Balian-Low Theorem 67
2.3.2 Application to Power Spectral Density Estimation 68
2.3.3 Application to Communications 73
2.4 Cosine-Modulated Local Fourier Bases 75
2.4.1 Lapped OrthogonalTransforms 76
2.4.2 Application to Audio Compression 83
2.5 Computational Aspects 85
Chapter at a Glance 87
Historical Remarks 88
Further Reading 90
3 Wavelet Bases on Sequences 91
3.1 Introduction 93
3.2 Tree-Structured Filter Banks 99
3.2.1 The Lowpass Channel and Its Properties 99
3.2.2 Bandpass Channels and Their Properties 103
3.2.3 RelationshipbetweenLowpassandBandpassChannels105
3.3 OrthogonalDiscrete Wavelet Transform 106
3.3.1 Definition of the OrthogonalDWT 106
3.3.2 Properties of the OrthogonalDWT 107
3.4 BiorthogonalDiscrete Wavelet Transform 112
3.4.1 Definition of the BiorthogonalDWT 112
3.4.2 Properties of the BiorthogonalDWT 114
3.5 Wavelet Packets 114
3.5.1 Definition of the Wavelet Packets 115
3.5.2 Properties of the Wavelet Packets 116
3.6 Computational Aspects 116
Chapter at a Glance 118
Historical Remarks 119
Further Reading 119
4 Local Fourier and Wavelet Frames on Sequences 121
4.1 Introduction 122
4.2 Finite-Dimensional Frames 133
4.2.1 Tight Frames for CN 133
4.2.2 General Frames for CN 140
4.2.3 Choosing the Expansion Coefficients 145
4.3 OversampledFilter Banks 151
4.3.1 Tight OversampledFilter Banks 152
4.3.2 Polyphase View of Oversampled Filter Banks 155
4.4 Local Fourier Frames 158
4.4.1 ComplexExponential-ModulatedLocalFourierFrames159
4.4.2 Cosine-Modulated Local Fourier Frames 163
4.5 Wavelet Frames 165
4.5.1 Oversampled DWT 165
4.5.2 Pyramid Frames 167
4.5.3 Shift-Invariant DWT 170
4.6 Computational Aspects 171
4.6.1 The Algorithm `a Trous 171
4.6.2 Efficient Gabor and Spectrum Computation 171
4.6.3 Efficient Sparse Frame Expansions 171
Chapter at a Glance 173
Historical Remarks 174
Further Reading 174
5 Local Fourier Transforms, Frames and Bases on Functions 177
5.1 Introduction 178
5.2 Local Fourier Transform 178
5.2.1 Definition of the Local Fourier Transform 178
5.2.2 Properties of the Local Fourier Transform 182
5.3 Local Fourier Frame Series 188
5.3.1 Sampling Grids 188
5.3.2 Frames from Sampled Local Fourier Transform 188
5.4 Local Fourier Series 188
5.4.1 Complex Exponential-Modulated Local Fourier Bases188
5.4.2 Cosine-Modulated Local Fourier Bases 188
5.5 Computational Aspects 188
5.5.1 Complex Exponential-Modulated Local Fourier Bases188
5.5.2 Cosine-Modulated Local Fourier Bases 188
Chapter at a Glance 188
Historical Remarks 188
Further Reading 188
6 Wavelet Bases, Frames and Transforms on Functions 189
6.1 Introduction 189
6.1.1 Scaling Function and Wavelets from Haar Filter Bank190
6.1.2 Haar Wavelet Series 195
6.1.3 Haar Frame Series 202
6.1.4 Haar Continuous Wavelet Transform 204
6.2 Scaling Function and Wavelets from OrthogonalFilter Banks 208
6.2.1 Iterated Filters 208
6.2.2 Scaling Function and its Properties 209
6.2.3 Wavelet Function and its Properties 218
6.2.4 Scaling Function and Wavelets from Biorthogonal
Filter Banks 220
6.3 Wavelet Series 222
6.3.1 Definition of the Wavelet Series 223
6.3.2 Properties of the Wavelet Series 227
6.3.3 Multiresolution Analysis 230
6.3.4 BiorthogonalWavelet Series 239
6.4 Wavelet Frame Series 242
6.4.1 Definition of the Wavelet Frame Series 242
6.4.2 Frames from Sampled Wavelet Series 242
6.5 Continuous Wavelet Transform 242
6.5.1 Definition of the Continuous Wavelet Transform 242
6.5.2 ExistenceandConvergenceoftheContinuousWavelet
Transform 243
6.5.3 Properties of the Continuous Wavelet Transform 244
6.6 Computational Aspects 254
6.6.1 Wavelet Series: Mallat’s Algorithm 254
6.6.2 Wavelet Frames 259
Chapter at a Glance 259
Historical Remarks 259
Further Reading 259
Bibliography 265
Quick Reference
Abbreviations
AR Autoregressive
ARMA Autoregressive moving average
AWGN Additivewhite Gaussian noise
BIBO Bounded input,bounded output
CDF Cumulative distribution function
DCT Discrete cosine transform
DFT Discrete Fourier transform
DTFT Discrete-time Fourier transform
DWT Discrete wavelet transform
FFT Fast Fourier transform
FIR Finite impulse response
i.i.d. Independentand identically distributed
IIR Infiniteimpulse response
KLT Karhunen–Lo`evetransform
LOT Lapped orthogonal transform
LPSV Linear periodically shift varying
LSI Linear shift invariant
MA Moving average
MSE Mean square error
PDF Probability density function
POCS Projection ontoconvex sets
ROC Region of convergence
SVD Singular value decomposition
WSCS Wide-sense cyclostationary
WSS Wide-sense stationary
Abbreviations used in tables and captions but not in the text
FT Fourier transform
FS Fourier series
LFT Local Fourier transform
WT Wavelet transform
Elements of Sets
natural numbers N 0, 1, ...
integers Z ..., 1, 0, 1, ...
positive integers Z+ 1, 2,−...
real numbers R ( , )
positive real numbers R+ (0−,∞)∞
complex numbers C a+∞jb or rejθ with a,b,r,θ R
∈
a generic index set
I
a generic vector space V
a generic Hilbert space H
real part of ( )
ℜ ·
imaginary part of ( )
ℑ ·
closure of set S S
functions x(t) argument t is continuousvalued, t R
sequences x argument n is an integer, n Z ∈
n
∈
ordered sequence (x )
n n
set containing x x
n n n
{ }
vectorx with x as elements [x ]
n n
∞
Dirac delta function δ(t) x(t)δ(t)dt=x(0)
Kroneckerdelta sequence δ Zδ−∞=1 for n=0; δ =0 otherwise
n n n
indicator function of interval I 1 (t) 1 (t)=1 for t I; 1 (t)=0 otherwise
I I I
∈
Elements of Real Analysis
integration by parts udv=uv vdu
−
Z Z
Elements of Complex Analysis
complex number z a+jb, rejθ, a,b R, r [0, ),θ [0,2π)
∈ ∈ ∞ ∈
conjugation z∗ a jb, re−jθ
−
conjugation of coefficients X (z) X∗(z∗)
∗
but not of z itself
principal root of unity WN e−j2π/N
Asymptotic Notation
big O x O(y) 0 x γy for all n n ; some n and γ >0
n n 0 0
∈ ≤ ≤ ≥
little o x o(y) 0 x γy for all n n ; some n , any γ >0
n n 0 0
∈ ≤ ≤ ≥
Omega x Ω(y) x γy for all n n ; some n and γ >0
n n 0 0
∈ ≥ ≥
Theta x Θ(y) x O(y) and x Ω(y)
∈ ∈ ∈
asymptotic equivalence x y lim x /y =1
n n n
≍ →∞
Standard Vector Spaces
Hilbert space of square-summable ℓ2(Z) x:Z C x 2 < with
n
( → | | | ∞)
n
X
sequences innerproduct hx, yi= xnyn∗
n
X
Hilbert space of square-integrable 2(R) x:R C x(t)2dt< with
L → | | | ∞
(cid:26) Z (cid:27)
functions innerproduct x, y = x(t)y(t)∗dt
h i
Z
normed vector space of sequenceswith ℓp(Z) x:Z C x p < with
n
( → | | | ∞)
n
finite p norm, 1 p< norm x =(Xx p)1/p
p n
≤ ∞ k k | |
n
X
normed vector space of functions with p(R) x:R C x(t)pdt< with
L → | | | ∞
(cid:26) Z (cid:27)
finite p norm, 1 p< norm x =( x(t)pdt)1/p
p
≤ ∞ k k | |
Z
normed vector space of boundedsequences with ℓ∞(Z) x:Z C sup xn < with
→ | | | ∞
(cid:26) n (cid:27)
supremum norm norm x =sup x
n
k k∞ n | |
normed vector space of boundedfunctions with ∞(R) x:R C sup x(t) < with
L → | | | ∞
(cid:26) t (cid:27)
supremum norm norm x =sup x(t)
k k∞ t | |
Bases and Frames for Sequences
standard Euclidean basis e e =1, for k=n, and 0 otherwise
n n,k
{ }
vector, element of basis or frame ϕ when applicable, a column vector
basis or frame Φ set of vectors ϕ
n
{ }
operator Φ concatenation of ϕ sin a linear
n
operator: [ϕ ϕ ... ϕ ]
0 1 N 1
−
vector, element of dual basis or frame ϕ when applicable, a column vector
Φ set of vectors ϕ
n
{ }
operator Φe concatenation of ϕnsin a linear
e operator: [ϕ0 ϕe1 ... ϕN 1]
−
expansion in a basis or frame xe=ΦΦ∗x e
e e e
e
Transforms
Fourier transform x(t) FT X(ω) X(ω)= ∞ x(t)e−jωtdt
←→
x(t)= 1Z−∞∞ X(ω) ejωtdω
2π
Z−∞
Fourier series x(t)←F→S Xk Xk = T1 T/2 x(t)e−j(2π/T)ktdt
x(t)= Z−XT/2ej(2π/T)kt
k
Xk∈Z
discrete-time Fourier transform xnDTFTX(ejω) X(ejω)= xne−jωn
←→
1 nX∈πZ
x = X(ejω)ejωndω
n 2π
Z−π
N 1
discrete Fourier transform x DFT X X = − x Wkn
n←→ k k n N
n=0
XN 1
xn= N1 − Xk WN−kn
n=0
X
local Fourier transform x(t) LFT X(Ω,τ) X(Ω,τ)= ∞ x(t)p(t τ)e−jΩtdt
←→ −
1 Z−∞∞ ∞
x(t)= X(Ω,τ)g (t)dΩdτ
2π Ω,τ
Z−∞Z−∞
continuouswavelet transform x(t)CWTX(a,b) X(a,b)= ∞ x(t)ψ (t)dt
a,b
←→
1 Z−∞∞ ∞ dbda
x(t)= X(a,b)ψ (t)
C a,b a2
ψ Z0 Z−∞
wavelet series x(t) WS β(ℓ) β(ℓ) = ∞ x(t)ψ (t)dt
←→ k k ℓ,k
x(t)=Z−∞ β(ℓ)ψ (t)
k ℓ,k
Xℓ∈ZXk∈Z
discrete wavelet transform x DWTα(J),β(J),...,β(1) α(J) = x g(J) ,β(ℓ) = x h(ℓ)
n ←→ k k k k n n 2Jk k n n 2ℓk
nX∈Z − J nX∈Z −
x = α(J)g(J) + β(ℓ)h(ℓ)
n k n 2Jk k n 2ℓk
Xk∈Z − Xℓ=1Xk∈Z −
N 1
discrete cosine transform x DCT X X = 1 − x
n←→ k 0 N n
r n=0
NX1
2 −
X = x cos((2π/2N)k(n+1/2))
k N n
r n=0
NX1
1 −
x = X
0 N k
r Xk=0
N 1
2 −
x = X cos((2π/2N)k(n+1/2))
n N k
r Xk=0
z-transform xn ZT X(z) X(z)= xnz−n
←→
nX∈Z
Discrete-Time Nomenclature
Sequence x signal, vector
n
Convolution
linear h x x h = h x
k n k k n k
∗ − −
XNk∈Z1 Xk∈Z N 1
− −
circular h⊛x x h = h x
k (n k)modN k (n k)modN
− −
Xk=0 Xk=0
(h x) nth element of theconvolution result
n
∗
h x x h
ℓ n n n m k m ℓ n+k
− ∗ − − −
Xk∈Z
Eigensequence v eigenfunction, eigenvector
n
infinite time v =ejωn h v=H(ejω)v
n
∗
finite time v =ej2πkn/N h⊛v=H v
n k
Frequency response eigenvalue corresponding to v
n
infinite time H(ejω) hne−jωn
nNX∈Z1 N 1
finite time Hk − hne−j2πkn/N = − hnWNkn
n=0 n=0
X X
Continuous-Time Nomenclature
Function x(t) signal
Convolution
∞ ∞
linear h x x(τ)h(t τ)dτ = h(τ)x(t τ)dτ
∗ − −
Z−T∞ Z−T∞
circular h⊛x x(τ)h(t τ)dτ = h(τ)x(t τ)dτ
− −
Z0 Z0
(h x)(t) convolution result at t
∗
Eigenfunction v(t) eigenvector
infinite time v(t)=ejωt h v=H(ω)v
∗
finite time v(t)=ej2πkt/T h⊛v=H v
k
Frequency response eigenvalue corresponding to v(t)
infinite time H(ω) ∞ h(t)e−jωtdt
Z−T∞/2
finite time Hk h(τ)e−j2πkτ/Tdτ
Z−T/2
