Table Of Content

Contents Preface xi Acknowledgments xiii 1. The Age of Wavelets 1 - 13 Introductioq 1 1.1 The Origins of Wavelets-Are They Fundamentally New? I 1.2 Wavelets and Other Reality Transforms 3 1.3 Managing Heisenberg's Uncertainty Ghost 5 1.4 History of Wavelet from Morlet to Daubechies Via Mallat 6 1.4.1 Different Communities of Wavelets 9 1.4.2 Different Families of Wavelets within Wavelet Communities 10 1.4.3 Interesting Recent Developments I1 1.5 Wavelets in the Future 12 Summary 13 References 13 2. Fourier Series and Geometry 14-30 introduction 14 2.1 Vector Space 15 2.1.1 Bases 15 2.1.2 Orthonormality 15 2.1.8 Projection 15 2.2 Functions and Function Spaces 16 2.2.1 Orthogonal Functions 16 2.2.2 Orthonormal Functions I7 2.2.3 Function Spaces 17 vi Contents 2.2.4 Orthogonal Basis Functions 20 2.2.5 Orthonormality and the Method of Finding the Coeficients 20 2.2.6 Complex Fourier Series 24 2.2.7 Orthogonality of Complex Exponential Bases 25 Sunurtary 27 Exercises 28 References 30 3. Continuous Wavelet and Short Time Fourier Transform 31-47 Introduction 3 1 3.1 Wavelet Transform-A First Level Introduction 31 3.2 Mathematical Preliminaries-Fourier Transform 35 3.2.1 Continuous Time-Frequency Representation of Signals 37 3.2.2 The Windowed Fourier Transform (Short Time Fourier Transform) 38 3.2.3 The Uncertainty Principle and Time Frequency Tiling 39 3.3 Properties of Wavelets Used in Continuous Wavelet Transform 43 3.4 Continuous Versus Discrete Wavelet Transform 43 Summary 45 Exercises 46 References 47 -- 4. Discrete Wavelet Transform 48-72 lntmduction 48 4.1 Haar Scaling Functions and Function Spaces 48 4.1.1 Translation and Scaling of #(r) 49 4.1.2 Orthogonality of Translates of fit) 50 4.1.3 Function Space Vo 51 4.1.4 Finer Haar Scaling Functions 53 4.2 Nested Spaces 54 4.3 Haar Wavelet Function 55 4.3.1 Scaled Haar Wavelet Functions 57 4.4 Orthogonality of $(t) and ~ ( t )62 4.5 Normalization of Haar Bases at Different Scales 63 4.6 Standardizing the Notations 65 4.7 Refinement Relation with Respect to Normalized Bases 65 4.8 Support of a Wavelet System 66 4.8.1 Triangle Scaling Function 67 4.9 Daubechies Wavelets 68 4.10 Seeing the Hidden-Plotting the Daubechies Wavelets 69 Summary 71 Exercises 71 References 72 i Contents vii 5. D- esigning Orthogonal Wavelet S ystems-A Direct Approach 73-85 Ir~troduction 73 5.1 Refinement Relation for Orthogonal Wavelet Systems 73 5.2 Restrictions on Filter Coefficients 74 5.2.1 Condition 1: Unit Area Under Scaling Function 74 5.2.2 Condition 2: Orthonormality of Translates of Scaling Functions 75 5.2.3 Condition 3: Orthonormality of Scaling and Wavelet Functions 76 5.2.4 Condition 4: Approximation Conditions (Smoothness Conditions) 77 5.3 Designing Daubechies Orthogonal Wavelet System Coefficients 79 5.3.1 Constraints for Daubechies' 6 Tap Scaling Function 80 5.4 Design of Coiflet wavelets 81 5.5 Symlets 82 Sunlmary 83 Exercises 83 Reference? 85 6. Discrete WaveIet Transform and Relation to Filter Banks 86-101 Introduction 86 6.1 Signal Decomposition (Analysis) 86 6.2 Relation with Filter Banks 89 6.3 Frequency Response 94 6.4 Signal Reconstruction: Synthesis from Coarse Scale to Fine Scale 96 6.4.1 Upsampling and Filtering 97 6.5 Perfect Matching Filters 98 6.6 Computing Initial sj+, Coefficients 100 Summary 100 Exercises 100 References 101 7. Generating and Plotting of Parametric Wavelets 102-124 lntroduction 102 7.1 Orthogonality Conditions and Parameterization 102 7.2 Polyphase Matrix and Recurrence Relation 104 7.2.1 Maple Code 106 7.2.2 Parameterization of Length-6 Scaling Filter 107 7.2.3 Parameterization of Length-8 Scaling Filter 107 7.2.4 Parameterization of Length-10 Scaling Filter 108 7.3 Pollen-type Parameterizations of Wavelet Bases 109 7.4 Precise Numerical Evaluation of # and y 110 7.4.1 Method 1. Cascade Algorithm: Direct Evaluation at Dyadic I Rational Points 110 7.4.2 Method 2. Successive Approximation 114 7.4.3 Method 3. Daubechies-Lagarias Algorithm 1 17 7.4.4 Method 4. Subdivision Scheme 120 Sumniary 123 Exercises 123 References 124 viii Contents 8. Biort hogonal Wavelets 125-140 introdircrior~ 125 8.1 Biorthogonality in Vector Space 125 1 8.2 B~orthoeonalW avelet Systems 127 8.3 Signal Representation Using Biorthogonal Wavelet System 129 8.4 B~orthogonalA nalysis 129 8.5 Biorthogonal Syn thesis-From Coarse Scale to Fine Scale 131 8.6 Construction of Biorthogonal Wavelet Systems 132 8.6.1 B-splines 132 8.6.2 B-spline Biorthogonal Wavelet System or Cohen-Daubechies- Feauveau Wavelets (CDF) 134 S~tmnzary 139 Exercises 139 References 140 9. Designing Wavelets-Frequency Domain Approach 141-152 Introd~iction 141 - 9.1 Basic. Properties of Filter Coefficients 141 9.2 Choice of Wavelet Function Coefficients (g(k)) 144 9.3 Vanishing Moment Conditions in Fourier Domain 146 9.4 Derivation of Daubechies Wavelets 147 9.4.1 Daubechies Wavelets with One Vanishing Moment 151 9.4.2 Daubechies Wavelets with Two Vanishing Moment 151 Surnmnry 152 Exercises 152 - References 152 10. Lifting Scheme 153-195 - Introduction 153 10.1 Wavelet Transform Using Polyphase Matrix Factorization 154 -- 10.1. l Inverse Lifting 158 10.1.2 Example: Forward Wavelet Transform 159 10.2 Geometrical Foundations of Lifting Scheme 161 - 10.2.1 Haar and Lifting 163 10.2.2 The Lifting Scheme 163 10.2.3 The Linear Wavelet Transform Using Lifting 167 - 10.2.4 Higher Order Wavelet Transform 170 10.3 Lifting Scheme in the 2-Domain 171 10.3.1 Design Example 1 176 10.3.2 Example 2: Lifting Haar Wavelet 180 10.4 Mathematical Preliminaries for Polyphase Factorization 182 10.4.1 Laurent Polynom~al 182 10.4.2 The Eucl~deanA lgor~thm 153 1 ift tin^ 10.4.3 Factoring Wavelet Transform into Saps-A Z-domain *Approach 186 Contents ix 10.5 Dealing with Signal Boundary I Yl 10.5.1 Circular Convolution 191 10.5.2 Padding Policies 192 10.5.3 Iteration Behaviour 193 Sulnnlary 193 Exercises 194 References 195 11. Image Compression 196-220 Irltroductiort 196 11 .1 Overview of Image Compression Techniques 197 1 1.1.1 The JPEG-Standard (ITU T.8 1) 199 - 11.2 Wavelet Transform of an Image 200 11.3 Quantization 203 11 .3.1 Uniform Quantization 203 - 11 .3.2 Subband Uniform Quantization 204 11.3.3 Uniform Dead-Zone Quantization 205 11 .3.4 Non-Uniform QuantizaJion 205 - 11.4 Entropy Encoding 206 1 1.4.1 Huffman Encoding 206 11.4.2 Run Length Encoding 207 -. 11.5 EZW Coding (Embedded Zero-tree Wavelet Coding) 208 11.5.1 EZW Performance 217 - 11.6 SPIHT (Set Partitioning in Hierarchical Tree) 218 11.7 EBCOT (Embedded Block Coding with Optimized Truncation) 218 Surnrnnry 218 - Weblems (Web based problents) 219 References 21 9 - 12. Denoising 221-237 Introduction 22 1 - 12.1 A Simple Explanation and a 1-D Example 221 12.2 Denoising Using Wavelet Shrinkage-Statistical Modelling and Estimation 222 - 12.3 Noise Estimation 223 12.4 Shrinkage Functions 225 - 12.5 Shrinkage Rules 227 12.5.1 Universal 227 12.5.2 Minimizing the False Discovery Rate 227 - 12.5.3 Top 228 12.5.4 Sure 228 12.5.5 Translation Invariant Thresholding 229 - 12.5.6 BayesShrink 229 12.6 Denoising Images with Matlab 230 x Contents 12.7 Matlab Prozrams for Denoising 232 12.8 Simulation for Finding Effectiveness of Thresholding Method 233 235 Sltlmlnn I E.rercises 236 References 237 1 13. Spline Wavelets: Introduction and Applications to Computer Graphics 238-283 13.1 Introduction To Curves and Surfaces 238 13.1.1 Spline Curves and Surfaces 241 13.1.2 Cubic Spline Interpolation Methods 245 13.1.3 Hermite Spline Interpolation 243' 13.1.4 Cubic Splines 247 13.1.5 Splines in Signal and Image Processing 249 13.1.6 Bezier Curves and Surfaces 250 13.1.7 Properties of Bezier Curves 252 13.1.8 Quadratic and Cubic Bezier Curves 253 13.1.9 Parametric Cubic Surfaces 255 13.1.10 Bezier Surfaces 257 13.1.1 1 B-Spline Curves 258 13.1.12 Cubic Periodic B-Splines 260 I 13.1.13 Conversion between Hermite, Bezier and B-Spline Representations 261 13.1.14 Non-Uniform B-Splines 262 13.1.15 Relation between Spline, Bezier and B-Spline Curves 264 13.1.16 B-Spline Surfaces 265 13.1.17 Beta-Splines and Rational Splines 265 13.2 Multiresolution Methods and Wavelet Analysis 266 13.3 The Filter Bank 268 13.3 Orthogonal and Semi-orthogonal Wavelets 269 13.5 Spline Wavelets 270 13.6 Properties of Spline Wavelets 27-3 13.7 Advantages of B-Spline Based Wavelets in Signal Processing Applications 275 13.8 B-Spline Filter Bank 276 13.9 Multiresolution Curves and Faces 276 13.10 Wavelet Based Curve and Surface Editing 277 13.1 1 Variational Modelling and the Finite Element Method 278 13.12 Adaptive Variational Modelling Using Wavelets 281 Szrrnnzar?) 282 Refererrces 282 Index Preface In the past few years, the study of wavelets and the exploration of the principles governing their behaviour have brought about sweeping changes in the disciplines of pure and applied mathematics and sciences. One of the most significant development is the realization that, in addition to the canonical tool of representing a function by its Fourier series, there is a different representation more adapted to certain problems in data compression, noise removal, pattern clrtssification and fast scientific computation. ! Many books are available on wavelets but most of them are written at such a level that only research mathematicians can avail them. The purpose of this book is to make wavelets accessible to anyone (for example, graduate and undergraduate students) with a modest background in basic linear algebra and to serve as an introduction for the non-specialist. The level of the applications and the format of this book are such as to make this suitable as a textbook for an introductory course on wavelets. Chapter 1 begins with a brief note on the origin of wavelets, mentioning the main early contributors who laid the foundations of the theory and on the recent developments and applications. Chapter 2 introduces the basic concepts in Fourier series and orients the reader to look at everything found in the Fourier kingdom from a geometrical point of view. In Chapter 3, the focus is on the continuous wavelet transform and its relation with short time Fourier transform. Readers who have not had much exposure to Fourier transforms earlier may skip this chapter, which is included only for the purpose of completeness. Chapter 4 places the wavelet theory in a concrete setting using the Haar scaling and wavelet function. The rest of the book builds on this material. To understand the concepts in this chapter fully, the reader need to have only an understanding of the basic concepts in lipear algebra: addition and multiplication of vectors by scalars, linear independence and dependence, orthogonal bases, basis set, vector spaces and function spaces and projection of vector/function on to the bases. The chapter introduces the concept of nested spaces, which is the corner stone of rnultiresolution analysis. Daubechies' wavelets are also introduced. he' chapter concludes with a note on the fact that most of the wavelets are fractal in nature and that iterative methods are requ~redr o display uavelets. Designing wavelets is traditionally carried out in the Fourier domain. Readers who are not experts in Fourier analysis usually find the theoretical arguments and terminology used quite xi xii Perface baffling and totally out of the world. This book, therefore, adopts a time domain approach to designing. The orthogonality and smoothness/regularity constraints are directly mapped on to constraints on the scaling and wavelet filter coefficients, which can then be solved using solvers available in Microsoft Excel-a spreadsheet package or a scientific computation package like MATLAB or Mathematica. Engineers often view signal processing in terms of filtering by appropriate filters. Thus, Chapter 6 is devoted to establish the relationship between 'signal expansion in terms of wavelet bases' and the 'filter bank' approach to signal analysis/synthesis. Chapter 7 discusses the theory behind parametric wavelets in an intuitive way rather than by using rigorous mathematical approach. The chapter also discusses various methods of plotting scaling and wavelet functions. The focus of Chapter 8 is on biorthogonal wavelets which is relatively a new concept. To drive home this concept to the readers, biorthogonality is explained using linear algebra. The chapter then goes on to discuss the design of elementary B-spline biorthogonal wavelets. Chapter 9 addresses orthogonal wavelet design using the Fourier domain approach. Chapter 10 is devoted to the lifting scheme which provides a simple means to design wavelets with the desirable properties. The chapter also shows how the lifting scheme allows faster implementation of wavelet decomposition/reconstruction. Chapter 11 to 13 describle applications of wavelets in Image Compression, Signal Denoising and Computer Graphics. The notations used in Chapter 13 are that used by researchers in this particular area and could be slightly different from those in the rest of the chapters. To make the book more useful to the readers, we propose to post the teaching material (mainly Powerpoint slides for each chapter, and MATLABIExcel demonstration programs) at the companion website of the book: www.amrita.edu/cen/publications/wavelets. We earnestly hope that this book will initiate several persons to this exciting and - vigorously growing area. Though we have spared no pains to make this book free from mistakes, some errors may still have survived our scrutiny. We gladly welcome all corrections, recommendations, suggestions and constructive criticism irom our readers. - K.P. SOMAN K.I. RAMACHANDRAN Acknowledgments First and foremost, we would like to express our gratitude to Brahmachari Abhayarnrita Chaitanya, who persuaded us to take this project, and never ceased to lend his encouragement and support. We thank Dr. P. Venkat Rangan, Vice Chancellor of the university and Dr. K.B.M. Nambudiripad, Dean (Researchj-our guiding stars-who continually showed us what perfection means and demanded perfection in everything that we did. We would like to thank, especially, Dr. P. Murali Knshna, a scientist at NPOL, Cochin, for his endearing support during the summer school on 'Wavelets Fractals and Chaos' that we conducted in 1998. It was then that we learned wavelets seriously. We take this opportunity to thank our research students C.R. Nitya, Shyam Divakar, Ajith Peter, Santhana Krishnan and V. Ajay for their help in simplifying the concepts. G. Sreenivasan and S. Sooraj, who helped us in drawing the various figures in the textbook, deserve a special thanks. Finally, we express our sincere gratitude to the editors of Prentice-Hall of India. - K.P. SOMAN K.I. RAMACHANDRAN xiii