Table Of Content

Lecture Notes in Applied and Numerical Harmonic Analysis Vasily N. Malozemov Sergey M. Masharsky Foundations of Discrete Harmonic Analysis Applied and Numerical Harmonic Analysis Lecture Notes in Applied and Numerical Harmonic Analysis Series Editor John J. Benedetto University of Maryland College Park, MD, USA Editorial Board Emmanuel Candes Stanford University Stanford, CA, USA Peter Casazza University of Missouri Columbia, MO, USA Gitta Kutyniok Technische Universität Berlin Berlin, Germany Ursula Molter Universidad de Buenos Aires Buenos Aires, Argentina Michael Unser Ecole Polytechnique Federal De Lausanne Lausanne, Switzerland More information about this subseries at http://www.springer.com/series/13412 Vasily N. Malozemov Sergey M. Masharsky (cid:129) Foundations of Discrete Harmonic Analysis Vasily N.Malozemov Sergey M.Masharsky Mathematics andMechanicsFaculty Mathematics andMechanicsFaculty Saint PetersburgState University Saint PetersburgState University Saint Petersburg, Russia Saint Petersburg, Russia ISSN 2296-5009 ISSN 2296-5017 (electronic) AppliedandNumerical Harmonic Analysis ISSN 2512-6482 ISSN 2512-7209 (electronic) Lecture Notesin AppliedandNumerical HarmonicAnalysis ISBN978-3-030-47047-0 ISBN978-3-030-47048-7 (eBook) https://doi.org/10.1007/978-3-030-47048-7 MathematicsSubjectClassification(2010): 42C10,42C20,65D07,65T50,65T60 ©SpringerNatureSwitzerlandAG2020 Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseof illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar ordissimilarmethodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained hereinorforanyerrorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregard tojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations. This book is published under the imprint Birkhäuser, www.birkhauser-science.com by the registered companySpringerNatureSwitzerlandAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland LN-ANHA Series Preface TheLectureNotesinAppliedandNumericalHarmonicAnalysis(LN-ANHA)book series is a subseries of the widely known Applied and Numerical Harmonic Analysis (ANHA) series. The Lecture Notes series publishes paperback volumes, ranging from 80 to 200 pages in harmonic analysis as well as in engineering and scientific subjects having a significant harmonic analysis component. LN-ANHA providesameansofdistributingbrief-yet-rigorousworksonsimilarsubjectsasthe ANHAseriesinatimelyfashion,reflectingthemostcurrentresearchinthisrapidly evolving field. The ANHA book series aims to provide the engineering, mathematical, and scientificcommunitieswithsignificantdevelopmentsinharmonicanalysis,ranging fromabstractharmonicanalysistobasicapplications.Thetitleoftheseriesreflects the importance of applications and numerical implementation, but richness and relevance of applications and implementation depend fundamentally on the struc- ture and depth of theoretical underpinnings. Thus, from our point of view, the interleaving of theory and applications and their creative symbiotic evolution is axiomatic. Harmonic analysis is a wellspring of ideas and applicability that has flourished, developed, and deepened over time within many disciplines and by means of creative cross-fertilization with diverse areas. The intricate and fundamental rela- tionship between harmonic analysis and fields such as signal processing, partial differential equations (PDEs), and image processing is reflected in our state-of-the-art ANHA series. Our vision of modem harmonic analysis includes mathematical areas such as wavelet theory, Banach algebras, classical Fourier analysis, time-frequency analysis, and fractal geometry, as well as the diverse topics that impinge on them. For example, wavelet theory can be considered an appropriate tool to deal with some basic problems in digital signal processing, speech and image processing, geophysics, pattern recognition, bio-medical engineering, and turbulence. These areas implement the latest technology from sampling methods on surfaces to fast algorithms and computer vision methods. The underlying mathematics of wavelet theorydependsnotonlyonclassicalFourieranalysisbutalsoonideasfromabstract v vi LN-ANHASeriesPreface harmonic analysis, including von Neumann algebras and the affine group. This leadstoastudyoftheHeisenberggroupanditsrelationshiptoGaborsystemsand of the metaplectic group for a meaningful interaction of signal decomposition methods. Theunifyinginfluenceofwavelettheoryintheaforementionedtopicsillustrates the justification for providing a means for centralizing and disseminating infor- mationfromthebroader,butstillfocused,areaofharmonicanalysis.Thiswillbea keyroleofANHA.Weintendtopublishwiththescope andinteraction thatsucha host of issues demands. Along with our commitment to publish mathematically significant works at the frontiersofharmonicanalysis,wehaveacomparablystrongcommitmenttopublish majoradvancesinapplicabletopicssuchasthefollowing,whereharmonicanalysis plays a substantial role: Bio-mathematics, bio-engineering, Machine learning; and bio-medical signal processing; Phaseless reconstruction; Communications and RADAR; Quantum informatics; Compressive sensing (sampling) Remote sensing; and sparse representations; Sampling theory; Data science, data mining Spectral estimation; and dimension reduction; Time-frequency and time-scale Fast algorithms; analysis–Gabor theory Frame theory and noise reduction; and wavelet theory Image processing and super-resolution; The above point ofview for the ANHA book series isinspiredby the history of Fourier analysis itself, whose tentacles reach into so many fields. In the last two centuries Fourier analysis has had a major impact on the development of mathematics, on the understanding of many engineering and scientific phenomena,and on the solution of some of themost importantproblems in mathematics and the sciences. Historically, Fourier series were developed in the analysis of some of the classical PDEs of mathematical physics; these series were usedtosolvesuchequations.InordertounderstandFourierseriesandthekindsof solutions they could represent, some of the most basic notions of analysis were defined, for example, the concept of “function.” Since the coefficients of Fourier series are integrals, it is no surprise that Riemann integrals were conceived to deal with uniqueness properties of trigonometric series. Cantor’s set theory was also developed because of such uniqueness questions. A basic problem in Fourier analysis is to show how complicated phenomena, suchassoundwaves,canbedescribedintermsofelementaryharmonics.Thereare twoaspectsofthisproblem:first,tofind,orevendefineproperly,theharmonicsor spectrumofagivenphenomenon,e.g.,thespectroscopyprobleminoptics;second, LN-ANHASeriesPreface vii to determine which phenomena can be constructed from given classes of har- monics, as done, for example, by the mechanical synthesizers in tidal analysis. Fourier analysis is also the natural setting for many other problems in engineering, mathematics, and the sciences. Forexample, Wiener’s Tauberian theorem in Fourier analysis not only characterizes the behavior of the prime numbers but alsoprovidesthepropernotionofspectrumforphenomenasuchaswhitelight;this latter process leads to the Fourier analysis associated with correlation functions in filtering and prediction problems, and these problems, in turn, deal naturally with Hardy spaces in the theory of complex variables. Nowadays, some of the theory of PDEs has given way to the study of Fourier integral operators. Problems in antenna theory are studied in terms of unimodular trigonometric polynomials. Applications of Fourier analysis abound in signal processing, whether with the fast Fourier transform (FFT), or filter design, or the adaptive modeling inherent in time-frequency-scale methods such as wavelet theory. The coherent states of mathematical physics are translated and modulated Fourier transforms, and these are used, in conjunction with the uncertainty principle,fordealingwithsignalreconstructionincommunicationstheory.Weareback to the raison d’être of the ANHA series! University of Maryland John J. Benedetto College Park, MD, USA Series Editor Preface Discrete harmonic analysis is a mathematical discipline predominately targeted to advanced applications of digital signal processing. A notion of a signal requires a closer definition. A signal in discrete harmonic analysis is defined as a complex-valued periodic function of an integer argument. Inthis bookwe study transforms of signals. One of thefundamentaltransforms isthediscreteFouriertransform(DFT).In1965,CooleyandTukeyintheirpaper [8] proposed the fast Fourier transform (FFT), a fast method of calculation of the DFT.Essentially,thisdiscoverysetthestagefordevelopmentofdiscreteharmonic analysis as a self-consistent discipline. TheDFTinversionformulacausesasignaltobeexpandedovertheexponential basis.Expandingasignalovervariousbasesisthemaintechniqueofdigitalsignal processing. An argument of a signal is interpreted as time. Components of the discrete Fourier transform comprise a frequency spectrum of a signal. Analysis in the time andfrequencydomainletsusuncoverthestructureofasignalanddetermineways of transforming a signal to obtain required properties. In practice, we are faced with a necessity to process signals of various natures suchasacoustic,television,seismic,radiosignals,orsignalscomingfromtheouter space.Thesesignalsarereceivedbyphysicaldevices.Whenwetakeareadingofa device at regular intervals we obtain a discrete signal. It is this signal that is a subject offurther digital processing. To start with, we calculate a frequency spec- trumofthediscretesignal.Itcorrespondstorepresentingasignalinaformofasum of simple summands being its frequency components. By manipulating with frequency components we achieve an improvement of specific features of a signal. Thisbookisaimedataninitialacquaintancewiththesubject.Itiswrittenonthe basisofthelecturecoursethatthefirstauthorhasbeendeliveringsince1995onthe Faculty of Mathematics and Mechanics of St. Petersburg State University. Thebookconsistsoffourchapters.Thefirstchapterbrieflyexposesthefactsthat are being used in the main text. These facts are well known and are related to residuals, permutations, complex numbers, and finite differences. ix x Preface In the second chapter we consider basic transforms of signals. The centerpieces arediscreteFouriertransform,cyclicconvolution,andcycliccorrelation.Westudy the properties of these transforms. As an application, we provide solutions to the problems of optimal interpolation and optimal signal–filter pair. Separate sections are devoted to ensembles of signals and to the uncertainty principle in discrete harmonic analysis. In the third chapter we introduce discrete periodic splines and study their fun- damentalproperties.Weestablishanextremalpropertyoftheinterpolationsplines. In terms of splines, we offer an elegant solution to the problem of smoothing of discreteperiodicdata.Weconstructasystemoforthogonalsplines.Withtheaidof dual splines, we solve the problem of spline processing of discrete data with the least squares method. We obtain a wavelet expansion of an arbitrary spline. We prove two limit theorems related to interpolation splines. Thefocusofthefourthchapterisonfastalgorithms:thefastFouriertransform, thefastHaartransform,andthefastWalshtransform.Tobuildafastalgorithmwe use an original approach stemming from introduction of a recurrent sequence of orthogonalbasesinthespaceofdiscreteperiodicsignals.Inthiswaywemanageto formwaveletbaseswhichaltogetherconstituteawaveletpacket.Inparticular,Haar basis is a wavelet one. We pay a lot of attention to it in the book. We investigate an important question of ordering of Walsh functions. We ana- lyze in detail Ahmed–Rao bases that fall in between Walsh basis and the expo- nential basis. The main version of the fast Fourier transform (it is called the Cooley–Tukey algorithm) is targeted to calculate the DFT whose order is a power of two. At the end of the fourth chapter, we show how to use the Cooley–Tukey algorithm to calculate a DFT of any order. A specific feature of the book is a big number of exercises. They allow us to lessentheburdenofthemaintext.Manyspecialandauxiliaryfactsareformalized as exercises. All the exercises are endowed with solutions. Separate exercises or exercisegroupsareindependent,soyouasareadercanselectonlythosethatseem interestingtoyou.Themostefficientwayissolving anexerciseandthenchecking your solution against the one presented in the book. It will let you actively master the matter. At the end of the book we put the list of references. We lay emphasis on the books [5, 41, 49] that we used to study up the fundamentals of discrete harmonic analysis back in the day. The first version ofthe book was publishedin 2003 asa preprint.In 2012 Lan’ publishers published the book in Russian. This English edition is an extended and improved version of the Russian edition.