Linear Algebra, Signal Processing, and Wavelets - A Unified Approach: MATLAB Version PDF
Preview Linear Algebra, Signal Processing, and Wavelets - A Unified Approach: MATLAB Version
Springer Undergraduate Texts in Mathematics and Technology Øyvind Ryan Linear Algebra, Signal Processing, and Wavelets - A Unifi ed Approach MATLAB Version Springer Undergraduate Texts in Mathematics and Technology SeriesEditors: HelgeHolden,NorwegianUniversityofScienceandTechnology,Trondheim,Norway KeriA.Kornelson,UniversityofOklahoma,Norman,OK,USA EditorialBoard: LisaGoldberg,UniversityofCalifornia,Berkeley,CA,USA ArminIske,UniversityofHamburg,Hamburg,Germany PalleE.T.Jorgensen,TheUniversityofIowa,IowaCity,IA,USA Springer UndergraduateTextsin MathematicsandTechnology(SUMAT)publishestext- books aimed primarily at the undergraduate. Each text is designed principally for students who are considering careers either in the mathematical sciences or in technology-based areas such as engineering,finance, informationtechnologyand computerscience, bioscience andmedicine,optimizationorindustry.Textsaimtobeaccessibleintroductionstoawiderange ofcoremathematicaldisciplinesandtheirpractical,real-worldapplications;andarefashioned bothforcourseuseandforindependentstudy. Moreinformationaboutthisseriesathttp://www.springer.com/series/7438 Øyvind Ryan Linear Algebra, Signal Processing, and Wavelets - A Unified Approach MATLAB Version 123 ØyvindRyan DepartmentofMathematics UniversityofOslo Oslo,Norway TheauthorhasreceivedfundingfromtheNorwegianNon-FictionWritersandTranslatorsAssociation. Additionalmaterialtothisbookcanbedownloadedfromhttp://extras.springer.com. ISSN1867-5506 ISSN1867-5514 (electronic) SpringerUndergraduateTextsinMathematicsandTechnology ISBN978-3-030-01811-5 ISBN978-3-030-01812-2 (eBook) https://doi.org/10.1007/978-3-030-01812-2 LibraryofCongressControlNumber:2018964077 MathematicsSubjectClassification(2010):15-01,65-01,15A99,65T50,65T60 ©SpringerNatureSwitzerlandAG2019 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartofthematerialis concerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broadcasting,reproduction onmicrofilmsorinanyotherphysicalway,andtransmissionorinformationstorageandretrieval,electronicadaptation, computersoftware,orbysimilarordissimilarmethodologynowknownorhereafterdeveloped. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublicationdoesnot imply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotectivelawsand regulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbookarebelieved tobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsortheeditorsgiveawarranty, expressorimplied,withrespecttothematerialcontainedhereinorforanyerrorsoromissionsthatmayhavebeenmade. Thepublisherremainsneutralwithregardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland To Natalia, Katrina, and Alina. Preface TheideaforthisbookbeganwithanewcourseattheUniversityofOslo,called“Appli- cationsofLinearAlgebra”,whichwasofferedforthefirsttimein2012.Attheuniversity we had come to realize that students who have had a first course in linear algebra, al- ready have the prerequisites to learn about severalimportant and interesting topics in signal processing and wavelet theory. Unfortunately, most textbooks on these subjects are written in a language poorly suited to a linear algebra background, making much literature only accessible to engineering and signal processing students, and excluding a large number of students. It also seems that it is not a common textbook strategy to introduce signalprocessingandwaveletstogether fromscratch,eventhoughthe two can very much motivate one another. Why not write a self-contained textbook, where linear algebra is the main foundation? This notion is what motivated the preparation of this book. The author must admit that he is not particularly fond of the majority of signal processingliterature,whichoftenhides ideasandresults whichessentiallyboildownto basic linear algebra, using a signal processing nomenclature that is difficult for many mathematicians to understand. Some examples are: • Matrix notation is often absent. Instead, linear operations, such as the DFT, are often expressed by component formulas. Matrix multiplication is absent, although basicoperationssuchasconvolutioncanbeinterpretedasthis.Asaresult,important matrix factorizations are not included. • Many operationsthat representchangesof coordinates(such as the DFT, DCT and DWT) are not represented as such. As a result, no tools, notation or results from linear algebra are used for these operations. • Eigenvalues and eigenvectors are not mentioned, even when these concepts could shed light on the theory: It is often not mentioned that the Fourier basis vectors are eigenvectors for filters, with the frequency response being the corresponding eigenvalues. The property that convolution in time corresponds to multiplication in frequency, an important notion in signal processing, can be summarized as follows in linear algebra: for matrices with the same eigenvectors, the eigenvalues of the product are the product of the eigenvalues. • Function spaces are rarely put into the context of vector or inner product spaces, evenif Fourier seriescanbe seen asa leastsquaresapproximationfromsuch spaces. VII VIII Preface Further, it is rarely mentioned that the formulas for the Fourier coefficients (the Fourier integrals) follow from the orthogonaldecomposition theorem. Others have also recognized the need to write new textbooks that employ more linear algebra. As an example, book [70] goes further in using matrix notation than many signal processing textbooks. Still, more can be done in this direction, so that students with only a basic linear algebra backgroundwill feel more at home. This book provides anintroduction to Fourier analysis and signalprocessing (in the first part of the book) and wavelets (in the second), assuming readers have completed anintroductorycourseinlinearalgebra.Withoutthisbackgroundknowledge,thebook willbe oflimitedvalue.Anappendixhasbeenincludedsothatstudentscanreviewthe most important linear algebra concepts, but a full course on these topics is preferred in order to follow the book. We use book [32] as the primary reference work for linear algebra,meaningthatmanyconcretedefinitionsandconceptsfromthatbookarecited by concreteidentifiers(i.e. pagenumber,theoremnumber,etc.). This isa verypopular choice for a first course in linear algebra,but there are many other excellent textbooks available. Leon [33] is a good example, which, just as [32], includes many exercises related to MATLAB. Several other good introductions to linear algebra claim to be applied in some way,such as [53], [75] and [69]. For a book that goes a bit further into the theory than these, please consult [49] or [37]. The book can be used at the undergraduate level, and can be taken directly after a first course in linear algebra. It is also possible to use the material at the graduate level. A number of theory aspects from basic linear algebra are further developed: • Complex vectorspaces andinner products areconsidered(many introductorylinear algebra textbooks concentrate only on realvector spaces and inner product spaces). • Inner product spaces that are function spaces are used extensively. Many introduc- tory linear algebra textbooks consider such spaces, but often fail to offer extensive practice with them. • More intuition on changes of coordinates is developed, in particular for the DFT, DCT and DWT. The book itself can be viewed as an extension to a basic linear algebra textbook, and may in the future become additional chapters in such a book. Thestyleofthebookisverydifferentfrommosttextbooksonsignalprocessing,since thelanguageoflinearalgebraisusedconsistently.Muchofthematerialandmanyproofs havebeenredesignedfromtheirsourcestofitintoalinearalgebracontext.Thematerial on wavelets has also been redesigned for this purpose, but also because it is not that common to introduce wavelets at the undergraduate level. The book also attempts to breakupmuchtheorywithexplanatorycode.Thisapproach,too,israrelyfoundinthe literature,where code is often movedto appendices, to separateit completely fromthe theory.Theauthorbelievesthatsuchseparationisunfortunateinmanycases,asitcan lead students to spend all their time on the theory alone. Due to these considerations, this book has been more than 6 years in the making. Sincethebookdoesnotofferacomprehensivetreatmentofallbasicsignalprocessing concepts or nomenclature, some engineering students may feel excluded. To make the bookmoreaccessibleforsuchstudents,asectionisincludedtohelpreconcilethe signal processing perspective and linear algebra perspective. The summaries throughout the book also address the connections and differences between the two perspectives. The section below on how to use the book in engineering courses provides further details. Preface IX The book attempts to go further than many books with the name “applications” in their title, in that it has a clear computational perspective. The theory motivates algorithmsandcode,forwhichmanyprogrammingandbestcodingpracticeissuesneed tobeaddressed.Thebookfocusesonintegratingapplications,suchasmodernstandards for the compression of sound and images (MPEG, JPEG, JPEG2000). A public and open source github repository for Fourier analysis and wavelets accompanies the book, whereallcodefromthebookcanbefound(inparticulartheFFTandDWTlibraries),as wellasaccompanyingtestcode,documentation,andsamplefiles.Throughoutthebook this repositorywillsimply be referredto as“the library”.The addressofthe repository is https://github.com/oyvindry/applinalgcode. The library contains more code than showninthebook,andthiscodewillbecontinuouslyupdatedandimproved.Notealso that the scope of the library is larger than that of the book. As an example, a paper on more general boundary modes for wavelets is being completed in parallel with the book. Also, since most algorithms in the book are well suited for parallel computing, future versions of the library and the book may support GPU-based programming. The library differs from many toolboxes in that the individual steps in a given im- plementation are firmly anchored in labeled formulas in the book. The focus on the computationalperspectivehasbeeninspiredbythe project“Computing inScienceEd- ucation”attheUniversityofOslo,aninitiativetointegratecomputationsintothebasic science curriculum at the university from the very first semester. Programming Itisassumedthatthestudenthasalreadybeenintroducedtoaprogramminglanguage or computational tool. Preferably, he or she will have already taken a full course in programming first, since the book does not include an introduction to primitives such as loops, conditional statements, lists, function definitions, file handling, or plotting. At the University of Oslo, most students take such a Python-based course during the first semester, focusing on such primitives. This course uses book [31], which provides an excellent introduction to Python programming for beginning students. Thisbookcomesintwoversions:onebasedonMATLAB,andoneonPython(where version3isthe supportedversion).The versionofthe bookyouarereadingusesMAT- LAB. There are severalother recent examples of books adapted to different languages: See[41]and[42]forMATLABandPythonversionsofatextonmechanics,and[34]and [35] for MATLAB andPython versionsofa computationalscience textbook (the latter are also recommended introductions to programming in these languages for beginning students).Ifyousearchtheinternetforrecommendationsaboutwhatprogramminglan- guage to use in a basic linear algebracourse,youmay find statements suchas “Python istooadvancedforsuchabeginningcourse”,or“Youcangetstartedmuchquickerwith MATLAB”. A good reply to such statements may be that the choice of programming language should most likely depend on how programming is already integrated into the courses at your university. Provided the integration at your university is thought- through and well-organized, the programming in the book should not prove to be too advanced for you, regardless of the choice of language. Todistinguishbetweenthelibraryandothertoolboxes,mostfunctionsinthelibrary havetheending"_impl".IfyoucomparethecodeintheMATLABandPythonversions of the book, you will see that the code in the two languages is very similar: X Preface • Functionsignaturesandvariablenamesarevirtuallythesame,followingthePython standard (words separated by "_" and using lowercase letters). • Code indentation follows the Python standard, where it is an important part of the syntax. • The heavily used Python packagenumpydoes not use a prefix, since MATLAB does not prefix code (this means that other packages with conflicting names must use a prefix). For the same reason, the modules developed in the book also do not use a prefix. There are also some differences in the MATLAB and Python versions, however. • The Python code is split into modules, a very important structuring concept in Python. The book explains which modules the corresponding code is part of. In MATLAB there is no module concept,andmany functions mustbe placedinfiles of the samename. As a result,there aremore files in the MATLAB partofthe library. • In Python, it is customary to place test code in the same module as the functions being tested. Since MATLAB does not use modules, separate files are instead used for test code instead. • MATLAB passes all input and return parameters by value, not by reference, as languages like Python do. This means that we can perform in-place computation in Python,i.e.theresultcanbewrittendirectlyintotheinputbuffer,avoidingmemory copy. This can lead to much more efficient code, so the Python code performs in- place operations wherever possible. This affects the signatures of many functions: Several Python functions have no return values, since the result is written directly into the input buffer, while the MATLAB counterparts use a return value for the result. Examples of in-place algorithms developed in this book are the DFT, the DCT, filters, the DWT, and filter bank transforms. There are also many other small differences between the two languages: • Indicesinvectorsstartwith1inMATLAB,and0inPython(indexingstartingwith 0 is more natural for many of the algorithms discussed in the book). • Thedistinctionbetweendifferentdimensions(1,2,3,andsoon)isclearerinPython. • Extracting subsets of an object corresponding to rows/columns with a given set of indices works more uniformly in MATLAB. Many of these differences can be easily overcome, however, and there are many trans- lation guides available online.1 MATLAB started as a command-based tool, and has evolved over the years to sup- port many features of modern programming languages. Python on the other hand was designed as a generalprogramminglanguage fromthe outset. As a result,MATLAB is not as structured as Python, and there are severalbad MATLAB programming habits around, which produce programs with illogical structures. We have sought to follow many Python conventions for MATLAB code, in an attempt to avoid many of these habits. Python has support for classes, but their use in the library has been limited for two reasons. First of all, class hierarchies don’t greatly simplify programming of the concepts in the book. Secondly, although MATLAB has some primitive support for classes, it remains unclear to the author how well this support actually works. By not using classes, the MATLAB and Python code in the library have virtually identical APIs. 1 Seeforinstancehttp://mathesaurus.sourceforge.net/matlab-numpy.html.
Description:The list of books you might like
