Digital Signal Processing (Second Edition) -

840 Pages·2016·4.44 MB·English

TTeecchhnnoollooggiiccaall UUnniivveerrssiittyy DDuubblliinn
AARRRROOWW@@TTUU DDuubblliinn
Books/Book chapters School of Electrical and Electronic Engineering
2006-01-01
DDiiggiittaall SSiiggnnaall PPrroocceessssiinngg ((SSeeccoonndd EEddiittiioonn))
Jonathan Blackledge
Technological University Dublin, [email protected]
Follow this and additional works at: https://arrow.tudublin.ie/engschelebk
Part of the Signal Processing Commons
RReeccoommmmeennddeedd CCiittaattiioonn
Blackledge, J.: Digital Signal Processing (Second Edition). Horwood Publishing, vol: ISBN: 1-904275-26-5.
2006.
This Book is brought to you for free and open access by
the School of Electrical and Electronic Engineering at
ARROW@TU Dublin. It has been accepted for inclusion in
Books/Book chapters by an authorized administrator of
ARROW@TU Dublin. For more information, please
contact [email protected],
[email protected].
This work is licensed under a Creative Commons
Attribution-Noncommercial-Share Alike 4.0 License
DIGITAL SIGNAL PROCESSING
“Talkingofeducation, people havenow a-days”(he said)“gota strangeopinionthat
every thing should be taught by lectures. Now, I cannot see that lectures can do so
much good as reading the books from which the lectures are taken. I know nothing
that can be best taught by lectures, except where experiments are to be shown. You
may teach chymestry by lectures - You might teach making of shoes by lectures!”
James Boswell: Life of Dr Samuel Johnson, 1766
Dedication
To all those students with whom I had the good fortune to
work and, in using the material herein, taught me how to
teach it.
ABOUT THE AUTHOR
Jonathan Blackledge graduatedin physics from Imperial College and music from the
Royal College of Music, London, in 1980 and obtained a Doctorate in theoretical
physics from the same university in 1983. He was appointed as Research Fellow of
Physics at Kings College, London from 1983 to 1988 specializing in inverse problems
in electromagnetism and acoustics. During this period, he worked on a number of
industrial research contracts undertaking theoretical and computational work on the
applications of inverse scattering theory for the analysis of signals and images.
In 1988, he joined the Applied Mathematics and Computing Group at Cran-
field University as Lecturer and later, as Senior Lecturer and Head of Group where
he promoted postgraduate teaching and research in applied, engineering and indus-
trial mathematics in areas which included computer aided engineering, digital signal
processing and computer graphics. In 1994, he was appointed Professor of Applied
Mathematics and Computing and Head of the Department of Mathematical Sciences
at De Montfort University where he established the Institute of Simulation Sciences.
HeiscurrentlyProfessorofComputerScienceintheDepartmentofComputerScience
at the University of the Western Cape, South Africa nd Professor of of Information
andCommunicationsTechnologyintheDepartmentofElectronicsandElectricalEn-
gineering at Loughborough University, England. He is also a co-founder and Direc-
tor of a group of companies specializing in communications technology and financial
analysis based in London and New York.
Professor Blackledge has published over one hundred scientific and engineering
research papers and technical reports for industry, six industrial software systems,
fifteenpatents,tenbooksandhasbeensupervisorto sixtyresearch(PhD)graduates.
He lectures widely to a variety of audiences composed of mathematicians, computer
scientists, engineers and technologists in areas that include cryptology, communica-
tions technology and the use of artificial intelligence in process engineering, financial
analysis and risk management. His current research interests include computational
geometry and computer graphics,image analysis, nonlinear dynamical systems mod-
elling and computer network security, working in both an academic and commercial
context. Heholds FellowshipswithEngland’sleadingscientific andengineeringInsti-
tutesandSocietiesincludingtheInstituteofPhysics,theInstituteofMathematicsand
its Applications, the Institution of Electrical Engineers, the Institution of Mechan-
ical Engineers, the British Computer Society, the Royal Statistical Society and the
Institute of Directors. He is a Chartered Physicist, Chartered Mathematician, Char-
teredElectricalEngineer,CharteredMechanicalEngineer,CharteredStatisticianand
a Chartered Information Technology Professional. He has an additional interest in
music for which he holds a Fellowship of the Royal Schools of Music, London.
DIGITAL SIGNAL PROCESSING
Mathematical and Computational Methods,
Software Development and Applications
Second Edition
JONATHAN M. BLACKLEDGE†
Professor of Information and Communications Technology,
Department of Electronic and Electrical Engineering,
Loughbourough University, England.
Horwood Publishing, Chichester,
West Sussex, England
†Professor of Computer Science, Department of Computer Science, University of the
Western Cape, Cape Town, South Africa.
HORWOOD PUBLISHING LIMITED
Coll House, Westergate, Chichester, West Sussex, PO20 3QL, England.
First published in 2003 and re-printed in 2006 with corrections and additions.
COPYRIGHT NOTICE
All Rights Reserved. No part of this publication may be reproduced, stored in a
retrievalsystem,ortransmitted,inanyformorbyanymeans,electronic,mechanical,
photocopy, recording, or otherwise, without the permission of Horwood Publishing
Limited, Coll House, Westergate, Chichester, West Sussex, PO20 3QL, England.
(cid:2)cJ. M. Blackledge, 2006
British Library Cataloguing in Publishing Data
A catalogue record of this book is available from the British Library.
ISBN 1-904275-26-5
Typeset and produced by the author using LaTeX, the TeXnicCenter graphical user
interface and the stylefile of the Institute of Mathematics and its Applications.
Printed and bound in Great Britain by Antony Rowe Limited.
v
Foreword to the Second Edition
I was flattered when the publisher, Ellis Horwood asked me to write this Foreword,
because my introduction to signal processing began in the Second World War when,
as a communications officer in the Royal Corps of Signals, I worked with a war-time
teleprinter. We used a system based on 5-bit letters and a pair of copper wires. It
providedabandwidthofabout200Hzthatcouldbeseparatedbyfiltersfromthe rest
of a voice channel without noticeably distorting the speech. Today the bandwidth
availableishugebycomparisonandinformationcanbeconveyedthroughamultitude
ofchannelsusingtinyglassfibres. However,althoughtheengineeringassociatedwith
information andcommunications technology in generalhas andcontinues to undergo
radical change, many of the underlying mathematical principles remain the same.
G H Hardy in his book A Mathematician’s Apology wrote that there were ‘no
interesting applications of pure mathematics’. This is no longer true and Professor
Blackledge’s book Digital Signal Processing will enable many people to make use
of their interest in, and perhaps fascination with, mathematics in such a way, and
throughafieldofstudy,thatwillhelpusallcommunicateourideasmorequicklyand
conveniently through the digital world of today.
The Earl Kitchener of Khartoum
vi
Preface to the Second Edition
Thisbookprovidesanaccountofthemathematicalbackground,computationalmeth-
ods and software engineering associated with digital signal processing. The aim has
beentoprovidethereaderwiththemathematicalmethodsrequiredforsignalanalysis
which are then used to develop models and algorithms for processing digital signals
and finally to encourage the reader to design software solutions for Digital Signal
Processing (DSP). In this way, the reader is invited to develop a small DSP library
that can then be expanded further with a focus on his/her research interests and
applications.
There are of course many excellent books and software systems available on this
subject area. However, in many of these publications, the relationship between the
mathematical methods associated with signal analysis and the software available for
processing data is not always clear. Either the publications concentrate on mathe-
maticalaspects thatarenotfocusedonpracticalprogrammingsolutionsor elaborate
on the software development of solutions in terms of working ‘black-boxes’ without
covering the mathematical background and analysis associated with the design of
these software solutions. Thus, this book has been written with the aim of giving
the reader a technical overview of the mathematics and software associated with the
‘art’ of developing numerical algorithms and designing software solutions for DSP,
all of which is built on firm mathematical foundations. For this reason, the work
is, by necessity, rather lengthy and covers a wide range of subjects compounded in
four principalparts. PartI providesthe mathematicalbackgroundforthe analysisof
signals, Part II considers the computational techniques (principally those associated
with linear algebra and the linear eigenvalue problem) required for array processing
and associated analysis (error analysis for example). Part III introduces the reader
to the essential elements of software engineering using the C programming language,
tailored to those features that are used for developing C functions or modules for
building a DSP library.
The material associated with parts I, II and III is then used to build up a DSP
systembydefininganumberof‘problems’andthenaddressingthesolutionsinterms
ofpresentinganappropriatemathematicalmodel,undertakingthenecessaryanalysis,
developinganappropriatealgorithmandthencodingthesolutioninC.Thismaterial
forms the basis for part IV of this work.
In most chapters, a series of tutorial problems is given for the reader to attempt
with answersprovidedin Appendix A. These problems include theoretical, computa-
tionalandprogrammingexercises. PartIIofthisworkisrelativelylongandarguably
contains too much material on the computational methods for linear algebra. How-
ever,thismaterialandthecomplementarymaterialonvectorandmatrixnormsforms
the computational basis for many methods of digital signal processing. Moreover,
this important and widely researchedsubject area forms the foundations, not only of
digital signal processing and control engineering for example, but also of numerical
analysis in general.
The material presented in this book is based on the lecture notes and supple-
mentary material developed by the author for an advanced Masters course ‘Digital
SignalProcessing’whichwasfirstestablishedatCranfieldUniversity,Bedfordin1990
and modified when the author moved to De Montfort University, Leicester in 1994.
vii
The programmes are still operating at these universities and the material has been
used by some 700++ graduates since its establishment and development in the early
1990s. The material was enhanced and developed further when the author moved
to the Department of Electronic and Electrical Engineering at Loughborough Uni-
versity in 2003 and now forms part of the Department’s post-graduate programmes
in Communication Systems Engineering. The original Masters programme included
a taught component covering a period of six months based on two semesters, each
Semester being composed of four modules. The material in this work covers the first
Semester and its four parts reflect the four modules delivered. The material deliv-
eredinthe secondSemesterispublishedasacompanionvolumetothis workentitled
Digital Image Processing, Horwood Publishing, 2005 which covers the mathematical
modellingofimagingsystemsandthetechniquesthathavebeendevelopedtoprocess
and analyse the data such systems provide.
Since the publication of the first edition of this work in 2003, a number of mi-
nor changes and some additions have been made. The material on programming
and software engineering in Chapters 11 and 12 has been extended. This includes
some additions and further solved and supplementary questions which are included
throughout the text. Nevertheless, it is worth pointing out, that while every effort
has been made by the author and publisher to provide a work that is error free, it is
inevitable that typing errors and various ‘bugs’ will occur. If so, and in particular, if
the reader starts to suffer from a lack of comprehension over certain aspects of the
material (due to errors or otherwise) then he/she should not assume that there is
something wrong with themselves, but with the author!
J M Blackledge, January 2006
viii
Acknowledgements
The material developed in this book has been helped and encouraged by numerous
colleaguesoftheauthorovertheyears. Theauthorwouldliketothankallofhisfellow
colleagues, but particular thanks go to Prof Roy Hoskins who, for many years, has
playedacentralroleinteachingaspectsofsignalanalysiscoupledwiththemathemat-
icalrigourrequiredtocometotermswithsuchentitiesastheDeltafunction. Thanks
also go to Dr Peter Sherar at Cranfield University who helped the author establish
the MSc programmein ‘Software Solutions for Digital Signal Processing’from which
much of the material presented in this work has been derived. Thanks also go to
Dr Martin Turner, Dr Mohammed Jaffar, Dr Martin Crane and Prof Gwynne Evans
at De Montfort University who have worked with the author for many years, and
as module leaders for De Montfort University’s advanced MSc programme in digital
signalprocessing,developedvaluableadditionstotheteachingandlearningmaterials
first established by the author. Further, many students of this MSc course were the
resultofresearchprogrammesestablishedbyProfBFoxonatDeMontfortUniversity
with links to universities and researchinstitutes in Russia, Poland, South Africa and
theUSA,towhomtheauthor(andthestudents)areverygrateful. Theauthorwould
alsoliketothankProfPeterSmithwhoasHeadofthe DepartmentofElectronicand
Electrical Engineering at Loughborough University has provided the infrastructure
for the author to operate in an internationally acknowledge center of engineering ex-
cellence. Thanks also go to Dr S Datta of LoughboroughUniversity who has worked
with the author for many years and provided him with many valuable recommen-
dations, ideas and research themes. In addition, the author would like to thank all
those organizations and industries that have provided funding for the development
of his teaching and research activities over the years including: the Engineering and
Physical Sciences Research Council, the Defense Evaluation and Research Agency,
the International Science and Technology Council, British Coal, British Gas, British
Petroleum, British Aerospace, Oxford Instruments, Microsharp, Marconi, Microsoft
and British Intelligence. Finally, the author would like to thank all those postgrad-
uate students (both MSc and PhD) who, over the years, have used the material in
this book as part of their technical training and educational development and have
provided critical and constructive appraisal from one year to the next. The material
waswrittenbytheauthorforstudentsundertakingadvancedMScprogrammesatthe
universitiesofCranfield,DeMontfortandmorerecently,atLoughboroughUniversity
and has derived great pleasure from presenting it and developing it further as part
of his teaching portfolio. In addition to the taught postgraduate programmes from
which the materialhereinhas been derived, numerousresearchstudents haveworked
closely with the author providing valuable insights and comments that have helped
to enhance and strengthen the material.
ix
Notation
Alphabetic
a Real coefficients of a Fourier cosine series
n
adjA Adjoint of matrix A
A≡(a ) Matrix with element at ith row and jth column
ij
AT Transpose of A
A−1 Inverse of A
[A|B] Augmented matrix formed from matrices A and B
|A| Determinant of A
A(t) Amplitude modulation (amplitude envelope)
A(ω) Amplitude spectrum
b Real coefficients of a Fourier sine series
n
b Data vector of linear system Ax=b
chirp(t) Unit chirp function(cid:2)[with complex form exp(−iαt2)]
comb(t) Comb function [= δ(t−nT)]
n
cond(A) Condition number of matrix A (=(cid:4)A(cid:4)×(cid:4)A−1(cid:4))
c Complex coefficients of a complex Fourier series for example
n
C Capacitance or the contour followed by a path of integration in
the z-plane
c Data vector associated with linear system x=Mx+c
D Fractal dimension or diagonal matrix
detA Determinant of A (also denoted by |A|)
e Error vector
f(t) Arbitrary real function - typically object function or system input
f(z) Function of a complex variable
|f | modulus of complex variable or function f
(cid:4)f(t)(cid:4) Norm (e.g. a Euclidean norm) of a function f(t)
(cid:4)f (cid:4) Norm of an array or vector f
i i
(cid:4)fi(cid:4)2 Euclidean norm of array or vector fi
(cid:4)x(cid:4) p-norm of vector x
p
(cid:4)x(cid:4)∞ ‘Infinity’ or uniform norm of a vector x
(cid:4)A(cid:4) Norm of matrix A
F(ω) Complex spectrum of function f(t)
F (ω) Real component of spectrum
r
F (ω) Imaginary component of spectrum
i
F Discrete complex spectrum of discrete function f
i i
g(t) Arbitrary function
g(t|t0,ω) Green’s function
H(t) Tophat function
I Unit or identity matrix
Im[f] Imaginary part of complex variable or function f