Introduction to Sound Processing

244 Pages·2003·3.386 MB·English

by  Rocchesso D.

Introduction to
Sound Processing
Davide Rocchesso∗
∗ UniversitàdiVerona
DipartimentodiInformatica
email:[email protected]
www:http://www.scienze.univr.it/˜rocchess
Copyright(cid:13)c 2003DavideRocchesso.Thisworkislicensedunder
theCreativeCommonsAttribution-ShareAlikeLicense.Toviewa
copyofthislicense,visithttp://creativecommons.org/licenses/by-
sa/1.0/orsendalettertoCreativeCommons,559NathanAbbott
Way,Stanford,California94305,USA.
Attribution-ShareAlike1.0
Youarefree:
• tocopy,distribute,display,andperformthework
• tomakederivativeworks
• tomakecommercialuseofthework
underthefollowingconditions:
Attribution.Youmustgivetheoriginalauthorcredit.
ShareAlike.Ifyoualter,transform,orbuilduponthiswork,youmaydistributethe
resultingworkonlyunderalicenseidenticaltothisone.
• Foranyreuseordistribution,youmustmakecleartoothersthelicensetermsofthiswork.
• Anyoftheseconditionscanbewaivedifyougetpermissionfromtheauthor.
Yourfairuseandotherrightsareinnowayaffectedbytheabove.
Thebookisaccessiblefromtheauthor’swebsite:http://www.scienze.univr.it/˜rocchess.
Thebookislistedinhttp://www.theassayer.org,wherereviewscanbeposted.
ISBN88-901126-1-1
CoverDesign:ClaudiaCalvaresi.
EditorialProductionStaff:NicolaBernardini,FedericoFontana,
AlessandraCeccherelli,NicolaGiosmin,AnnaMeo.
ProducedfromLATEXtextsourcesandPostScriptandTIFFimages.
CompiledwithVTEX/free.
OnlinedistributedinPortableDocumentFormat.
PrintedandboundinItalybyPHASARSrl,Firenze.
Contents
1 Systems,SamplingandQuantization 1
1.1 Continuous-TimeSystems . . . . . . . . . . . . . . . . . . . 1
1.2 TheSamplingTheorem . . . . . . . . . . . . . . . . . . . . . 3
1.3 Discrete-TimeSpectralRepresentations . . . . . . . . . . . . 6
1.4 Discrete-TimeSystems . . . . . . . . . . . . . . . . . . . . . 11
1.4.1 TheImpulseResponse . . . . . . . . . . . . . . . . . 12
1.4.2 TheShiftTheorem . . . . . . . . . . . . . . . . . . . 13
1.4.3 StabilityandCausality . . . . . . . . . . . . . . . . . 14
1.5 Continuous-timetodiscrete-timesystem
conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.5.1 ImpulseInvariance . . . . . . . . . . . . . . . . . . . 15
1.5.2 BilinearTransformation . . . . . . . . . . . . . . . . 17
1.6 Quantization. . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2 DigitalFilters 23
2.1 FIRFilters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.1.1 TheSimplestFIRFilter . . . . . . . . . . . . . . . . 24
2.1.2 ThePhaseResponse . . . . . . . . . . . . . . . . . . 29
2.1.3 Higher-OrderFIRFilters . . . . . . . . . . . . . . . 32
2.1.4 RealizationsofFIRFilters . . . . . . . . . . . . . . . 40
2.2 IIRFilters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.2.1 TheSimplestIIRFilter . . . . . . . . . . . . . . . . 43
2.2.2 Higher-OrderIIRFilters . . . . . . . . . . . . . . . . 47
2.2.3 AllpassFilters . . . . . . . . . . . . . . . . . . . . . 55
2.2.4 RealizationsofIIRFilters . . . . . . . . . . . . . . . 57
2.3 Complementaryfiltersandfilterbanks . . . . . . . . . . . . . 62
2.4 Frequencywarping . . . . . . . . . . . . . . . . . . . . . . . 64
i
ii D.Rocchesso: SoundProcessing
3 DelaysandEffects 67
3.1 TheCircularBuffer . . . . . . . . . . . . . . . . . . . . . . . 67
3.2 Fractional-LengthDelayLines . . . . . . . . . . . . . . . . . 68
3.2.1 FIRInterpolationFilters . . . . . . . . . . . . . . . . 69
3.2.2 AllpassInterpolationFilters . . . . . . . . . . . . . . 72
3.3 TheNon-RecursiveCombFilter . . . . . . . . . . . . . . . . 74
3.4 TheRecursiveCombFilter . . . . . . . . . . . . . . . . . . . 76
3.4.1 TheComb-AllpassFilter . . . . . . . . . . . . . . . 78
3.5 SoundEffectsBasedonDelayLines . . . . . . . . . . . . . . 79
3.6 Spatialsoundprocessing . . . . . . . . . . . . . . . . . . . . 81
3.6.1 Spatialization . . . . . . . . . . . . . . . . . . . . . . 81
3.6.2 Reverberation . . . . . . . . . . . . . . . . . . . . . . 89
4 SoundAnalysis 99
4.1 Short-TimeFourierTransform . . . . . . . . . . . . . . . . . 99
4.1.1 TheFilterbankView . . . . . . . . . . . . . . . . . . 99
4.1.2 TheDFTView . . . . . . . . . . . . . . . . . . . . . 100
4.1.3 Windowing . . . . . . . . . . . . . . . . . . . . . . . 103
4.1.4 Representations . . . . . . . . . . . . . . . . . . . . . 108
4.1.5 Accuratepartialestimation . . . . . . . . . . . . . . . 110
4.2 Linearpredictivecoding(withFedericoFontana) . . . . . . . . 113
5 SoundModelling 117
5.1 Spectralmodelling . . . . . . . . . . . . . . . . . . . . . . . 117
5.1.1 Thesinusoidalmodel . . . . . . . . . . . . . . . . . . 117
5.1.2 Sines+Noise+Transients . . . . . . . . . . . . . . . 122
5.1.3 LPCModelling . . . . . . . . . . . . . . . . . . . . . 123
5.2 Time-domainmodels . . . . . . . . . . . . . . . . . . . . . . 124
5.2.1 TheDigitalOscillator . . . . . . . . . . . . . . . . . 124
5.2.2 TheWavetableOscillator . . . . . . . . . . . . . . . . 125
5.2.3 Wavetablesamplingsynthesis . . . . . . . . . . . . . 127
5.2.4 Granularsynthesis(withGiovanniDePoli) . . . . . . . 129
5.3 Nonlinearmodels . . . . . . . . . . . . . . . . . . . . . . . . 130
5.3.1 Frequencyandphasemodulation. . . . . . . . . . . . 130
5.3.2 Nonlineardistortion . . . . . . . . . . . . . . . . . . 135
5.4 Physicalmodels . . . . . . . . . . . . . . . . . . . . . . . . . 137
5.4.1 Aphysicaloscillator . . . . . . . . . . . . . . . . . . 137
5.4.2 Coupledoscillators . . . . . . . . . . . . . . . . . . . 138
5.4.3 One-dimensionaldistributedresonators . . . . . . . . 141
iii
A MathematicalFundamentals 145
A.1 ClassesofNumbers . . . . . . . . . . . . . . . . . . . . . . . 145
A.1.1 Fields . . . . . . . . . . . . . . . . . . . . . . . . . . 145
A.1.2 Rings . . . . . . . . . . . . . . . . . . . . . . . . . . 146
A.1.3 ComplexNumbers . . . . . . . . . . . . . . . . . . . 147
A.2 VariablesandFunctions . . . . . . . . . . . . . . . . . . . . . 148
A.3 Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
A.4 VectorsandMatrices . . . . . . . . . . . . . . . . . . . . . . 154
A.4.1 SquareMatrices . . . . . . . . . . . . . . . . . . . . 158
A.5 ExponentialsandLogarithms . . . . . . . . . . . . . . . . . 158
A.6 TrigonometricFunctions . . . . . . . . . . . . . . . . . . . . 161
A.7 DerivativesandIntegrals . . . . . . . . . . . . . . . . . . . . 164
A.7.1 DerivativesofFunctions . . . . . . . . . . . . . . . . 164
A.7.2 IntegralsofFunctions . . . . . . . . . . . . . . . . . 168
A.8 Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
A.8.1 TheLaplaceTransform . . . . . . . . . . . . . . . . 170
A.8.2 TheFourierTransform . . . . . . . . . . . . . . . . . 171
A.8.3 TheZTransform . . . . . . . . . . . . . . . . . . . . 172
A.9 ComputerArithmetics . . . . . . . . . . . . . . . . . . . . . 173
A.9.1 IntegerNumbers . . . . . . . . . . . . . . . . . . . . 173
A.9.2 RationalNumbers . . . . . . . . . . . . . . . . . . . 175
B ToolsforSoundProcessing
(withNicolaBernardini) 177
B.1 SoundsinMatlabandOctave . . . . . . . . . . . . . . . . . . 178
B.1.1 Digression . . . . . . . . . . . . . . . . . . . . . . . 179
B.2 LanguagesforSoundProcessing . . . . . . . . . . . . . . . . 182
B.2.1 Unitgenerator . . . . . . . . . . . . . . . . . . . . . 185
B.2.2 ExamplesinCsound,SAOL,andCLM . . . . . . . . 186
B.3 InteractiveGraphicalBuildingEnvironments . . . . . . . . . 192
B.3.1 ExamplesinARES/MARSandpd . . . . . . . . . . 193
B.4 Inlinesoundprocessing . . . . . . . . . . . . . . . . . . . . . 195
B.4.1 Time-DomainGraphicalEditingandProcessing . . . 196
B.4.2 Analysis/ResynthesisPackages . . . . . . . . . . . . 198
B.5 StructureofaDigitalSignalProcessor . . . . . . . . . . . . . 200
B.5.1 MemoryManagement . . . . . . . . . . . . . . . . . 202
B.5.2 InternalArithmetics . . . . . . . . . . . . . . . . . . 203
B.5.3 ThePipeline . . . . . . . . . . . . . . . . . . . . . . 205
iv D.Rocchesso: SoundProcessing
C Fundamentalsofpsychoacoustics 207
C.1 Theear . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
C.2 SoundIntensity . . . . . . . . . . . . . . . . . . . . . . . . . 209
C.2.1 Psychophysics . . . . . . . . . . . . . . . . . . . . . 213
C.3 Pitch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
C.4 CriticalBand . . . . . . . . . . . . . . . . . . . . . . . . . . 217
C.5 Masking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
C.6 Spatialsoundperception . . . . . . . . . . . . . . . . . . . . 219
Index 222
References 229
Preface
What you have in your hands, or on your screen, is an introductory book
on sound processing. By reading this book, you may expect to acquire some
knowledge on the mathematical, algorithmic, and computational tools that I
considertobeimportantinordertobecomeproficientsounddesignersorma-
nipulators.
Thebookistargetedatbothscience-andart-orientedreaders,eventhough
thelattermayfindithardiftheyarenotfamiliarwithcalculus.Forthispurpose
an appendix of mathematical fundamentals has been prepared in such a way
that the book becomes self contained. Of course, the mathematical appendix
isnotintendedtobeasubstituteofathoroughmathematicalpreparation,but
rather as a shortcut for those readers that are more eager to understand the
applications.
Indeed, this book was conceived in 1997, when I was called to teach in-
troductory audio signal processing in the course “Specialisti in Informatica
Musicale”organizedbytheCentroTempoRealeinFirenze.Inthatclass,the
majorityofthestudentswereexcellent(nokidding,reallysuperb!)musiccom-
posers.Onlytwostudentshadascientificbackground(indeed,areallystrong
scientific background!). The task of introducing this audience to filters and
trasforms was so challenging for me that I started planning the lectures and
laboratory material much earlier and in a structured form. This was the ini-
tialformofthisbook.Thecourseturnedouttobeanexcitingexperiencefor
me and, based on the music and the research material that I heard from them
afterward,Ihavetheimpressionthatthestudentsalsomadegooduseofit.
AfterthecourseinFirenze,Iexpandedandimprovedthebookduringfour
editions of my course on sound processing for computer science students at
the University of Verona. The mathematical background of these students is
differentfromthatoftypicalelectricalengineeringstudents,asitisstrongerin
discretemathematicsandalgebra,andwithnotmuchfamiliaritywithadvanced
v
vi D.Rocchesso: ElaborazionedelSuono
andappliedcalculus.Therefore,thebookspresentsthebasicsofsignals,sys-
tems,andtransformsinawaythatcanbeimmediatelyusedinapplicationsand
experiencedincomputerlaboratorysessions.
This is a free book, thus meaning that it was written using free software
tools,anditisfreelydownloadable,modifiable,anddistributableinelectronic
orprintedform,providedthattheenclosedlicenseandlinktoitsoriginalweb
locationareincludedinanyderivativedistribution.Thebookwebsitealsocon-
tainsthesourcecodeslistedinthebook,andotherauxiliarysoftwaremodules.
I encourage additions that may be useful to the reader. For instance, it
wouldbenicetohaveeachchapterendedbyasectionthatcollectsannotations,
solutions to the problems that I proposed in footnotes, and other problems or
exercises.Feelfreetoexploittheopennatureofthisbooktoproposeyourad-
ditionalcontents.
Venezia,11thFebruary2004 DavideRocchesso
Chapter 1
Systems, Sampling and
Quantization
1.1 Continuous-Time Systems
Soundisusuallyconsideredasamono-dimensionalsignal(i.e.,afunction
oftime)representingtheairpressureintheearcanal.Forthepurposeofthis
book,aSingle-InputSingle-Output(SISO)Systemisdefinedasanyalgorithm
ordevicethattakesasignalininputandproducesasignalinoutput.Mostof
ourdiscussionwillregardlinearsystems,thatcanbedefinedasthosesystems
forwhichthesuperpositionprincipleholds:
SuperpositionPrinciple :ify andy aretheresponsestotheinputsequences
1 2
x andx ,respectively,thentheinputax +bx producestheresponse
1 2 1 2
ay +by .
1 2
The superposition principle allows us to study the behavior of a linear sys-
temstartingfromtestsignalssuchasimpulsesorsinusoids,andobtainingthe
responsestocomplicatedsignalsbyweightedsumsofthebasicresponses.
A linear system is said to be linear time-invariant (LTI), if a time shift in
theinputresultsinthesametimeshiftintheoutputor,inotherwords,ifitdoes
notchangeitsbehaviorintime.
Any continuous-time LTI system can be described by a differential equa-
tion.TheLaplacetransform,definedinappendixA.8.1isamathematicaltool
that is used to analyze continuous-time LTI systems, since it allows to trans-
formcomplicateddifferentialequationsintoratiosofpolynomialsofacomplex
1
2 D.Rocchesso: SoundProcessing
variables.SuchratioofpolynomialsiscalledthetransferfunctionoftheLTI
system.
Example 1.ConsidertheLTIsystemhavingasinputandoutputthefunc-
tionsoftime(i.e.,thesignals)x(t)andy(t),respectively,anddescribedbythe
differentialequation
dy
−s y =x. (1)
dt 0
This equation, transformed into the Laplace domain according to the rules of
appendixA.8.1,becomes
sY (s)−s Y (s)=X (s). (2)
L 0 L L
Here,asinmostofthebook,weimplicitlyassumethattheinitialconditionsare
zero,otherwiseeq.(2)shouldalsocontainaterminy(0).Fromthealgebraic
equation(2)thetransferfunctionisderivedastheratiobetweentheoutputand
inputtransforms:
1
H(s)= . (3)
s−s
0
###
Thecoefficients ,rootofthedenominatorpolynomialof(3),iscalledthe
0
poleofthetransferfunction(orpoleofthesystem).Anyrootofthenumerator
wouldbecalledazeroofthesystem.
TheinverseLaplacetransformofthetransferfunctionisanequivalentde-
scriptionofthesystem.Inthecaseofexample1.1,ittakestheform
es0t t≥0
h(t)=(cid:26) , (4)
0 t<0
andsuchfunctioniscalledacausalexponential.
Ingeneral,thefunctionh(t),inversetransformofthetransferfunction,is
calledtheimpulseresponseofthesystem,sinceitistheoutputobtainedfrom
thesystemasaresponsetoanidealimpulse1.
Thetwoequivalentdescriptionsofalinearsysteminthetimedomain(im-
pulse response) and in the Laplace domain (transfer function) correspond to
twoalternativewaysofexpressingtheoperationsthatthesystemperformsin
ordertoobtaintheoutputsignalfromtheinputsignal.
1Arigorousdefinitionoftheidealimpulse,orDiracfunction,isbeyondthescopeofthisbook.
Thereadercanthinkofanidealimpulseasasignalhavingallitsenergylumpedatthetimeinstant
0.