Bayesian Filtering and Smoothing

256 Pages·2013·2.264 MB·English

by  Saerkkae S.

more information - www.cambridge.org/9781107030657

BayesianFilteringandSmoothing
Filteringandsmoothingmethodsareusedtoproduceanaccurateestimateofthestate
ofatime-varyingsystembasedonmultipleobservationalinputs(data).Interestin
thesemethodshasexplodedinrecentyears,withnumerousapplicationsemergingin
fieldssuchasnavigation,aerospaceengineering,telecommunications,andmedicine.
Thiscompact,informalintroductionforgraduatestudentsandadvanced
undergraduatespresentsthecurrentstate-of-the-artfilteringandsmoothingmethods
inaunifiedBayesianframework.Readerslearnwhatnon-linearKalmanfiltersand
particlefiltersare,howtheyarerelated,andtheirrelativeadvantagesand
disadvantages.Theyalsodiscoverhowstate-of-the-artBayesianparameterestimation
methodscanbecombinedwithstate-of-the-artfilteringandsmoothingalgorithms.
Thebook’spracticalandalgorithmicapproachassumesonlymodestmathematical
prerequisites.ExamplesincludeMATLABcomputations,andthenumerous
end-of-chapterexercisesincludecomputationalassignments.MATLAB/GNUOctave
sourcecodeisavailablefordownloadatwww.cambridge.org/sarkka,promoting
hands-onworkwiththemethods.
simo sa¨rkka¨ worked,from2000to2010,withNokiaLtd.,IndagonLtd.,andthe
NalcoCompanyinvariousindustrialresearchprojectsrelatedtotelecommunications,
positioningsystems,andindustrialprocesscontrol.Currently,heisaSenior
ResearcherwiththeDepartmentofBiomedicalEngineeringandComputational
ScienceatAaltoUniversity,Finland,andAdjunctProfessorwithTampereUniversity
ofTechnologyandLappeenrantaUniversityofTechnology.In2011hewasavisiting
scholarwiththeSignalProcessingandCommunicationsLaboratoryoftheDepartment
ofEngineeringattheUniversityofCambridge.Hisresearchinterestsareinstateand
parameterestimationinstochasticdynamicsystemsand,inparticular,Bayesian
methodsinsignalprocessing,machinelearning,andinverseproblemswith
applicationstobrainimaging,positioningsystems,computervision,andaudio
signalprocessing.HeisaSeniorMemberoftheIEEE.
INSTITUTE OF MATHEMATICAL STATISTICS
TEXTBOOKS
EditorialBoard
D.R.Cox(UniversityofOxford)
A.Agresti(UniversityofFlorida)
B.Hambly(UniversityofOxford)
S.Holmes(StanfordUniversity)
X.-L.Meng(HarvardUniversity)
IMSTextbooksgiveintroductoryaccountsoftopicsofcurrentconcernsuitablefor
advancedcoursesatmaster’slevel,fordoctoralstudentsandforindividualstudy.They
aretypicallyshorterthanafullydevelopedtextbook,oftenarisingfrommaterial
createdforatopicalcourse.Lengthsof100–290pagesareenvisaged.Thebooks
typicallycontainexercises.
Bayesian Filtering and Smoothing
SIMO SA¨ RKKA¨
AaltoUniversity,Finland
cambridge university press
Cambridge,NewYork,Melbourne,Madrid,CapeTown,
Singapore,Sa˜oPaulo,Delhi,MexicoCity
CambridgeUniversityPress
TheEdinburghBuilding,CambridgeCB28RU,UK
PublishedintheUnitedStatesofAmericabyCambridgeUniversityPress,NewYork
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781107030657
(cid:2)C SimoSa¨rkka¨2013
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2013
PrintedandboundintheUnitedKingdombytheMPGBooksGroup
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary
LibraryofCongressCataloguinginPublicationdata
ISBN978-1-107-03065-7Hardback
ISBN978-1-107-61928-9Paperback
Additionalresourcesforthispublicationatwww.cambridge.org/sarkka
CambridgeUniversityPresshasnoresponsibilityforthepersistenceor
accuracyofURLsforexternalorthird-partyinternetwebsitesreferredto
inthispublication,anddoesnotguaranteethatanycontentonsuch
websitesis,orwillremain,accurateorappropriate.
Contents
Preface ix
Symbolsandabbreviations xiii
1 WhatareBayesianfilteringandsmoothing? 1
1.1 ApplicationsofBayesianfilteringandsmoothing 1
1.2 OriginsofBayesianfilteringandsmoothing 7
1.3 OptimalfilteringandsmoothingasBayesianinference 8
1.4 AlgorithmsforBayesianfilteringandsmoothing 12
1.5 Parameterestimation 14
1.6 Exercises 15
2 Bayesianinference 17
2.1 PhilosophyofBayesianinference 17
2.2 Connectiontomaximumlikelihoodestimation 17
2.3 ThebuildingblocksofBayesianmodels 19
2.4 Bayesianpointestimates 20
2.5 Numericalmethods 22
2.6 Exercises 24
3 BatchandrecursiveBayesianestimation 27
3.1 Batchlinearregression 27
3.2 Recursivelinearregression 29
3.3 Batchversusrecursiveestimation 31
3.4 Driftmodelforlinearregression 33
3.5 Statespacemodelforlinearregressionwithdrift 36
3.6 Examplesofstatespacemodels 39
3.7 Exercises 46
4 Bayesianfilteringequationsandexactsolutions 51
4.1 Probabilisticstatespacemodels 51
4.2 Bayesianfilteringequations 54
4.3 Kalmanfilter 56
v
vi Contents
4.4 Exercises 62
5 ExtendedandunscentedKalmanfiltering 64
5.1 Taylorseriesexpansions 64
5.2 ExtendedKalmanfilter 69
5.3 Statisticallinearization 75
5.4 Statisticallylinearizedfilter 77
5.5 Unscentedtransform 81
5.6 UnscentedKalmanfilter 86
5.7 Exercises 92
6 GeneralGaussianfiltering 96
6.1 Gaussianmomentmatching 96
6.2 Gaussianfilter 97
6.3 Gauss–Hermiteintegration 99
6.4 Gauss–HermiteKalmanfilter 103
6.5 Sphericalcubatureintegration 106
6.6 CubatureKalmanfilter 110
6.7 Exercises 114
7 Particlefiltering 116
7.1 MonteCarloapproximationsinBayesianinference 116
7.2 Importancesampling 117
7.3 Sequentialimportancesampling 120
7.4 Sequentialimportanceresampling 123
7.5 Rao–Blackwellizedparticlefilter 129
7.6 Exercises 132
8 Bayesiansmoothingequationsandexactsolutions 134
8.1 Bayesiansmoothingequations 134
8.2 Rauch–Tung–Striebelsmoother 135
8.3 Two-filtersmoothing 139
8.4 Exercises 142
9 Extendedandunscentedsmoothing 144
9.1 ExtendedRauch–Tung–Striebelsmoother 144
9.2 StatisticallylinearizedRauch–Tung–Striebelsmoother 146
9.3 UnscentedRauch–Tung–Striebelsmoother 148
9.4 Exercises 152
10 GeneralGaussiansmoothing 154
10.1 GeneralGaussianRauch–Tung–Striebelsmoother 154
10.2 Gauss–HermiteRauch–Tung–Striebelsmoother 155
Contents vii
10.3 CubatureRauch–Tung–Striebelsmoother 156
10.4 Generalfixed-pointsmootherequations 159
10.5 Generalfixed-lagsmootherequations 162
10.6 Exercises 164
11 Particlesmoothing 165
11.1 SIRparticlesmoother 165
11.2 Backward-simulationparticlesmoother 167
11.3 Reweightingparticlesmoother 169
11.4 Rao–Blackwellizedparticlesmoothers 171
11.5 Exercises 173
12 Parameterestimation 174
12.1 Bayesianestimationofparametersinstatespacemodels 174
12.2 Computationalmethodsforparameterestimation 177
12.3 Practicalparameterestimationinstatespacemodels 185
12.4 Exercises 202
13 Epilogue 204
13.1 WhichmethodshouldIchoose? 204
13.2 Furthertopics 206
Appendix Additionalmaterial 209
A.1 PropertiesofGaussiandistribution 209
A.2 Choleskyfactorizationanditsderivative 210
A.3 ParameterderivativesfortheKalmanfilter 212
A.4 ParameterderivativesfortheGaussianfilter 214
References 219
Index 229