Optimal Estimation of Dynamic Systems, Second Edition

750 Pages·2011·7.526 MB·English

by  John L. Crassidis, John L. Junkins

Optimal Estimation of
Dynamic Systems
Second Edition
© 2012 by Taylor & Francis Group, LLC
CHAPMAN & HALL/CRC APPLIED MATHEMATICS
AND NONLINEAR SCIENCE SERIES
Series Editor Goong Chen
Published Titles
Advanced Differential Quadrature Methods, Zhi Zong and Yingyan Zhang
Computing with hp-ADAPTIVE FINITE ELEMENTS, Volume 1, One and Two Dimensional
Elliptic and Maxwell Problems, Leszek Demkowicz
Computing with hp-ADAPTIVE FINITE ELEMENTS, Volume 2, Frontiers: Three
Dimensional Elliptic and Maxwell Problems with Applications, Leszek Demkowicz,
Jason Kurtz, David Pardo, Maciej Paszy´nski, Waldemar Rachowicz, and Adam Zdunek
CRC Standard Curves and Surfaces with Mathematica®: Second Edition,
David H. von Seggern
Discovering Evolution Equations with Applications: Volume 1-Deterministic Equations,
Mark A. McKibben
Discovering Evolution Equations with Applications: Volume 2-Stochastic Equations,
Mark A. McKibben
Exact Solutions and Invariant Subspaces of Nonlinear Partial Differential Equations in
Mechanics and Physics, Victor A. Galaktionov and Sergey R. Svirshchevskii
Fourier Series in Several Variables with Applications to Partial Differential Equations,
Victor L. Shapiro
Geometric Sturmian Theory of Nonlinear Parabolic Equations and Applications,
Victor A. Galaktionov
Green’s Functions and Linear Differential Equations: Theory, Applications,
and Computation, Prem K. Kythe
Introduction to Fuzzy Systems, Guanrong Chen and Trung Tat Pham
Introduction to non-Kerr Law Optical Solitons, Anjan Biswas and Swapan Konar
Introduction to Partial Differential Equations with MATLAB®, Matthew P. Coleman
Introduction to Quantum Control and Dynamics, Domenico D’Alessandro
Mathematical Methods in Physics and Engineering with Mathematica, Ferdinand F. Cap
Mathematical Theory of Quantum Computation, Goong Chen and Zijian Diao
Mathematics of Quantum Computation and Quantum Technology, Goong Chen,
Louis Kauffman, and Samuel J. Lomonaco
Mixed Boundary Value Problems, Dean G. Duffy
Modeling and Control in Vibrational and Structural Dynamics, Peng-Fei Yao
Multi-Resolution Methods for Modeling and Control of Dynamical Systems,
Puneet Singla and John L. Junkins
Optimal Estimation of Dynamic Systems, Second Edition, John L. Crassidis and John L. Junkins
Quantum Computing Devices: Principles, Designs, and Analysis, Goong Chen,
David A. Church, Berthold-Georg Englert, Carsten Henkel, Bernd Rohwedder,
Marlan O. Scully, and M. Suhail Zubairy
A Shock-Fitting Primer, Manuel D. Salas
Stochastic Partial Differential Equations, Pao-Liu Chow
© 2012 by Taylor & Francis Group, LLC
CHAPMAN & HALL/CRC APPLIED MATHEMATICS
AND NONLINEAR SCIENCE SERIES
Optimal Estimation of
Dynamic Systems
Second Edition
John L. Crassidis
University at Buffalo, State University of New York
Amherst, New York, USA
John L. Junkins
Texas A&M University
College Station, Texas, USA
© 2012 by Taylor & Francis Group, LLC
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2012 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 2011912
International Standard Book Number-13: 978-1-4398-3986-7 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and informa-
tion, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have
attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has
not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic,
mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval
system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copy-
right Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without
intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
© 2012 by Taylor & Francis Group, LLC
To Pamand Lucas, and inmemory ofLucas G.J.Crassidis
and
To Elouise, Stephen, and Kathryn
© 2012 by Taylor & Francis Group, LLC

Contents
Preface xiii
1 LeastSquaresApproximation 1
1.1 ACurveFittingExample . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 LinearBatchEstimation . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.1 LinearLeastSquares . . . . . . . . . . . . . . . . . . . . . 9
1.2.2 WeightedLeastSquares . . . . . . . . . . . . . . . . . . . 14
1.2.3 ConstrainedLeastSquares . . . . . . . . . . . . . . . . . . 16
1.3 LinearSequentialEstimation . . . . . . . . . . . . . . . . . . . . . 19
1.4 NonlinearLeastSquaresEstimation . . . . . . . . . . . . . . . . . 25
1.5 BasisFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
1.6 AdvancedTopics . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
1.6.1 MatrixDecompositionsinLeastSquares . . . . . . . . . . 40
1.6.2 KroneckerFactorizationandLeastSquares . . . . . . . . . 43
1.6.3 Levenberg-MarquardtMethod . . . . . . . . . . . . . . . . 48
1.6.4 ProjectionsinLeastSquares . . . . . . . . . . . . . . . . . 50
1.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2 ProbabilityConceptsinLeastSquares 63
2.1 MinimumVarianceEstimation . . . . . . . . . . . . . . . . . . . . 63
2.1.1 EstimationwithoutaprioriStateEstimates . . . . . . . . . 64
2.1.2 EstimationwithaprioriStateEstimates . . . . . . . . . . . 68
2.2 UnbiasedEstimates . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.3 Crame´r-RaoInequality . . . . . . . . . . . . . . . . . . . . . . . . 76
2.4 ConstrainedLeastSquaresCovariance . . . . . . . . . . . . . . . . 82
2.5 MaximumLikelihoodEstimation . . . . . . . . . . . . . . . . . . 84
2.6 PropertiesofMaximumLikelihoodEstimation . . . . . . . . . . . 88
2.6.1 InvariancePrinciple . . . . . . . . . . . . . . . . . . . . . 88
2.6.2 ConsistentEstimator . . . . . . . . . . . . . . . . . . . . . 88
2.6.3 AsymptoticallyGaussianProperty . . . . . . . . . . . . . . 90
2.6.4 AsymptoticallyEfficientProperty . . . . . . . . . . . . . . 90
2.7 BayesianEstimation . . . . . . . . . . . . . . . . . . . . . . . . . 91
2.7.1 MAPEstimation . . . . . . . . . . . . . . . . . . . . . . . 91
2.7.2 MinimumRiskEstimation . . . . . . . . . . . . . . . . . . 95
2.8 AdvancedTopics . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
2.8.1 NonuniquenessoftheWeightMatrix . . . . . . . . . . . . 98
2.8.2 AnalysisofCovarianceErrors . . . . . . . . . . . . . . . . 101
vii
© 2012 by Taylor & Francis Group, LLC
viii Contents
2.8.3 RidgeEstimation . . . . . . . . . . . . . . . . . . . . . . . 103
2.8.4 TotalLeastSquares . . . . . . . . . . . . . . . . . . . . . . 108
2.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3 SequentialStateEstimation 135
3.1 ASimpleFirst-OrderFilterExample . . . . . . . . . . . . . . . . 136
3.2 Full-OrderEstimators . . . . . . . . . . . . . . . . . . . . . . . . 138
3.2.1 Discrete-TimeEstimators . . . . . . . . . . . . . . . . . . 142
3.3 TheDiscrete-TimeKalmanFilter . . . . . . . . . . . . . . . . . . 143
3.3.1 KalmanFilterDerivation . . . . . . . . . . . . . . . . . . . 144
3.3.2 StabilityandJoseph’sForm . . . . . . . . . . . . . . . . . 149
3.3.3 InformationFilterandSequentialProcessing . . . . . . . . 151
3.3.4 Steady-StateKalmanFilter . . . . . . . . . . . . . . . . . . 153
3.3.5 RelationshiptoLeastSquaresEstimation . . . . . . . . . . 156
3.3.6 CorrelatedMeasurementandProcessNoise . . . . . . . . . 158
3.3.7 Crame´r-RaoLowerBound . . . . . . . . . . . . . . . . . . 159
3.3.8 OrthogonalityPrinciple. . . . . . . . . . . . . . . . . . . . 164
3.4 TheContinuous-TimeKalmanFilter . . . . . . . . . . . . . . . . . 168
3.4.1 KalmanFilterDerivationinContinuousTime . . . . . . . . 168
3.4.2 KalmanFilterDerivationfromDiscreteTime . . . . . . . . 171
3.4.3 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
3.4.4 Steady-StateKalmanFilter . . . . . . . . . . . . . . . . . . 176
3.4.5 CorrelatedMeasurementandProcessNoise . . . . . . . . . 182
3.5 TheContinuous-DiscreteKalmanFilter . . . . . . . . . . . . . . . 182
3.6 ExtendedKalmanFilter . . . . . . . . . . . . . . . . . . . . . . . 184
3.7 UnscentedFiltering . . . . . . . . . . . . . . . . . . . . . . . . . . 192
3.8 ConstrainedFiltering . . . . . . . . . . . . . . . . . . . . . . . . . 199
3.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
4 AdvancedTopicsinSequentialStateEstimation 219
4.1 FactorizationMethods . . . . . . . . . . . . . . . . . . . . . . . . 219
4.2 Colored-NoiseKalmanFiltering . . . . . . . . . . . . . . . . . . . 223
4.3 ConsistencyoftheKalmanFilter . . . . . . . . . . . . . . . . . . 228
4.4 ConsiderKalmanFiltering . . . . . . . . . . . . . . . . . . . . . . 231
4.4.1 ConsiderUpdateEquations. . . . . . . . . . . . . . . . . . 232
4.4.2 ConsiderPropagationEquations . . . . . . . . . . . . . . . 234
4.5 DecentralizedFiltering . . . . . . . . . . . . . . . . . . . . . . . . 238
4.5.1 CovarianceIntersection. . . . . . . . . . . . . . . . . . . . 240
4.6 AdaptiveFiltering . . . . . . . . . . . . . . . . . . . . . . . . . . 244
4.6.1 BatchProcessingforFilterTuning . . . . . . . . . . . . . . 244
4.6.2 Multiple-ModelingAdaptiveEstimation . . . . . . . . . . . 249
4.6.3 InteractingMultiple-ModelEstimation . . . . . . . . . . . 252
4.7 EnsembleKalmanFiltering . . . . . . . . . . . . . . . . . . . . . 257
4.8 NonlinearStochasticFilteringTheory . . . . . . . . . . . . . . . . 260
4.8.1 Itoˆ StochasticDifferentialEquations . . . . . . . . . . . . . 263
© 2012 by Taylor & Francis Group, LLC
Contents ix
4.8.2 Itoˆ Formula . . . . . . . . . . . . . . . . . . . . . . . . . . 265
4.8.3 Fokker-PlanckEquation . . . . . . . . . . . . . . . . . . . 267
4.8.4 KushnerEquation. . . . . . . . . . . . . . . . . . . . . . . 269
4.9 GaussianSumFiltering . . . . . . . . . . . . . . . . . . . . . . . . 270
4.10 ParticleFiltering . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
4.10.1 OptimalImportanceDensity . . . . . . . . . . . . . . . . . 277
4.10.2 BootstrapFilter . . . . . . . . . . . . . . . . . . . . . . . . 279
4.10.3 Rao-BlackwellizedParticleFilter . . . . . . . . . . . . . . 287
4.10.4 NavigationUsingaRao-BlackwellizedParticleFilter . . . . 291
4.11 ErrorAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
4.12 RobustFiltering . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
4.13 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
5 BatchStateEstimation 325
5.1 Fixed-IntervalSmoothing . . . . . . . . . . . . . . . . . . . . . . 326
5.1.1 Discrete-TimeFormulation. . . . . . . . . . . . . . . . . . 327
5.1.2 Continuous-TimeFormulation . . . . . . . . . . . . . . . . 339
5.1.3 NonlinearSmoothing . . . . . . . . . . . . . . . . . . . . . 349
5.2 Fixed-PointSmoothing . . . . . . . . . . . . . . . . . . . . . . . . 353
5.2.1 Discrete-TimeFormulation. . . . . . . . . . . . . . . . . . 353
5.2.2 Continuous-TimeFormulation . . . . . . . . . . . . . . . . 357
5.3 Fixed-LagSmoothing . . . . . . . . . . . . . . . . . . . . . . . . 360
5.3.1 Discrete-TimeFormulation. . . . . . . . . . . . . . . . . . 360
5.3.2 Continuous-TimeFormulation . . . . . . . . . . . . . . . . 363
5.4 AdvancedTopics . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
5.4.1 Estimation/ControlDuality . . . . . . . . . . . . . . . . . . 367
5.4.2 InnovationsProcess. . . . . . . . . . . . . . . . . . . . . . 375
5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
6 ParameterEstimation:Applications 391
6.1 AttitudeDetermination . . . . . . . . . . . . . . . . . . . . . . . . 391
6.1.1 VectorMeasurementModels . . . . . . . . . . . . . . . . . 392
6.1.2 MaximumLikelihoodEstimation . . . . . . . . . . . . . . 395
6.1.3 OptimalQuaternionSolution . . . . . . . . . . . . . . . . . 396
6.1.4 InformationMatrixAnalysis . . . . . . . . . . . . . . . . . 400
6.2 GlobalPositioningSystemNavigation . . . . . . . . . . . . . . . . 403
6.3 SimultaneousLocalizationandMapping . . . . . . . . . . . . . . 407
6.3.1 3DPointCloudRegistrationUsingLinearLeastSquares . . 408
6.4 OrbitDetermination . . . . . . . . . . . . . . . . . . . . . . . . . 411
6.5 AircraftParameterIdentification . . . . . . . . . . . . . . . . . . . 419
6.6 EigensystemRealizationAlgorithm . . . . . . . . . . . . . . . . . 425
6.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
© 2012 by Taylor & Francis Group, LLC