Table Of Content(cid:2)
ModelIdentificationandDataAnalysis
(cid:2) (cid:2)
(cid:2)
(cid:2)
Model Identification and Data Analysis
SergioBittanti
PolitecnicodiMilano
Milan,Italy
(cid:2) (cid:2)
(cid:2)
(cid:2)
Thiseditionfirstpublished2019
©2019JohnWiley&Sons,Inc.
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,or
transmitted,inanyformorbyanymeans,electronic,mechanical,photocopying,recordingor
otherwise,exceptaspermittedbylaw.Adviceonhowtoobtainpermissiontoreusematerialfrom
thistitleisavailableathttp://www.wiley.com/go/permissions.
TherightofSergioBittantitobeidentifiedastheauthorofthisworkhasbeenassertedin
accordancewithlaw.
RegisteredOffice
JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,USA
EditorialOffice
111RiverStreet,Hoboken,NJ07030,USA
Fordetailsofourglobaleditorialoffices,customerservices,andmoreinformationaboutWiley
productsvisitusatwww.wiley.com.
Wileyalsopublishesitsbooksinavarietyofelectronicformatsandbyprint-on-demand.Some
contentthatappearsinstandardprintversionsofthisbookmaynotbeavailableinotherformats.
LimitofLiability/DisclaimerofWarranty
Whilethepublisherandauthorshaveusedtheirbesteffortsinpreparingthiswork,theymakeno
representationsorwarrantieswithrespecttotheaccuracyorcompletenessofthecontentsofthis
workandspecificallydisclaimallwarranties,includingwithoutlimitationanyimpliedwarranties
ofmerchantabilityorfitnessforaparticularpurpose.Nowarrantymaybecreatedorextendedby
salesrepresentatives,writtensalesmaterialsorpromotionalstatementsforthiswork.Thefact
thatanorganization,website,orproductisreferredtointhisworkasacitationand/orpotential
sourceoffurtherinformationdoesnotmeanthatthepublisherandauthorsendorsethe
(cid:2) (cid:2)
informationorservicestheorganization,website,orproductmayprovideorrecommendationsit
maymake.Thisworkissoldwiththeunderstandingthatthepublisherisnotengagedin
renderingprofessionalservices.Theadviceandstrategiescontainedhereinmaynotbesuitable
foryoursituation.Youshouldconsultwithaspecialistwhereappropriate.Further,readersshould
beawarethatwebsiteslistedinthisworkmayhavechangedordisappearedbetweenwhenthis
workwaswrittenandwhenitisread.Neitherthepublishernorauthorsshallbeliableforanyloss
ofprofitoranyothercommercialdamages,includingbutnotlimitedtospecial,incidental,
consequential,orotherdamages.
LibraryofCongressCataloging-in-PublicationData
Names:Bittanti,Sergio,author.
Title:Modelidentificationanddataanalysis/SergioBittanti,Politecnico
diMilano,Milan,Italy.
Description:Hoboken,NJ,USA:Wiley,[2019]|Includesbibliographical
referencesandindex.|
Identifiers:LCCN2018046965(print)|LCCN2018047956(ebook)|ISBN
9781119546412(AdobePDF)|ISBN9781119546313(ePub)|ISBN9781119546368
(hardcover)
Subjects:LCSH:Mathematicalmodels.|Quantitativeresearch.|System
identification.
Classification:LCCTA342(ebook)|LCCTA342.B582019(print)|DDC
511/.8–dc23
LCrecordavailableathttps://lccn.loc.gov/2018046965
Coverdesign:Wiley
Coverimage:©OleksiiLishchyshyn/Shutterstock,©gremlin/GettyImages
Setin10/12ptWarnockProbySPiGlobal,Chennai,India
PrintedintheUnitedStatesofAmerica
10 9 8 7 6 5 4 3 2 1
(cid:2)
(cid:2)
v
Contents
Introduction xi
Acknowledgments xv
1 StationaryProcessesandTimeSeries 1
1.1 Introduction 1
1.2 ThePredictionProblem 1
1.3 RandomVariable 4
1.4 RandomVector 5
(cid:2) 1.4.1 CovarianceCoefficient 7 (cid:2)
1.5 StationaryProcess 9
1.6 WhiteProcess 11
1.7 MAProcess 12
1.8 ARProcess 16
1.8.1 StudyoftheAR(1)Process 16
1.9 Yule–WalkerEquations 20
1.9.1 Yule–WalkerEquationsfortheAR(1)Process 20
1.9.2 Yule–WalkerEquationsfortheAR(2)andAR(n)Process 21
1.10 ARMAProcess 23
1.11 SpectrumofaStationaryProcess 24
1.11.1 SpectrumProperties 24
1.11.2 SpectralDiagram 25
1.11.3 MaximumFrequencyinDiscreteTime 25
1.11.4 WhiteNoiseSpectrum 25
1.11.5 ComplexSpectrum 26
1.12 ARMAModel:StabilityTestandVarianceComputation 26
1.12.1 RuzickaStabilityCriterion 28
1.12.2 VarianceofanARMAProcess 32
1.13 FundamentalTheoremofSpectralAnalysis 35
1.14 SpectrumDrawing 38
1.15 ProofoftheFundamentalTheoremofSpectralAnalysis 43
1.16 RepresentationsofaStationaryProcess 45
(cid:2)
(cid:2)
vi Contents
2 EstimationofProcessCharacteristics 47
2.1 Introduction 47
2.2 GeneralPropertiesoftheCovarianceFunction 47
2.3 CovarianceFunctionofARMAProcesses 49
2.4 EstimationoftheMean 50
2.5 EstimationoftheCovarianceFunction 53
2.6 EstimationoftheSpectrum 55
2.7 WhitenessTest 57
3 Prediction 61
3.1 Introduction 61
3.2 FakePredictor 62
3.2.1 PracticalDeterminationoftheFakePredictor 64
3.3 SpectralFactorization 66
3.4 WhiteningFilter 70
3.5 OptimalPredictorfromData 71
3.6 PredictionofanARMAProcess 76
3.7 ARMAXProcess 77
3.8 PredictionofanARMAXProcess 78
(cid:2) 4 ModelIdentification 81 (cid:2)
4.1 Introduction 81
4.2 SettingtheIdentificationProblem 82
4.2.1 LearningfromMaxwell 82
4.2.2 AGeneralIdentificationProblem 84
4.3 StaticModeling 85
4.3.1 LearningfromGauss 85
4.3.2 LeastSquaresMadeSimple 86
4.3.2.1 TrendSearch 86
4.3.2.2 SeasonalitySearch 86
4.3.2.3 LinearRegression 87
4.3.3 EstimatingtheExpansionoftheUniverse 90
4.4 DynamicModeling 92
4.5 ExternalRepresentationModels 92
4.5.1 BoxandJenkinsModel 92
4.5.2 ARXandARModels 93
4.5.3 ARMAXandARMAModels 94
4.5.4 MultivariableModels 96
4.6 InternalRepresentationModels 96
4.7 TheModelIdentificationProcess 100
4.8 ThePredictiveApproach 101
4.9 ModelsinPredictiveForm 102
4.9.1 BoxandJenkinsModel 103
(cid:2)
(cid:2)
Contents vii
4.9.2 ARXandARModels 103
4.9.3 ARMAXandARMAModels 104
5 IdentificationofInput–OutputModels 107
5.1 Introduction 107
5.2 EstimatingARandARXModels:TheLeastSquaresMethod 107
5.3 Identifiability 110
̄
5.3.1 TheRMatrixfortheARX(1,1)Model 111
̄
5.3.2 TheRMatrixforaGeneralARXModel 112
5.4 EstimatingARMAandARMAXModels 115
5.4.1 ComputingtheGradientandtheHessianfromData 117
5.5 AsymptoticAnalysis 123
5.5.1 DataGenerationSystemWithintheClassofModels 125
5.5.2 DataGenerationSystemOutsidetheClassofModels 127
5.5.2.1 SimulationTrial 132
5.5.3 GeneralConsiderationsontheAsymptoticsofPredictive
Identification 132
5.5.4 EstimatingtheUncertaintyinParameterEstimation 132
5.5.4.1 DeductionoftheFormulaoftheEstimationCovariance 134
5.6 RecursiveIdentification 138
(cid:2) 5.6.1 RecursiveLeastSquares 138 (cid:2)
5.6.2 RecursiveMaximumLikelihood 143
5.6.3 ExtendedLeastSquares 145
5.7 RobustnessofIdentificationMethods 147
5.7.1 PredictionErrorandModelError 147
5.7.2 FrequencyDomainInterpretation 148
5.7.3 Prefiltering 149
5.8 ParameterTracking 149
6 ModelComplexitySelection 155
6.1 Introduction 155
6.2 Cross-validation 157
6.3 FPECriterion 157
6.3.1 FPEConcept 157
6.3.2 FPEDetermination 158
6.4 AICCriterion 160
6.4.1 AICVersusFPE 161
6.5 MDLCriterion 161
6.5.1 MDLVersusAIC 162
6.6 Durbin–LevinsonAlgorithm 164
6.6.1 Yule–WalkerEquationsforAutoregressiveModelsofOrders1
and2 165
6.6.2 Durbin–LevinsonRecursion:FromAR(1)toAR(2) 166
(cid:2)
(cid:2)
viii Contents
6.6.3 Durbin–LevinsonRecursionforModelsofAnyOrder 169
6.6.4 PartialCovarianceFunction 171
7 IdentificationofStateSpaceModels 173
7.1 Introduction 173
7.2 HankelMatrix 175
7.3 OrderDetermination 176
7.4 DeterminationofMatricesGandH 177
7.5 DeterminationofMatrixF 178
7.6 MidSummary:AnIdealProcedure 179
7.7 OrderDeterminationwithSVD 179
7.8 ReliableIdentificationofaStateSpaceModel 181
8 PredictiveControl 187
8.1 Introduction 187
8.2 MinimumVarianceControl 188
8.2.1 DeterminationoftheMVControlLaw 190
8.2.2 AnalysisoftheMVControlSystem 192
8.2.2.1 Structure 193
8.2.2.2 Stability 193
(cid:2) 8.3 GeneralizedMinimumVarianceControl 196 (cid:2)
8.3.1 ModelReferenceControl 198
8.3.2 PenalizedControlDesign 200
8.3.2.1 ChoiceAforQ(z) 201
8.3.2.2 ChoiceBforQ(z) 203
8.4 Model-BasedPredictiveControl 204
8.5 Data-DrivenControlSynthesis 205
9 KalmanFilteringandPrediction 209
9.1 Introduction 209
9.2 KalmanApproachtoPredictionandFilteringProblems 210
9.3 TheBayesEstimationProblem 212
9.3.1 BayesProblem–ScalarCase 213
9.3.2 BayesProblem–VectorCase 215
9.3.3 RecursiveBayesFormula–ScalarCase 215
9.3.4 Innovation 217
9.3.5 RecursiveBayesFormula–VectorCase 219
9.3.6 GeometricInterpretationofBayesEstimation 220
9.3.6.1 GeometricInterpretationoftheBayesBatchFormula 220
9.3.6.2 GeometricInterpretationoftheRecursiveBayesFormula 222
9.4 One-step-aheadKalmanPredictor 223
9.4.1 TheInnovationintheStatePredictionProblem 224
9.4.2 TheStatePredictionError 224
9.4.3 OptimalOne-Step-AheadPredictionoftheOutput 225
9.4.4 OptimalOne-Step-AheadPredictionoftheState 226
(cid:2)
(cid:2)
Contents ix
9.4.5 RiccatiEquation 228
9.4.6 Initialization 231
9.4.7 One-step-aheadOptimalPredictorSummary 232
9.4.8 Generalizations 236
9.4.8.1 System 236
9.4.8.2 Predictor 236
9.5 MultistepOptimalPredictor 237
9.6 OptimalFilter 239
9.7 Steady-StatePredictor 240
9.7.1 GainConvergence 241
9.7.2 ConvergenceoftheRiccatiEquationSolution 244
9.7.2.1 ConvergenceUnderStability 244
9.7.2.2 ConvergenceWithoutStability 246
9.7.2.3 Observability 250
9.7.2.4 Reachability 251
9.7.2.5 GeneralConvergenceResult 256
9.8 InnovationRepresentation 265
9.9 InnovationRepresentationVersusCanonical
Representation 266
9.10 K-TheoryVersusK–WTheory 267
9.11 ExtendedKalmanFilter–EKF 271
(cid:2) (cid:2)
9.12 TheRobustApproachtoFiltering 273
9.12.1 NormofaDynamicSystem 274
9.12.2 RobustFiltering 276
10 ParameterIdentificationinaGivenModel 281
10.1 Introduction 281
10.2 KalmanFilter-BasedApproaches 281
10.3 Two-StageMethod 284
10.3.1 FirstStage–DataGenerationandCompression 285
10.3.2 SecondStage–CompressedDataFitting 287
11 CaseStudies 291
11.1 Introduction 291
11.2 KobeEarthquakeDataAnalysis 291
11.2.1 ModelingtheNormalSeismicActivityData 294
11.2.2 ModelValidation 296
11.2.3 AnalysisoftheTransitionPhaseviaDetectionTechniques 299
11.2.4 Conclusions 300
11.3 EstimationofaSinusoidinNoise 300
11.3.1 FrequencyEstimationbyNotchFilterDesign 301
11.3.2 FrequencyEstimationwithEKF 305
AppendixA LinearDynamicalSystems 309
A.1 StateSpaceandInput–OutputModels 309
(cid:2)
(cid:2)
x Contents
A.1.1 CharacteristicPolynomialandEigenvalues 309
A.1.2 OperatorRepresentation 310
A.1.3 TransferFunction 310
A.1.4 Zeros,Poles,andEigenvalues 310
A.1.5 RelativeDegree 311
A.1.6 EquilibriumPointandSystemGain 311
A.2 LagrangeFormula 312
A.3 Stability 312
A.4 ImpulseResponse 313
A.4.1 ImpulseResponsefromaStateSpaceModel 314
A.4.2 ImpulseResponsefromanInput–OutputModel 314
A.4.3 QuadraticSummabilityoftheImpulseResponse 315
A.5 FrequencyResponse 315
A.6 MultiplicityofStateSpaceModels 316
A.6.1 ChangeofBasis 316
A.6.2 RedundancyintheSystemOrder 317
A.7 ReachabilityandObservability 318
A.7.1 Reachability 318
A.7.2 Observability 320
A.7.3 PBHTestofReachabilityandObservability 321
A.8 SystemDecomposition 323
(cid:2) (cid:2)
A.8.1 ReachabilityandObservabilityDecompositions 323
A.8.2 CanonicalDecomposition 324
A.9 StabilizabilityandDetectability 328
AppendixB Matrices 331
B.1 Basics 331
B.2 Eigenvalues 335
B.3 DeterminantandInverse 337
B.4 Rank 340
B.5 AnnihilatingPolynomial 342
B.6 AlgebraicandGeometricMultiplicity 345
B.7 RangeandNullSpace 345
B.8 QuadraticForms 346
B.9 DerivativeofaScalarFunctionwithRespecttoaVector 349
B.10 MatrixDiagonalizationviaSimilarity 350
B.11 MatrixDiagonalizationviaSingularValueDecomposition 351
B.12 MatrixNormandConditionNumber 353
AppendixC ProblemsandSolutions 357
Bibliography 391
Index 397
(cid:2)
