Linear Stochastic Systems: A Geometric Approach to Modeling, Estimation and Identification

788 Pages·2015·8.661 MB·English

by  Anders Lindquist, Giorgio Picci (auth.)

Series in Contemporary Mathematics 1
Anders Lindquist
Giorgio Picci
Linear
Stochastic
Systems
A Geometric Approach to Modeling,
Estimation and Identifi cation
Series in Contemporary Mathematics
Volume 1
Editor-in-Chief
TatsienLi
Editors
PhilippeG.Ciarlet Jean-MichelCoron WeinanE
JianshuLi JunLi TatsienLi FanghuaLin
Zhi-mingMa AndrewJ.Majda CédricVillani
Ya-xiangYuan WeipingZhang
Series in Contemporary Mathematics, featuring high-quality mathematical
monographs, is to presents original and systematic findings from the fields of
puremathematics,appliedmathematicsandmath-relatedinterdisciplinarysubjects.
IthasahistoryofoverfiftyyearssincethefirsttitlewaspublishedbyShanghai
Scientific&TechnicalPublishersin1963.ProfessorHUALuogeng(Lo-KengHua)
served as Editor-in-Chief of the first editorial board, while Professor SU Buqing
actedasHonoraryEditor-in-ChiefandProfessorGUChaohaoasEditor-in-Chiefof
thesecondeditorialboardsince 1992.Now thethirdeditorialboardis established
andProfessorLITatsienassumesthepositionofEditor-in-Chief.
Theserieshasalreadypublishedtwenty-sixmonographsinChinese,andamong
theauthorsaremanydistinguishedChinesemathematicians,includingthefollowing
members of the Chinese Academy of Sciences: SU Buqing, GU Chaohao, LU
Qikeng, ZHANG Gongqing, CHEN Hanfu, CHEN Xiru, YUAN Yaxiang, CHEN
Shuxingetc.Themonographshavesystematicallyintroducedanumberofimportant
research findings which not only play a vital role in China, but also exert huge
influence all over the world. Eight of them have been translated into English and
publishedabroad.
The new editorial board will inherit and carry forward the former traditions
and strengths of the series, and plan to further reform and innovation in terms of
internalization so as to improve and ensure the quality of the series, extend its
global influence, and strive to forge it into an internationally significant series of
mathematicalmonographs.
Moreinformationaboutthisseriesathttp://www.springer.com/series/13634
Anders Lindquist • Giorgio Picci
Linear Stochastic Systems
A Geometric Approach to Modeling,
Estimation and Identification
123
AndersLindquist GiorgioPicci
DepartmentsofAutomation DepartmentofInformationEngineering
andMathematics UniversityofPadova
ShanghaiJiaotongUniversity Padova,Italy
Shanghai,China
DepartmentofMathematics
RoyalInstituteofTechnology
Stockholm,Sweden
ISSN2364-009X ISSN2364-0103 (electronic)
SeriesinContemporaryMathematics
ISBN978-3-662-45749-8 ISBN978-3-662-45750-4 (eBook)
DOI10.1007/978-3-662-45750-4
LibraryofCongressControlNumber:2015936904
MathematicsSubjectClassification:93-XX,60-XX,47L05,47L30
SpringerHeidelbergNewYorkDordrechtLondon
©Springer-VerlagBerlinHeidelberg2015
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
Springer-VerlagGmbHBerlinHeidelbergispartofSpringerScience+Business Media(www.springer.
com)
Preface
This book is intended to be a treatise on the theory and modeling of second-
orderstationary processeswith an expositionof some applicationareas which we
believeareimportantinengineeringandappliedsciences.Thefoundationalissues
regardingstationaryprocessesdealtwithin thebeginningofthebookhavea long
history and have been developed in the literature starting at least from the 1940s
withtheworkofKolmogorov,Wiener,Cramèrandhisstudents,inparticularWold,
and have been refined and complemented by many others. Problems of filtering
and modelingof stationary random signals and systems have also been addressed
and studied, fostered by the advent of modern digital computers, since the early
1960s with the fundamental work of R.E. Kalman. Classical books on random
processesdidnotaddresstheselastissues,inparticularstate-spacemodeling,which
isparticularlyimportantinapplications.
Whenwe firststartedplanningthisbookseveraldecadesago,the drivingforce
was ourconvictionthatbasic resultson modeling,estimationandidentificationin
the literature were presented in a rather scattered and incomplete way, sometimes
obscured by formula manipulations, and, in our opinion, a lack of a conceptual
threadunifyingthevarioussubjectswasparticularlyevident.Manydetailsandsome
technicalaspectsthatoccurfrequentlyaretraditionallyignoredalsotoday.Forthis
reason, we wanted to offer a unified and logically consistent view of the subject
based on simple ideas from Hilbert space geometry and coordinate-freethinking.
Inthisframework,theconceptsofstochasticstatespaceandstate-spacemodeling
based on the idea of conditionalindependenceof the past and future flows of the
relevantsignalsturnouttobeafundamentalunifyingidea.
Since then, books by P.E. Caines, Gy. Michaletzky et al. and others have
appeared,coveringsomeoftheseconcepts,howeverwithadifferentfocusthanwe
hadplanned.Alsoasthesubjecthasevolved,newtheoryandapplicationshavebeen
added. Most of the material presented in this monographhas appeared in journal
papers,buttherearealsonewresultsappearinghereforthefirsttime.
Being the result of some decadesof jointeffort, this is notmeantto be a book
for “seasonal consumption”. Quite immodestly, we imagine (or at least wish) it
to be a lasting reference for students and researchers interested in this important
v
vi Preface
and beautiful area of applied mathematics. We are indebted to a large number of
coworkers in developing the theory of this book, especially G. Ruckebusch, M.
Pavon,F.Badawi,Gy.Michaletzky,A.ChiusoandA.Ferrante,butalsoC.I.Byrnes,
S.V. Gusev, A. Blomqvist, R. Nagamune and P. Enqvist, as well as to number
of colleagues, among them O. Staffans, J. Malinen, P. Enqvist, Gy. Michaletzky,
T.T. Georgiou, J. Karlsson and A. Ringh for reading parts of the manuscript and
providingvaluablesuggestionsforimprovement.
Shanghai,China,andStockholm,Sweden AndersLindquist
Padova,Italy GiorgioPicci
Contents
1 Introduction................................................................. 1
1.1 GeometricTheoryofStochasticRealization........................ 2
1.1.1 MarkovianSplittingSubspaces............................ 4
1.1.2 Observability,ConstructibilityandMinimality ........... 4
1.1.3 FundamentalRepresentationTheorem.................... 6
1.1.4 PredictorSpacesandPartialOrdering..................... 8
1.1.5 TheFrameSpace ........................................... 9
1.1.6 Generalizations ............................................. 10
1.2 SpectralFactorizationandUniformlyChosenBases............... 10
1.2.1 TheLinearMatrixInequalityandHankelFactorization.. 11
1.2.2 Minimality.................................................. 13
1.2.3 RationalCovarianceExtension ............................ 13
1.2.4 UniformChoiceofBases .................................. 14
1.2.5 TheMatrixRiccatiEquation............................... 15
1.3 Applications.......................................................... 16
1.3.1 Smoothing .................................................. 16
1.3.2 Interpolation ................................................ 17
1.3.3 SubspaceIdentification..................................... 17
1.3.4 BalancedModelReduction ................................ 19
1.4 AnBriefOutlineoftheBook........................................ 21
1.5 BibliographicalNotes................................................ 23
2 GeometryofSecond-OrderRandomProcesses......................... 25
2.1 HilbertSpaceofSecond-OrderRandomVariables................. 25
2.1.1 NotationsandConventions................................. 26
2.2 OrthogonalProjections............................................... 27
2.2.1 LinearEstimationandOrthogonalProjections............ 28
2.2.2 FactsAboutOrthogonalProjections....................... 31
2.3 AnglesandSingularValues.......................................... 33
2.3.1 CanonicalCorrelationAnalysis............................ 36
2.4 ConditionalOrthogonality........................................... 38
vii
viii Contents
2.5 Second-OrderProcessesandtheShiftOperator.................... 40
2.5.1 Stationarity.................................................. 42
2.6 ConditionalOrthogonalityandModeling........................... 44
2.6.1 TheMarkovProperty....................................... 44
2.6.2 StochasticDynamicalSystems............................. 47
2.6.3 FactorAnalysis ............................................. 49
2.6.4 ConditionalOrthogonalityandCovarianceSelection .... 55
2.6.5 CausalityandFeedback-FreeProcesses................... 57
2.7 ObliqueProjections .................................................. 58
2.7.1 Computing Oblique Projections in the
Finite-DimensionalCase................................... 61
2.8 StationaryIncrementsProcessesinContinuousTime.............. 62
2.9 BibliographicalNotes................................................ 63
3 SpectralRepresentationofStationaryProcesses ....................... 65
3.1 Orthogonal-IncrementsProcessesandtheWienerIntegral ........ 65
3.2 HarmonicAnalysisofStationaryProcesses ........................ 70
3.3 TheSpectralRepresentationTheorem .............................. 73
3.3.1 ConnectionstotheClassicalDefinitionof
StochasticFourierTransform.............................. 75
3.3.2 Continuous-TimeSpectralRepresentation................ 77
3.3.3 RemarkonDiscrete-TimeWhiteNoise ................... 78
3.3.4 RealProcesses.............................................. 78
3.4 Vector-ValuedProcesses............................................. 79
3.5 FunctionalsofWhiteNoise.......................................... 82
3.5.1 TheFourierTransform ..................................... 85
3.6 SpectralRepresentationofStationaryIncrementProcesses........ 88
3.7 MultiplicityandtheModuleStructureofH.y/..................... 91
3.7.1 DefinitionofMultiplicityandtheModule
StructureofH.y/........................................... 92
3.7.2 BasesandSpectralFactorization .......................... 95
3.7.3 ProcesseswithanAbsolutelyContinuous
DistributionMatrix......................................... 99
3.8 BibliographicalNotes................................................ 100
4 Innovations,WoldDecomposition,andSpectralFactorization....... 103
4.1 TheWiener-KolmogorovTheoryofFilteringandPrediction...... 103
4.1.1 The Role of the Fourier Transformand
SpectralRepresentation .................................... 104
4.1.2 AcausalandCausalWienerFilters ........................ 105
4.1.3 CausalWienerFiltering.................................... 108
4.2 OrthonormalizableProcessesandSpectralFactorization .......... 110
4.3 HardySpaces......................................................... 115
4.4 AnalyticSpectralFactorization...................................... 118
4.5 TheWoldDecomposition............................................ 119
4.5.1 Reversibility ................................................ 127
Contents ix
4.6 TheOuterSpectralFactor............................................ 129
4.6.1 InvariantSubspacesandtheFactorizationTheorem...... 131
4.6.2 InnerFunctions............................................. 135
4.6.3 ZerosofOuterFunctions................................... 136
4.7 ToeplitzMatricesandtheSzegöFormula........................... 138
4.7.1 AlgebraicPropertiesofToeplitzMatrices................. 146
4.8 BibliographicalNotes................................................ 150
5 SpectralFactorizationinContinuousTime ............................. 153
5.1 TheContinuous-TimeWoldDecomposition........................ 153
5.2 HardySpacesoftheHalf-Plane ..................................... 154
5.3 AnalyticSpectralFactorizationinContinuousTime............... 159
5.3.1 OuterSpectralFactorsinW2 .............................. 160
5.4 WideSenseSemimartingales........................................ 163
5.4.1 StationaryIncrementsSemimartingales................... 166
5.5 StationaryIncrementsSemimartingalesintheSpectralDomain .. 168
5.5.1 ProofofTheorem5.4.4..................................... 171
5.5.2 DegenerateStationaryIncrementsProcesses.............. 172
5.6 BibliographicalNotes................................................ 174
6 LinearFinite-DimensionalStochasticSystems.......................... 175
6.1 StochasticStateSpaceModels ...................................... 175
6.2 AnticausalStateSpaceModels...................................... 179
6.3 GeneratingProcessesandtheStructuralFunction.................. 183
6.4 TheIdeaofStateSpaceandCoordinate-FreeRepresentation..... 186
6.5 Observability,ConstructibilityandMinimality..................... 188
6.6 TheForwardandtheBackwardPredictorSpaces.................. 191
6.7 TheSpectralDensityandAnalyticSpectralFactors ............... 196
6.7.1 TheConverseProblem ..................................... 198
6.8 Regularity............................................................. 204
6.9 TheRiccatiInequalityandKalmanFiltering ....................... 207
6.10 BibliographicNotes.................................................. 213
7 TheGeometryofSplittingSubspaces.................................... 215
7.1 Deterministic Realization TheoryRevisited: The
AbstractIdeaofStateSpaceConstruction.......................... 215
7.2 PerpendicularIntersection........................................... 217
7.3 SplittingSubspaces................................................... 220
7.4 MarkovianSplittingSubspaces...................................... 225
7.5 TheMarkovSemigroup.............................................. 232
7.6 MinimalityandDimension .......................................... 234
7.7 PartialOrderingofMinimalSplittingSubspaces................... 238
7.7.1 UniformChoicesofBases ................................. 240
7.7.2 OrderingandScatteringPairs.............................. 243
7.7.3 TheTightestInternalBounds.............................. 246
7.8 BibliographicNotes.................................................. 250