Subspace Methods for System Identification: A Realization Approach PDF
Preview Subspace Methods for System Identification: A Realization Approach
Communications and Control Engineering Publishedtitlesinclude: StabilityandStabilizationofInfiniteDimensionalSystemswithApplications Zheng-HuaLuo,Bao-ZhuGuoandOmerMorgul NonsmoothMechanics(Secondedition) BernardBrogliato NonlinearControlSystemsII AlbertoIsidori L2-GainandPassivityTechniquesinnonlinearControl ArjanvanderSchaft ControlofLinearSystemswithRegulationandInputConstraints AliSaberi,AntonA.StoorvogelandPeddapullaiahSannuti RobustandH∞Control BenM.Chen ComputerControlledSystems EfimN.RosenwasserandBernhardP.Lampe DissipativeSystemsAnalysisandControl RogelioLozano,BernardBrogliato,OlavEgelandandBernhardMaschke ControlofComplexandUncertainSystems StanislavV.EmelyanovandSergeyK.Korovin RobustControlDesignUsingH∞Methods IanR.Petersen,ValeryA.UgrinovskiandAndreyV.Savkin ModelReductionforControlSystemDesign GoroObinataandBrianD.O.Anderson ControlTheoryforLinearSystems HarryL.Trentelman,AntonStoorvogelandMaloHautus FunctionalAdaptiveControl SimonG.FabriandVisakanKadirkamanathan Positive1Dand2DSystems TadeuszKaczorek IdentificationandControlUsingVolterraModels FrancisJ.DoyleIII,RonaldK.PearsonandBobatundeA.Ogunnaike Non-linearControlforUnderactuatedMechanicalSystems IsabelleFantoniandRogelioLozano RobustControl(Secondedition) JürgenAckermann FlowControlbyFeedback OleMortenAamoandMiroslavKrsti´c LearningandGeneralization(Secondedition) MathukumalliVidyasagar ConstrainedControlandEstimation GrahamC.Goodwin,MaríaM.SeronandJoséA.DeDoná RandomizedAlgorithmsforAnalysisandControlofUncertainSystems RobertoTempo,GiuseppeCalafioreandFabrizioDabbene SwitchedLinearSystems ZhendongSunandShuzhiS.Ge Tohru Katayama Subspace Methods for System Identification With66Figures 123 TohruKatayama,PhD DepartmentofAppliedMathematicsandPhysics, GraduateSchoolofInformatics,KyotoUniversity,Kyoto606-8501,Japan SeriesEditors E.D.Sontag·M.Thoma·A.Isidori·J.H.vanSchuppen BritishLibraryCataloguinginPublicationData Katayama,Tohru,1942- Subspacemethodsforsystemidentification:arealization approach.-(Communicationsandcontrolengineering) 1.Systemindentification2.Stochasticanalysis I.Title 003.1 ISBN-10:1852339810 LibraryofCongressControlNumber:2005924307 Apartfromanyfairdealingforthepurposesofresearchorprivatestudy,orcriticismorreview,as permittedundertheCopyright,DesignsandPatentsAct1988,thispublicationmayonlybereproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers,orinthecaseofreprographicreproductioninaccordancewiththetermsoflicencesissued bytheCopyrightLicensingAgency.Enquiriesconcerningreproductionoutsidethosetermsshouldbe senttothepublishers. CommunicationsandControlEngineeringSeriesISSN0178-5354 ISBN-10 1-85233-981-0 ISBN-13 978-1-85233-981-4 SpringerScience+BusinessMedia springeronline.com ©Springer-VerlagLondonLimited2005 MATLAB®istheregisteredtrademarkofTheMathWorks,Inc.,3AppleHillDriveNatick,MA01760- 2098,U.S.A.http://www.mathworks.com Theuseofregisterednames,trademarks,etc.inthispublicationdoesnotimply,evenintheabsenceof aspecificstatement,thatsuchnamesareexemptfromtherelevantlawsandregulationsandtherefore freeforgeneraluse. Thepublishermakesnorepresentation,expressorimplied,withregardtotheaccuracyoftheinfor- mationcontainedinthisbookandcannotacceptanylegalresponsibilityorliabilityforanyerrorsor omissionsthatmaybemade. Typesetting:Camerareadybyauthor Production:LE-TEXJelonek,Schmidt&VöcklerGbR,Leipzig,Germany PrintedinGermany 69/3141-543210Printedonacid-freepaperSPIN11370000 To myfamily Preface Numerouspapersonsystemidentificationhavebeenpublishedoverthelast40years. Though there were substantial developmentsin the theory of stationary stochastic processesandmultivariablestatisticalmethodsduring1950s,itiswidelyrecognized thatthetheoryofsystemidentificationstartedonlyinthemid-1960swith thepub- lication of two importantpapers;onedue to A˚stro¨mand Bohlin[17],in which the maximumlikelihood(ML)methodwasextendedtoaseriallycorrelatedtimeseries to estimate ARMAX models, and the other due to Ho and Kalman [72], in which thedeterministicstatespacerealizationproblemwassolvedforthefirsttimeusinga certainHankelmatrixformedintermsofimpulseresponses.Thesetwopapershave laid the foundationfor thefuturedevelopmentsof system identificationtheoryand techniques[55]. The scope of the ML identification method of A˚stro¨m and Bohlin [17] was to build single-input, single-output(SISO) ARMAX models from observed input- outputdatasequences.Sincetheappearanceoftheirpaper,manystatisticalidentifi- cationtechniqueshavebeendevelopedintheliterature,mostofwhicharenowcom- prised under the label ofpredictionerror methods(PEM)or instrumentalvariable (IV)methods.ThishasculminatedinthepublicationofthevolumesLjung[109]and So¨derstro¨mandStoica[145].Atthismomentwecansaythattheoryofsystemiden- tification forSISO systemsis established,andthe variousidentificationalgorithms havebeenwelltested,andarenowavailableasMATLABR programs. (cid:0) Also,identificationofmulti-input,multi-output(MIMO)systemsisanimportant problemwhich is notdealt with satisfactorily by PEM methods.The identification problembasedontheminimizationofapredictionerrorcriterion(oraleast-squares typecriterion),whichingeneralisacomplicatedfunctionofthesystemparameters, has to be solved by iterative descentmethodswhich may getstuck into local min- ima. Moreover, optimization methods need canonical parametrizations and it may be difficult to guess a suitable canonicalparametrizationfrom the outset. Since no single continuous parametrization covers all possible multivariable linear systems withafixedMcMillandegree,itmaybenecessarytochangeparametrizationinthe courseoftheoptimizationroutine.Thustheuseofoptimizationcriteriaandcanon- ical parametrizations can lead to local minima far from the true solution, and to viii Preface numericallyill-conditionedproblemsduetopooridentifiability,i.e.,tonearinsensi- tivity of the criterionto thevariationsof some parameters.Hence itseems thatthe PEMmethodhasinherentdifficultiesforMIMOsystems. On the other hand, stochastic realization theory, initiated by Faurre [46] and Akaike[1]andothers,hasbroughtinadifferentphilosophyofbuildingmodelsfrom data,whichisnotbasedonoptimizationconcepts.Akeystepinstochasticrealiza- tioniseithertoapplythedeterministicrealizationtheorytoacertainHankelmatrix constructedwithsampleestimatesoftheprocesscovariances,ortoapplythecanon- icalcorrelationanalysis(CCA)tothefutureandpastoftheobservedprocess.These algorithmshavebeenshowntobeimplementedveryefficientlyandinanumerically stablewaybyusingthetoolsofmodernnumericallinearalgebrasuchasthesingular valuedecomposition(SVD). Then,aneweffortindigitalsignalprocessingandsystemidentificationbasedon theQRdecompositionandtheSVDemergedinthemid-1980sandmanypapershave been published in the literature [100,101,118,119], etc. These realization theory- basedtechniqueshaveledtoadevelopmentofvariousso-calledsubspaceidentifica- tionmethods,including[163,164,169,171–173],etc.Moreover,VanOverscheeand DeMoor[165]havepublishedafirstcomprehensivebookonsubspaceidentification of linear systems. An advantageofsubspace methodsis thatwe do notneed(non- linear) optimization techniques, nor we need to impose to the system a canonical form,so thatsubspacemethodsdonotsufferfromtheinconveniencesencountered inapplyingPEMmethodstoMIMOsystemidentification. Though I have been interested in stochastic realization theory for many years, it was around1990that I actually resumed studies on realizationtheory,including subspace identification methods. However, realization results developed for deter- ministicsystemsontheonehand,andstochasticsystemsontheother,couldnotbe applied to the identification of dynamic systems in which both a deterministic test inputandastochasticdisturbanceareinvolved.Infact,thedeterministicrealization resultdoesnotconsideranynoise,andthestochasticrealizationtheorydevelopedup totheearly1990sdidaddressmodelingofstochasticprocesses,ortimeseries,only. Then,Inoticedatoncethatweneededanewrealizationtheorytounderstandmany existing subspace methodsand their underlyingrelations and to develop advanced algorithms. Thus I was fully convinced that a new stochastic realization theory in thepresenceofexogenousinputswasneededforfurtherdevelopmentsofsubspace systemidentificationtheoryandalgorithms. While we were attending the MTNS (The International Symposium on Math- ematical Theory of Networks and Systems) at Regensburg in 1993, I suggested to GiorgioPicci,UniversityofPadova,thatweshoulddojointworkonstochasticre- alizationtheoryinthepresenceofexogenousinputsandacollaborationbetweenus started in 1994 when he stayed at Kyoto University as a visiting professor.Also, I successivelyvisitedhimattheUniversityofPadovain1997.Thecollaborationhas resulted in several joint papers[87–90,93,130,131].Professor Picci has in partic- ular introduced the idea of decomposing the output process into deterministic and stochastic componentsby using a preliminaryorthogonaldecomposition,and then applyingtheexistingdeterministicandstochasticrealizationtechniquestoeachcom- Preface ix ponentto geta realizationtheoryin thepresenceofexogenousinput.On theother hand,inspiredbytheCCA-basedapproach,Ihavedevelopedamethodofsolvinga multi-stageWienerpredictionproblemtoderiveaninnovationrepresentationofthe stationaryprocesswithanobservableexogenousinput,fromwhichsubspaceidenti- ficationmethodsaresuccessfullyobtained. This book is an outgrowthof the joint work with Professor Picci on stochastic realizationtheoryandsubspaceidentification.Itprovidesanin-depthintroductionto subspacemethodsforsystemidentificationofdiscrete-timelinearsystems,together with ourresultsonrealizationtheoryin thepresenceofexogenousinputsandsub- spacesystemidentificationmethods.Ihaveincludedproofsoftheoremsandlemmas asmuchaspossible,aswellassolutionstoproblems,inordertofacilitatethebasic understanding of the material by the readers and to minimize the effort needed to consultmanyreferences. Thistextbookisdividedintothreeparts:PartIincludesreviewsofbasicresults, fromnumericallinearalgebratoKalmanfiltering,tobeusedthroughoutthisbook, Part II provides deterministic and stochastic realization theories developed by Ho andKalman,Faurre,andAkaike,andPartIIIdiscussesstochasticrealizationresults in the presenceof exogenousinputsand their adaptationto subspace identification methods;seeSection1.6formoredetails.Thus,variouspeoplecanreadthisbookac- cordingtotheirneeds.Forexample,peoplewithagoodknowledgeoflinearsystem theory and Kalman filtering can begin with Part II. Also, people mainly interested inapplicationscanjustreadthealgorithmsofthevariousidentificationmethodsin Part III, occasionally returning to Part I and/or Part II when needed. I believe that this textbookshouldbe suitable foradvancedstudents, appliedscientists andengi- neerswhowanttoacquiresolidknowledgeandalgorithmsofsubspaceidentification methods. IwouldliketoexpressmysincerethankstoGiorgioPicciwhohasgreatlycon- tributed to our fruitful collaboration on stochastic realization theory and subspace identificationmethodsoverthelasttenyears.IamdeeplygratefultoHideakiSakai, who has read the whole manuscriptcarefullyand providedinvaluablesuggestions, whichhaveledtomanychangesinthemanuscript.IamalsogratefultoKiyotsugu Takaba and Hideyuki Tanaka for their useful comments on the manuscript. I have benefited from joint works with Takahira Ohki, Toshiaki Itoh, Morimasa Ogawa, andHajimeAse,whotoldmeaboutmanyproblemsregardingmodelingandidenti- ficationofindustrialprocesses. Therelatedresearchfrom1996through2004hasbeensponsoredbytheGrant- in-AidforScientificResearch,theJapanSocietyofPromotionofSciences,whichis gratefullyacknowledged. TohruKatayama Kyoto,Japan January2005 Contents 1 Introduction................................................... 1 1.1 SystemIdentification ....................................... 1 1.2 ClassicalIdentificationMethods .............................. 4 1.3 PredictionErrorMethodforStateSpaceModels ................ 6 1.4 SubspaceMethodsofSystemIdentification..................... 8 1.5 HistoricalRemarks ......................................... 11 1.6 OutlineoftheBook......................................... 13 1.7 NotesandReferences ....................................... 14 PartI Preliminaries 2 LinearAlgebraandPreliminaries................................ 17 2.1 VectorsandMatrices........................................ 17 2.2 SubspacesandLinearIndependence........................... 19 2.3 NormsofVectorsandMatrices ............................... 21 2.4 QRDecomposition ......................................... 23 2.5 ProjectionsandOrthogonalProjections ........................ 27 2.6 SingularValueDecomposition................................ 30 2.7 Least-SquaresMethod ...................................... 33 2.8 RankofHankelMatrices .................................... 36 2.9 NotesandReferences ....................................... 38 2.10 Problems ................................................. 39 3 Discrete-TimeLinearSystems ................................... 41 3.1 -Transform............................................... 41 (cid:0) 3.2 Discrete-TimeLTISystems .................................. 44 3.3 NormsofSignalsandSystems ............................... 47 3.4 StateSpaceSystems ........................................ 48 3.5 LyapunovStability ......................................... 50 3.6 ReachabilityandObservability ............................... 51
The list of books you might like
