Errors-in-variables methods in system identification

495 Pages·2018·3.104 MB·English

by  Soderstrom T.

Communications and Control Engineering
Torsten Söderström
Errors-in-Variables
Methods in System
Identification
Communications and Control Engineering
Series editors
Alberto Isidori, Roma, Italy
Jan H. van Schuppen, Amsterdam, The Netherlands
Eduardo D. Sontag, Boston, USA
Miroslav Krstic, La Jolla, USA
Communications and Control Engineering is a high-level academic monograph
series publishing research in control and systems theory, control engineering and
communications. It has worldwide distribution to engineers, researchers, educators
(severalofthetitlesinthisseriesfinduseasadvancedtextbooksalthoughthatisnot
their primary purpose), and libraries.
Theseriesreflects themajortechnological andmathematical advances that have
a great impact in the fields of communication and control. The range of areas to
which control and systems theory is applied is broadening rapidly with particular
growth being noticeable in the fields of finance and biologically-inspired control.
Books in this series generally pull together many related research threads in more
matureareasofthesubjectthanthehighly-specialisedvolumesofLectureNotesin
ControlandInformationSciences.Thisseries’smathematicalandcontrol-theoretic
emphasis is complemented by Advances in Industrial Control which provides a
much more applied, engineering-oriented outlook.
Publishing Ethics: Researchers should conduct their research from research
proposaltopublication inlinewithbestpracticesandcodesofconductofrelevant
professional bodies and/or national and international regulatory bodies. For more
details on individual ethics matters please see:
https://www.springer.com/gp/authors-editors/journal-author/journal-author-
helpdesk/publishing-ethics/14214
More information about this series at http://www.springer.com/series/61
ö ö
Torsten S derstr m
Errors-in-Variables Methods
fi
in System Identi cation
123
TorstenSöderström
Division of Systems andControl,
Department of Information Technology
Uppsala University
Uppsala
Sweden
ISSN 0178-5354 ISSN 2197-7119 (electronic)
Communications andControl Engineering
ISBN978-3-319-75000-2 ISBN978-3-319-75001-9 (eBook)
https://doi.org/10.1007/978-3-319-75001-9
LibraryofCongressControlNumber:2018930366
MathematicsSubjectClassification(2010): 62F10,93E12
©SpringerInternationalPublishingAG,partofSpringerNature2018
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
Printedonacid-freepaper
ThisSpringerimprintispublishedbytheregisteredcompanySpringerInternationalPublishing
AGpartofSpringerNature
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
To Andreas, Christer, David, Elisabet, Frida,
Gunnar, Gustav, Hjalmar, Johanna, Klara,
Marianne and Olof
Preface
This book is intended to give a comprehensive overview of errors-in-variables
(EIV) problems in system identification. This problem is about modeling of
dynamic systems when all measured variables and signals are noise-corrupted.
Anumberofdifferentapproachesaredescribed andanalyzed.The area has beena
central one in my own research for a long time, and this experience has influence
my own way of thinking of how to describe and categorize the many proposed
methodsavailableintheliterature.Theareacontinuestobeactivetoday,andthere
isasteadyinflowofarticlesonEIVfordynamicsystems,toleadingconferencesas
well as to journals.
As a proper background the reader is expected to have at least elementary
knowledge of system identification. The textbooks (1999) and Söderström and
Stoica(1989)canberecommended.Theycovermuchmorethanwhatisrequiredin
this context.
This book starts with giving a background for the errors-in-variables
(EIV) problem. First static systems are treated in some detail. The dominating
partofthisbookcopeswithdynamicsystems.TheEIVproblemassuchiscarefully
analyzed, and it is demonstrated that some additional assumption(s) must be
imposed if a unique solution is to be found. Several approaches and EIV methods
are presented and analyzed. This book ends with a chapter on users’ perspectives
for applying EIV methods in practice. See also Sect. 1.2 for a more detailed
description of this book.
It is a pleasure to express my gratitude to many colleagues with whom I over
theyearshavediscussed,learnedfrom,andpublishedworktogetherwithonerrors-in-
variables problems. These colleagues include Juan Carlos Agüero, Brian D. O.
Anderson,TheodoreAnderson,KeithBurnham,MatsCedervall,Han-FuChen,Bart
DeMoor,ManfredDeistler,RobertoDiversi,MatsEkman,HuguesGarnier,Marion
Gilson, Tryphon Georgiou, Graham Goodwin, Roberto Guidorzi, Christiaan Heij,
HåkanHjalmarsson,MeiHongBjerstedt,AlirezaKarimi,ErlendurKarlsson,David
Kreiberg, Alexander Kukush, Tomas Larkowski, Erik K. Larsson, Jens Linden,
KaushikMahata,IvanMarkovsky,MagnusMossberg,RikPintelon,AgnesRensfelt,
CristianRojas,WolfgangScherrer,JohanSchoukens,VirginijaŠimonyte_,Joachim
vii
viii Preface
Sorelius,UmbertoSoverini,PetreStoica,StephaneThil,KlaskevanHeusden,Sabine
VanHuffel,KiyoshiWada,FanYangWallentin,andWeiXingZheng.
I am truly indebted to many colleagues who have read earlier versions of the
book, in whole or in part, and pointed out a large number of errors and unclear
points, and also generously given me proposals for additional ideas. For example,
thelastchapterofthebookwaswrittenandincludedbased onsuchfeedback.The
reviewers who provided this indispensable help are Juan Carlos Agüero, Roberto
Diversi,HåkanHjalmarsson,DavidKreiberg,IvanMarkovsky,MagnusMossberg,
Giorgio Picci, Rik Pintelon, Johan Schoukens, and Umberto Soverini.
I would also like to thank the personnel at Springer (Oliver Jackson,
Meertinus Faber, Geethajayalaxmi Govindarjan, Komala Jaishankar, Ravikrishnan
Karunanandam, and Balaganesh Sukumar) for a smooth cooperation in producing
the book.
It happens sometimes that I have found manuscripts with ‘error-in-variables’,
ratherthan‘errors-in-variables’inthetitle(yes,ithashappenedalsoinmyowndraft
papers!).Asisexplainedinthebook,forEIVsystemsitisindeedakeyaspectthat
there are errors on both input and output measurements, and therefore one must
use plural! During the work with the book manuscript I have corrected quite a
numberoferrors,anditismysincerehopethatnottoomanyremain,eventhough
theremaystillbemorethanasingleone!DuringmyscientificcareerIhavemainly
been active in the control community, and therefore I believe in feedback. In
particular,Iwouldwelcomethereaders’commentsonthetextandpossiblypointing
out any remaining error. I can be reached on the e-mail address: torsten.
[email protected].
SomeyearsagoItoldmyfamilythatIwasplanningtowriteanotherbook.Some
suggestedthatthistimeIshouldwriteathriller.Aplotwaslaidoutaboutamurder
thatwasdetectedattheopeningceremonyofamajorcontrolconference.Iquickly
turned down this idea, that was not so serious anyway. It would demonstrate my
inabilitytowritesomethingexcitingfromafictionpointofview.Furthermore,the
theme does not match my general impressions from almost half a century with the
control community. I have mainly found it to be characterized by friendly and
helpfulpeople.Ihavededicatedthisbooktomyextendedfamily(grandchildrenare
included): Andreas,…, Olof. You are the Aand Otome!
Vattholma and Uppsala, Sweden Torsten Söderström
December 2017
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Four Motivating Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Outline of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3 Some Important Concepts in System Identification . . . . . . . . . 10
1.4 Some Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5 Extensions and Bibliographical Notes. . . . . . . . . . . . . . . . . . . 12
2 The Static Case. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1 Line Fitting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.1 Some System Theoretic Considerations
of Identifiability . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2 Confirmatory Factor Analysis . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.1 The Modeling Part . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.2 Estimation Part. . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3 The Frisch Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.4 Extensions and Bibliographical Notes. . . . . . . . . . . . . . . . . . . 30
2.A Further Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.A.1 Further Results for Line Fitting . . . . . . . . . . . . . . . . 35
2.A.2 Consistency of the CFA Estimate . . . . . . . . . . . . . . 46
3 The Errors-in-Variables Problem for Dynamic Systems . . . . . . . . . 49
3.1 The EIV Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2 About Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.3 Two Special Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.4 Some Naïve Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.4.1 Neglecting the Input Noise . . . . . . . . . . . . . . . . . . . 59
3.4.2 Estimating the Noise-Free Input Signal . . . . . . . . . . 62
3.4.3 Rewriting the Model into Standard Form . . . . . . . . . 66
3.5 Extensions and Bibliographical Notes. . . . . . . . . . . . . . . . . . . 68
ix
x Contents
4 Identifiability Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.1 Some General Aspects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 Identifiability Analysis for Parametric Models. . . . . . . . . . . . . 78
4.3 Identifiability When Using Multiple Experiments . . . . . . . . . . 82
4.4 Closed-Loop Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.5 Extensions and Bibliographical Notes. . . . . . . . . . . . . . . . . . . 87
5 Modeling Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.1 Problem Statement and Notations. . . . . . . . . . . . . . . . . . . . . . 89
5.2 Using Models with an Arbitrary Delay. . . . . . . . . . . . . . . . . . 92
5.3 Continuous-Time EIV Models and Conversion
to Discrete-Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.4 Modeling the Noise Properties. . . . . . . . . . . . . . . . . . . . . . . . 95
5.5 Frequency Domain Models . . . . . . . . . . . . . . . . . . . . . . . . . . 96
5.6 Modeling the Total System . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.7 Models for Multivariable Systems . . . . . . . . . . . . . . . . . . . . . 99
5.8 Classification of Estimators Based on Data Compression. . . . . 102
5.9 Model Order Determination. . . . . . . . . . . . . . . . . . . . . . . . . . 103
5.9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
5.9.2 Some Approaches. . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.9.3 About the Rank Tests . . . . . . . . . . . . . . . . . . . . . . . 107
5.9.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.10 Extensions and Bibliographical Notes. . . . . . . . . . . . . . . . . . . 110
5.A Further Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
5.A.1 Discrete-Time Model Approximation. . . . . . . . . . . . 112
5.A.2 Analyzing Effects of Small Singular Values . . . . . . . 118
6 Elementary Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
6.1 The Least Squares Method . . . . . . . . . . . . . . . . . . . . . . . . . . 121
6.2 The Instrumental Variable Method. . . . . . . . . . . . . . . . . . . . . 123
6.2.1 Description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.2.2 Consistency Analysis . . . . . . . . . . . . . . . . . . . . . . . 124
6.2.3 User Choices. Examples of Instrumental Vectors . . . 125
6.2.4 Instrumental Variable Methods Exploiting
Higher-Order Statistics . . . . . . . . . . . . . . . . . . . . . . 128
6.2.5 Other Instrumental Variable Techniques. . . . . . . . . . 130
6.3 Extensions and Bibliographical Notes. . . . . . . . . . . . . . . . . . . 133
7 Methods Based on Bias-Compensation . . . . . . . . . . . . . . . . . . . . . . 135
7.1 The Basic Idea of Bias-Compensation . . . . . . . . . . . . . . . . . . 135
7.2 The Bias-Eliminating Least Squares Method. . . . . . . . . . . . . . 138
7.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
7.2.2 White Output Noise . . . . . . . . . . . . . . . . . . . . . . . . 139
7.2.3 Correlated Output Noise . . . . . . . . . . . . . . . . . . . . . 142