State Space Modeling of Time Series

323 Pages·1987·6.379 MB·English

by  Prof. Masanao Aoki (auth.)

Universitext
Masanao Aoki
State Space Modeling
of Time Series
With 76 Figures
Springer-Verlag Berlin Heidelberg New York
London Paris Tokyo
Prof. Masanao Aoki
Department of Computer Science
and Department of Economics
University of California
405 Hilgard Ave.
Los Angeles, CA 90024
USA
ISBN-I 3: 978-3-540-17257-4 e-ISBN-13: 978-3-642-96985-0
001: 10.1007/978-3-642-96985-0
This work is subject to copyright. All rights are reserved, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, re-use of illustrations,
recitation, broadcasting, reproduction on microfilms or in other ways, and storage in data
banks. Duplication of this publication or parts thereof is only permitted under the provi
sions of the German Copyright Law of September 9, 1965, in its version of June 24,1985,
and a copyright fee must always be paid. Violations fall under the prosecution act of the
German Copyright Law.
© Springer-Verlag Berlin Heidelberg 1987
2142/3140-543210
Prefuce _________________________________________ _
Many books are now available on data analysis. Some show how to fit data to
linear regression or other equations. Others discuss how to separate data into
various clusters. Such books are directed primarily at discovering static relations
among data.
Books on time series, i.e., those which explicitly deal with time-indexed
data, are also numerous, reflecting many advances made in the last four decades
or so. Probably the single most influential book is that by Box and Jenkins
[1970], which showed how to fit autoregressive moving average models to
(differenced) time series data. No book, however, incorporates the advances in
the construction and approximation of dynamic systems that have occurred in
system theory since the late 1970s. This seems to be particularly true for books
on time series of economic origin. Because of fundamental differences in the na
ture of data generation mechanisms, it is not surprising that litte communication
has taken place between econometricians and systems theorists - two important
groups of people engaged in time series modeling.
I believe that modeling of (economic) time series (some of the objectives of
which are mentioned a little later) can benefit from recent advances in system
theory. I have already compiled my lectures delivered at several universities as
Notes on Economic Time Series Analysis to facilitate communication between the
two groups, or more precisely, to bring some new methods for model construc
tion to the attention of econometricians and macroeconomists, (Aoki [1983]).
This book builds on Notes by expanding theoretical materials and improving the
exposition as well as adding more computational examples based on further
classroom testing of Notes.
Model building is the essence of time series analysis. Models are then used
(i) to describe the data behavior succinctly, (ii) to explain behavior of the time
series in terms of exogenous variables, (iii) to forecast, and even (iv) to control,
e.g., to synthesize macroeconomic policies under rational expectations. Thus,
one major objective of this book is to describe and evaluate several model con
struction methods, yielding markovian or state space models of time series not
easily found elsewhere. Two justifications may be offered for treating several
time series as a single vector-valued time series. First, such a treatment allows us
to understand the dynamic relations between the series. For example, leading in
dicators exploit dynamic relations that are assumed to exist either explicitly or
implicitly between various time series. Second, more accurate forecasting is
provided by using joint information contained in vector-valued time series.
A second objective is to assess computational, structural and other properties
of models thus constructed, and to compare them with those of more traditional
models. For example, our construction method leads to stable Markovian
models, while autoregressive models constructed using the traditional methods
may not.
How is a state space model constructed?, i.e., how is the dimension of the
state vector determined and the model parameters estimated? How good is the
VI Preface
model's predictive capability? These are some of the questions that need to be
answered in proposing any time series model construction method. This book
addresses these questions in Part II. Briefly, the covariance matrices between
past data and future realizations of time series are used to build a matrix called
the Hankel matrix. Information needed for constructing models is extracted from
the Hankel matrix. For example, its numerically determined rank will be the di
mension of the state model. Thus the model dimension is determined by the
data, after balancing several sources of error for such model construction. The
covariance matrix of the model forecasting error vector is determined by solving
a certain matrix Riccati equation. This matrix is also the covariance matrix of
the innovation process which drives the model in generating model forecasts. In
these model construction steps, a particular model representation, here referred to
as balanced, is used extensively. This mode of model representation facilitates
error analysis, such as assessing the error of using a lower dimensional model
than that indicated by the rank of the Hankel matrix.
The well-known Akaike's canonical correlation method for model construc
tion is similar to the one used in this book. There are some important differ
ences, however. Akaike uses the normalized Hankel matrix to extract canonical
vectors, while the method used in this book does not normalize the Hankel ma
trix. Consequently, Akaike's choice of state vector to summarize information
contained in the data also differs from the one in this book. What is more his
models are generally not balanced.
This book falls naturally into four parts: After a brief explanatory chapter,
Part I introduces, in three chapters, material preparatory to Part II, which is the
main body of this book. The notion of state is basic to our representation of time
series dynamics by Markovian models; hence it is introduced early in Chapter 2.
Chapter 3 deals with preparing raw data for analysis as explained in this book,
and time domain representation of time series is summarized. Theoretical
equivalence of this newer mode of representation of time series by state space or
Markovian models to the more traditional one is established in Chapter 4.
Part II describes innovation models in balanced representation, describes a
method for model construction, and presents some results on sensitivity and error
analysis in Chapters 5 through 10. Part III, consisting of a single chapter,
comprises several numerical examples based on macroeconomic time series.
Other, smaller examples are found throughout the book. Part IV comprises some
background material in appendices, and discusses time series models generated
by intertemporal optimization and by rational expectations. Even though Appen
dices 17 and 18 of this part do not directly deal with model constructions from
data, models of these two appendices are of interest to macroeconomists.
Chapter 5 is one of the key chapters of the book because it develops proper
ties of Hankel matrices and balanced innovation models, and for easy reference
collects in one place many of the theoretical constructs used in this book.
Chapter 6 discusses how to predict time series when their models have a special
structure. The important notion of innovation is introduced, and a method for
calculating orthogonal projections to produce innovation sequences is explained.
Preface VII
The most general way to generate innovation sequences is provided by Kalman
filters as discussed in Chapter 7. Kalman filters can even deal with non stationary
time series which traditional methods have difficulty in handling.
Chapters 8 through 10 embody the new contributions of this book. Chapter 8
considers several alternative ways of introducing state vectors to provide approxi
mate summaries of information contained in the data vectors. For example,
Akaike's canonical correlation method is shown to lead to a slightly different
state vector specification from ours. These alternative methods for introducing
state vectors are shown to be optimal with respect to different optimality criteria.
Chapter 9 describes how to compute the system matrices of the innovation
models and the innovation vector covariance matrices from the Hankel matrices.
Structural sensitivity and error analysis of approximate models are conducted in
Chapter 10. Chapter 11 contains several numerical examples drawn from some
macroeconomic time series of OECD countries.
The research activities reported here have been partially supported by a grant
from the National Science Foundation, program for economics, for which the au
thor is very grateful. Many people helped me in preparing this book. I would
like to mention Dr. Genshiro Kitagawa, and Professor Hirotugu Akaike for mak
ing TIMSAC programs available to the author. Without the initial help of Profes
sor Arthur Havenner and Mr. Bjorn Ellertsson with computation, the book would
never have been started. Professor Havenner has also read most of the
manuscript and made many useful comments. Dr. Jorma Rissanen made avail
able to the author some references of which he was unaware. Mr. Brian Young
served as a capable research assistant for some of the computations reported in
this book, and proofread some of the manuscript.
Contents _____________________
1 Introduction ....................................................................... 1
2 The Notion of State ............................................................... 3
3 Representation of Time Series ................................................... 9
3.1 Time Domain Representation ............................................... 9
Preprocessing of Data ........................................................ 11
Parameter Shifts and Structural Changes ................................... 24
Time Invariant Linear Models .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.2 Frequency Domain Representation ......................................... 27
4 State Space and ARMA Representation ......................................... 30
4.1 State Space Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.2 Unit Roots .................................................................... 32
4.3 Conversion to State Space Representation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Scalar Models ................................................................. 41
Observability Canonical Form. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 48
Conversion of State-Space Models into ARMA Models ................... 51
Vector Models................................................................ 53
5 Properties of State Space Models ...... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5.1 Observability ................................................................. 59
Observability and Consistency of Least Squares Estimates . . . . . . . . . . . . . . . . 60
Grammian .................................................................... 61
5.2 Covariance and Impulse Response Matrices ............................... 62
5.3 The Hankel Matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Covariance and Hankel Matrices ............................................ 64
5.4 System Parameters and Innovation Models ................................ 67
Spectral Factorization ........................................................ 68
5.5 Singular Value Decomposition .............................................. 70
Sensitivity of Singular Values................................................ 71
Rank and Singular Values ................................................... 72
Approximate Regression Analysis.......................................... 74
5.6 Balanced Realization of State Space Model................................. 75
Properties of Balanced Models .............................................. 76
Stability ....................................................................... 76
Parametrization ........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Analytical Example ........................................................... 79
5.7 Hankel Norm ofa Transfer Function ................. ......... ............. 81
5.8 Singular Value Decomposition in the z-Domain ........................... 82
x
Contents
6 Innovation Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.1 Cholesky Decomposition and Innovations ................................. 85
6.2 Orthogonal Projections. ............. . . ....... . ..... . . . . ................. ..... 87
7 Kalman Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.1 Innovation Models ........................................................... 93
7.2 Kalman Filters................................................................ 96
7.3 Causal Invertibility and Innovation ......................................... 100
Innovation Models ........................................................... 101
7.4 Likelihood Functions and Identification .................................... 102
7.5 A Non-Iterative Algorithm for Riccati Equations .......................... 103
7.6 Forecasting Equations ....................................................... 107
Out-of-Sample Forecasts ..................................................... 107
ARMA Forecasting Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 107
8 State Vectors and Optimality Measures .......................................... 111
8.1 State Vectors.. ..... ..... .................. . .................. .. ............... 111
8.2 Optimality Measures ......................................................... 116
Mutual Information .......................................................... 117
Prediction Error .............................................................. 117
Hankel Norm ................................................................. 118
9 Compution of System Matrices .................................................. 119
9.1 System Matrices .............................................................. 119
9.2 Balanced Models for Scalar Time Series .................................... 123
Examples ..................................................................... 123
9.3 Prediction Error Analysis .................................................... 127
9.4 Non-Stationary Models ...................................................... 127
9.5 Rescaling and Other Transformation of Variables .......................... 128
9.6 Dynamic MUltipliers ......................................................... 132
9.7 Numerical Examples...... . .................................................. 134
10 Approximate Models and Error Analysis ...................................... 149
10.1 Structural Sensitivity ...................................................... 149
Bivariate Series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 149
Adding Another Time Series ............................................. 157
10.2 Error Norms ............................................................... 161
10.3 Error Propagation .......................................................... 168
10.4 Some Statistical Aspects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 172
II Numerical Examples ............................................................ 177
11.1 Chemical Process Yields .................................................. 177
11.2 IBM Stock Prices .......................................................... 179
Contents XI
11.3 Canadian GNP and Money Data ......................................... 179
11.4 Gennany ................................................................... 182
11.5 United Kingdom ........................................................... 190
11.6 Combined Models for the United Kingdom and Gennany ............. 202
11.7 Japan ....................................................................... 204
11.8 Japan-US Interactions .................................................... 214
11.9 The United States of America ............................................ 216
11.10 Comparison with VAR Models ........................................... 216
Appendices ........................................................................... 229
Al Differences Equations ........................................................ 230
First Order Stable Equations ................................................. 230
First Order Unstable Equations .............................................. 233
Second Order Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 234
State Space Method ........................................................... 236
A2 Geometry of Weakly Stationary Stochastic Sequences ..................... 238
A.3 The z-Transfonn .............................................................. 241
A4 Discrete and Continuous Time System Correspondences .................. 247
A5 Calculation of the Inverse .................................................... 250
A6 Some Useful Relations for Matrix Quadratic Fonns ........................ 251
A 7 Spectral Decomposition Representation ..................................... 253
A8 Computation of Sample Covariance Matrices ............................... 254
A9 Vector Autoregressive Models ................................................ 256
AlO Properties of Symplectic Matrices ............................................ 258
All Common Factors in ARMA Models ......................................... 263
A12 Singular Value Decomposition Theorem .................................... 265
A13 Hankel Matrices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 267
A14 Spectrum and Factorization .................................................. 270
A15 Intertemporal Optimization by Dynamic Programming .................... 277
A16 Solution of Scalar Riccati Equations ......................................... 282
A17 Time Series from Intertemporal Optimization .............................. 283
A18 Time Series from Rational Expectations Models ............................ 291
A.19 Data Sources .................................................................. 299
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 301
Subj ect Index ........................................................................ 311
1 Introduction ___________________
Study of time series has a history much older than modern system theory. Proba
bility theorists, statisticians and economitricians have all contributed to our
understanding of time series over the past several decades, as is evidenced by
many well established books. One may wonder what system theory can add to
this well established field and doubt if any new perspective or insight can be
gained by this relative newcomer to the field. The history of science shows us,
however, that a given problem can fruitfully be examined by different disciplines,
partly because when it is viewed from new perspectives, implications of alterna
tive assumptions are explored by researchers with different backgrounds or
interests, and partly because new techniques developed elsewhere are imported to
explore areas left untouched by the discipline in which the problem originated.
Although a latecomer to the field of time series analysis, system theory has intro
duced a set of viewpoints, concepts and tools sufficiently different from the tradi
tional ones, and these have proved effective in dealing with vector-valued time
indexed data.
I believe that the new modeling procedure and insights thus obtained into
vector-valued time series are of potential interest to econometricians, economists,
statisticians, and to researchers in biology, medicine and social sciences in gen
eral. I hope that this book will help them overcome the language barriers and
make these results and new tools accessible to all those interested in time series,
so that they may benefit from system theory in their own research activities.
What are the new perspectives and results referred to? First, time series are
represented differently. Loosely put, traditional time series analysis is primarily
directed toward scalar-valued data, and usually represents time series or their
differenced version by (scalar) autoregressive, moving average or
autoregressive-moving average models. Modem control and system theory treat
several variables simultaneously as vector-valued variables, and have introduced
an alternative mode of representing dynamic phenomena, called state space or
Markovian representation, by defining internal or state space variables as useful
auxiliary variables. This book uses this alternative mode of representing vector
valued time-indexed data which directly connects exogenous variables at several
time points with endogenous variables also at serveral time points. As shown in
this book, these two ways of representing dynamic phenomena are theoretically
equivalent. However, their numerical and statistical properties are different.
State space representation is decidedly superior in several ways, as is detailed in
this book. At any rate, having two alternative ways of dealing with vector-valued
time series is definitely worthwhile.
Prompted by this different view of dynamic phenomena, or different means
of describing such phenomena, and by the necessity of paying greater attention to
the interrelation of vector components, system theory has introduced theoretical
notions that are nontrivial only for vector-valued time series, such as reachability,
observability, and miniaml realization which are not found in the traditional, i.e.,
scalar-value oriented time series literature. These notions turn out to be essential