Random Coefficient Autoregressive Models: An Introduction

159 Pages·1982·3.247 MB·English

by  Des F. Nicholls, Barry G. Quinn (auth.)

Lecture Notes in
Statistics
Edited by D. Brillingei', S. Fienberg, J. Gani,
J. Hartigan, and K. Krickeberg
11
Des F. Nicholls
Barry G. Quinn
Random Coefficient
Autoregressive Models:
An Introduction
Springer-Verlag
New York Heidelberg Berlin
Des F. Nicholls Barry G. Quinn
Reader in Statistics Lecturer in Statistics
Australian National University University of Wollongong
Canberra Wollongong
Australia Australia
AMS Classification: 62H99, 62102, 62J99, 62K99, 62L99
Library of Congress Cataloging in Publication Data
Nicholls, Des F.
Random coefficient autoregressive models.
(Lecture notes in statistics; v. 11)
Bibliography: p.
Includes index.
1. Regression analysis. 2. Random variables.
I. Quinn, Barry G. II. Title. III. Series: Lecture
notes in statistics (Springer-Verlag); v. Ii.
QA278.2.N5 1982 519.5'36 82-10619
With 11 Illustrations
© 1982 by Springer-Verlag New York Inc.
All rights reserved. No part of this book may be translated or reproduced in any form
without written permission from Springer-Verlag, 175 Fifth Avenue, New York, New
York 10010, U.S.A.
9 8 7 6 5 4 3 2 1
ISBN-13: 978-0-387-90766-6 e-ISBN-13: 978-1-4684-6273-9
DOl: 10.1007/978-1-4684-6273-9
iii
PREFACE
In this monograph we have considered a class of autoregressive
models whose coefficients are random. The models have special appeal
among the non-linear models so far considered in the statistical
literature, in that their analysis is quite tractable. It has been
possible to find conditions for stationarity and stability, to derive
estimates of the unknown parameters, to establish asymptotic properties
of these estimates and to obtain tests of certain hypotheses of
interest.
We are grateful to many colleagues in both Departments of
Statistics at the Australian National University and in the Department
of Mathematics at the University of Wo110ngong. Their constructive
criticism has aided in the presentation of this monograph. We would
also like to thank Dr M.A. Ward of the Department of Mathematics,
Australian National University whose program produced, after minor
modifications, the "three dimensional" graphs of the log-likelihood
functions which appear on pages 83-86.
Finally we would like to thank J. Radley, H. Patrikka and
D. Hewson for their contributions towards the typing of a difficult
manuscript.
IV
CONTENTS
CHAPTER 1 INTRODUCTION
1.1 Introduction 1
Appendix 1.1 11
Appendix 1.2 14
CHAPTER 2 STATIONARITY AND STABILITY 15
2.1 Introduction 15
2.2 Singly-Infinite Stationarity 16
2.3 Doubly-Infinite Stationarity 19
2.4 The Case of a Unit Eigenvalue 31
2.5 Stability of RCA Models 33
2.6 Strict Stationarity 37
Appendix 2.1 38
CHAPTER 3 LEAST SQUARES ESTIMATION OF SCALAR MODELS 40
3.1 Introduction 40
3.2 The Estimation Procedure 42
3.3 Strong Consistency and the Central
Limit Theorem 43
3.4 The Consistent Estimation of the Covariance
Matrix of the Estimates 51
Appendix 3.1 52
Appendix 3.2 57
CHAPTER 4 MAXIMUM LIKELIHOOD ESTIMATION OF SCALAR MODELS 59
4.1 Introduction 59
4.2 The Maximum Likelihood Procedure 60
4.3 The Strong Consistency of the Estimates 64
4.4 The Central Limit Theorem 70
4.5 Some Practical Aspects 73
Appendix 4.1 75
Appendix 4.2 77
CHAPTER 5 A MONTE CARLO STUDY 81
5.1 Simulation and Estimation Procedures 81
5.2 First and Second Order Random Coefficient
Autoregressions 88
5.3 Summary 97
v
Page
CHAPTER 6 TESTING THE RANDOMNESS OF THE COEFFICIENTS 98
6.1 Introduction 98
6.2 The Score Test 99
6.3 An Alternative Test 104
6.4 Power Comparisons 108
Appendix 6.1 III
CHAPTER 7 THE ESTIMATION OF MULTIVARIATE MODELS 124
7.1 Preliminary 124
7.2 The Least Squares Estimation Procedure 124
7.3 The Asymptotic Properties of the Estimates 127
7.4 Maximum Likelihood Estimation 132
7.5 Conclusion 135
Appendix 7.1 136
CHAPTER 8 AN APPLICATION 139
8.1 Introduction 139
8.2 A Non-Linear Model for the Lynx Data 140
REFERENCES 150
AUTHOR AND SUBJECT INDEX 153
CHAPTER 1
INTRODUCTION
1.1 Introduction
Until recently the models considered for time series have usually been
linear with constant coefficients. In most situations one would not expect
such models to be the "best" class of model to fit to a set of real data,
although one tacitly makes the assumption that the linear model under con
sideration is a close approximation to physical reality. A number of
factors have resulted in a consideration of different classes of non-linear
models, not the least of which is that the theory of linear models is
essentially complete. A large amount of the research into these models is
now being concentrated on the construction and application of computationally
efficient algorithms to determine order and obtain estimates of the
unknown parameters which have desirable statistical properties. The
increased' power and speed of modern computers has also had a
significant effect on the direction in which time series research has headed.
This is clearly demonstrated for example by the computational requirements
of Akaike's criterion (see Akaike (1978» to determine the order of a
particular linear time series model. With the increase in computer capa
bilities the application of such criteria has become routine.
The steadily increasing interest in various classes of non-linear time
series models is clearly demonstrated by the time series literature over the
past decade. Granger and Andersen (1978) have introduced the now familiar
class of bilinear models (see Robinson (1977) and Subba Rao (1981) also)
while random coefficient and time varying parameter models have received atten
tion in both the engineering and econometric literature. Indeed the Annals of
Economic and Social Measurement has allocated an entire issue (volume 2,
2
number 4, 1973) to the consideration of such models.
Subba Rao (1970) has discussed autoregressive models with time
dependent coefficients and has considered their weighted least squares
estimation at a particular instant of time. Tong (1978) and Tong and
Lim (1980) have considered threshold autoregressive models, which
approximate non-linear time series by means of different linear autoregressive
models fitted to subsets of the data, and have discussed the estimation
and application of these models to various data sets. Ozaki (1980) has
investigated the case of an autoregression for which the coefficients are
functions of time which decay exponentially, the exponential term having
least effect when a past value of the time series is large, and most effect
when the value is small (see Ozaki (1980) p.89-90). The models of Tong,
Lim and Ozaki were developed to explain the natural phenomenon known as
the limit cycle (see Tong and Lim (1980) p.248).
A class of non-linear models which includes the bilinear, threshold
autoregressive and exponential autoregressive models as special cases
has been discussed by Priestley (1980). He has described a recursive
algorithm for the estimation of these 'state-dependent' models and has
shown how such models may be used for forecasting.
Jones (1978) has investigated a first order non-linear autoregression
where an observation X(t) at time t is the sum of a fixed non-linear
function at time (t-l) and a disturbance term i.e.
X(t) = f{X(t-l)} + E(t) ,
where f(.) is the fixed function and {E(t), t = 0, ±l, ±2, ... } is
a sequence of identically and independently distributed random variables.
Jones has presented methods for approximating the stationary distributions
of such processes and derived expressions by which moments, joint moments
and densities of stationary processes can be obtained. His theoretical
results are illustrated by a number of simulations.
3
As yet there has been little statistical theory (properties of the
estimates, central limit theorems, tests of hypotheses etc.) developed
for the bilinear, the threshold autoregressive or the exponential damping
coefficient autoregressive models. On the other hand a substantial
amount of theory has been developed for certain classes of varying para-
meter models. Pagan (1980) gives an excellent bibliography of recent
contributors who have considered problems associated with these models. In
the case of varying parameter models there have as yet, however, been few
applications of the theory developed to real data. Kendall (1953) was
one of the first to attempt an empirical investigation of such models.
He considered a number of economic series and fitted second order auto
regressions, the coefficients of which were slowly changing through time
as the economy changed. In fact he chose his coefficients to follow
quadratic trends. It is enlightening to read this early work of Kendall
as it illustrates the point made earlier that developments in computer
technology have made it possible for researchers to examine problems which,
through computational difficulty, could not have been considered a few years
ago. The estimation and interpretation of the spectra of these autoregressive
models with time trending coefficients have been considered by Granger
and Hatanaka (i964, Chapter 9).
As Kendall (1953) has pointed out, when considering the modelling of
economic data, it seems reasonable to generalize the constant coefficient
model to one where the constants are themselves changing through time as
the economy changes. Kendall, Subba Rao and Jones have restricted their
attention to non-linear autoregressive models for which the coefficients,
while non-linear. are non-random, while Garbade (1977) has considered the
estimation of regression models where the coefficients are assumed to
follow a simple random walk. Garbade's approach requires the numerical
maximization of a concentrated likelihood function.
4
A natural.variation of these models is the random coefficient auto-
regressive (RCA) models. These models are in fact the class of model with
which we shall be concerned in this monograph. There has been some
investigation of these and closely related models in the economic literature.
Turnovsky (1968) has considered stochastic models where the errors are
mUltiplicative i.e. models of the form X(t) = (a+u(t»X(t-l), where a is
a constant and the u(t) are uncorrelated random variables with E{u(t)} = 0,
More recently, Ledolter (1980) has extended Garbade's (1977) procedure
to include autoregressive models, while Conlisk (1974), (1976) has derived
conditions for the stability of RCA models. Andel (1976) has argued that when
modelling time series data in such fields as hydrology, meteorology and
biology, the coefficients of the model under consideration arise "as a
result of complicated processes and actions which usually have many random
features". This has led him to consider scalar RCA models and to derive con-
ditions for their second order stationarity. In what follows, for certain
classes of RCA models, we shall develop a rigorous statistical theory along
the lines of that which exists for constant coefficient autoregressions.
A p-variate time series {X(t)} will be said to follow a random
coefficient autoregressive model of order n, i.e. RCA(n), if X(t)
satisfies an equation of the form
n
(1.1.1) X(t) E {Si + Bi(t)}X(t-i) + £(t).
i=l
For this model the following assumptions are made.
(i) {£(t); t = 0, ±l, ±2, .•. } is an independent sequence of
p-variate random variables with mean zero and covariance
matrix G.
5
(ii) The pxp matrices ~i' i = 1, •••· ,n are constants.
(iii) Letting B(t) = [Bn(t), ••• ,B1(t)], then {B(t); t = 0, ±1, ±2, •••• }
is an independent sequence of pxnp matrices with mean zero and
E[B(t) 8 B(t)] = C. {B(t)} is also independent of {E(t)}.
From (1.1.1) it can be seen that if the elements of C are small
compared with those of the matrices ~i' then realizations of {X(t)} would
be expected to resemble realizations of constant coefficient autoregressions.
If however it were possible for some Bi(t) to have elements which were
large compared with ~i' one might expect to see some large values of X(t)
over a long realization, especially if several elements of C were
relatively large. Such behaviour would generally be associated with non
stationarity, but may only be an indication of the non-linear nature of the
RCA model. The phenomenon is well illustrated in figures 1.1-1.4 where,
for samples of size two thousand and for various values of ~, C and G,
a number of scalar RCA(l) models have been simulated.
In chapter 2 we shall derive conditions for the second order
stationarity of models of the form (1.1.1) generalizing Andel's (1976) work,
which is concerned with a similar problem for scalar RCA models. The latter
part of chapter 2 considers conditions for stability and the relationship
between stability and stationarity. Chapter 3-4 will be concerned with the
estimation (both least squares and maximum likelihood) of scalar RCA models,
as well as a derivation of the asymptotic properties of the estimates.
Chapter 5 presents the results of a number of computer experiments (using
simulated data) which illustrate the theoretical procedures and results
developed in the previous two chapters. Chapter 6 examines the problem of
testing the randomness of the coefficients of the model (1.1.1), while
chapter 7 discusses the estimation of multivariate RCA models. The final
chapter considers an application of the theory developed to the well known
Canadian lynx series. An RCA(2) modeL h~ been fitted to the first 100