Robust and Nonlinear Time Series Analysis: Proceedings of a Workshop Organized by the Sonderforschungsbereich 123 “Stochastische Mathematische Modelle”, Heidelberg 1983

296 Pages·1984·11.6 MB·English

by  Hirotugu Akaike (auth.)

Lecture Notes in Statistics
Vol. 1: R. A. Fisher: An Appreciation. Edited by S. E. Fienberg and D. V. Hinkley. XI, 208 pages,
1980.
Vol. 2: Mathematical Statistics and Probability Theory. Proceedings 1978. Edited by W. Klonecki,
A. Kozek, and J. Rosinski. XXIV, 373 pages, 1980.
Vol. 3: B. D. Spencer, Benefit-Cost Analysis of Data Used to Allocate Funds. VIII, 296 pages, 1980.
Vol. 4: E. A. van Doorn, Stochastic Monotonicity and Queueing Applications of Birth-Death Proces
ses. VI, 118 pages, 1981.
Vol. 5: T. Rolski, Stationary Random Processes Associated with Point Processes. VI, 139 pages,
1981.
Vol. 6: S. S. Gupta and D.-y' Huang, Multiple Statistical Decision Theory: Recent Developments. VIII,
104 pages, 1981.
Vol. 7: M. Akahira and K. Takeuchi, Asymptotic Efficiency of Statistical Estimators. VIII, 242 pages,
1981.
Vol. 8: The First Pannonian Symposium on Mathematical Statistics. Edited by P. Revesz, L. Schmet
terer, and V. M. Zolotarev. VI, 308 pages, 1981.
Vol. 9: 8. J0rgensen, Statistical Properties of the Generalized Inverse Gaussian Distribution. VI, 188
pages, 1981.
Vol. 10: A. A. Mcintosh, Fitting Linear Models: An Application on Conjugate Gradient Algorithms. VI,
200 pages, 1982.
Vol. 11: D. F. Nicholls and B. G. Quinn, Random Coefficient Autoregressive Models: An Introduction.
V, 154 pages, 1982.
Vol. 12: M. Jacobsen, Statistical Analysis of Counting Processes. VII, 226 pages, 1982.
Vol. 13: J. Pfanzagl (with the assistance of W. Wefelmeyer), Contributions to a General Asymptotic
Statistical Theory. VII, 315 pages, 1982.
Vol. 14: GUM 82: Proceedings of the International Conference on Generalised Linear Models.
Edited by R. Gilchrist. V, 188 pages, 1982.
Vol. 15: K. R. W. Brewer and M. Hanif, Sampling with Unequal Probabilities. IX, 164 pages, 1983.
Vol. 16: Specifying Statistical Models: From Parametric to Non-Parametric, Using Bayesian or Non
Bayesian Approaches. Edited by J. P. Florens, M. Mouchart, J. P. Raoult, L. Simar, and A. F. M.
Smith. XI, 204 pages, 1983.
Vol. 17: I. V. Basawa and D. J. Scott, Asymptotic Optimal Inference for Non-Ergodic Models. IX,
170 pages, 1983.
Vol. 18: W. Britton, Conjugate Duality and the Exponential Fourier Spectrum. V, 226 pages, 1983.
Vol. 19: L. Fernholz, von Mises Calculus For Statistical Functionals. VIII, 124 pages, 1983.
Vol. 20: Mathematical Learning Models - Theory and Algorithms: Proceedings of a Conference.
Edited by U. Herkenrath, D. Kalin, W. Vogel. XIV, 226 pages, 1983.
Vol. 21: H. Tong, Threshold Models in Non-linear Time Series Analysis. X, 323 pages, 1983.
Vol. 22: S. Johansen, Functional Relations, Random Coefficients and Nonlinear Regression with
Application to Kinetic Data. VIII, 126 pages. 1984.
Vol. 23: D. G. Saphire, Estimation of Victimization Prevalence Using Data from the National Crime
Survey. V, 165 pages. 1984.
Vol. 24: T. S. Rao, M. M. Gabr, An Introduction to Bispectral Analysis and Bilinear Time Series
Models. VIII, 280 pages, 1984.
Vol. 25: Time Series Analysis of Irregularly Observed Data. Proceedings, 1983. Edited by E. Parzen.
VII, 363 pages, 1984.
Lectu re Notes in
Statistics
Edited by D. Bri"inger, S. Fienberg, J. Gani,
J. Hartigan, and K. Krickeberg
26
Robust and Nonlinear
Time Series Analysis
Proceedings of a Workshop Organized by the
Sonderforschungsbereich 123 "Stochastische
Mathematische Modelle", Heidelberg 1983
Edited by J. Franke, W Hardie and D. Martin
Springer-Verlag
New York Berlin Heidelberg Tokyo 1984
Editors
JOrgen Franke
Wolfgang Hardie
Fachbereich Mathematik, Univetsitat Frankfurt
Robert Mayer Str. 6-10, 6000 Frankfurt, FRG
Douglas Martin
Department of Statistics, GN-22, University of Washington
Seattle, WA 98195, USA
ISBN-13: 978-0-387-96102-6 e-ISBN-13: 978-1-4615-7821-5
DOl: 10.1007/978-1-4615-7821-5
Library of Congress Cataloging in Publication Data. Main entry under title: Robust and nonlinear
time series analysis. (Lecture notes in statistics; 26) Bibliography: p. 1. Time-series analysis
Congresses. 2. Robust statistics-Congresses. I. Franke, J. (JOrgen) II. Hardie, Wolfgang.
III. Martin, D. (Douglas) IV. Sonderforschungsbereich 123-"Stochastische Mathematische
Modelle." V. Series: Lecture notes in statistics (Springer-Verlag); v. 26.
0A280.R638 1984 519.5'5 84-23619
This work is subject to copyright. All rights are reserved, whether the whole or part of the material
is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting,
reproduction by photocopying machine or similar means, and storage in data banks. Under
§ 54 of the German Copyright Law where copies are made for other than private use, a fee is
payable to "Verwertungsgesellschaft Wort", Munich.
© by Springer-Verlag Berlin Heidelberg 1984
2146/3140-543210
PREFACE
Classical time series methods are based on the assumption that a
particular stochastic process model generates the observed data. The,
most commonly used assumption is that the data is a realization of a
stationary Gaussian process. However, since the Gaussian assumption is
a fairly stringent one, this assumption is frequently replaced by the
weaker assumption that the process is wide~sense stationary and that
only the mean and covariance sequence is specified. This approach of
specifying the probabilistic behavior only up to "second order" has
of course been extremely popular from a theoretical point of view be
cause it has allowed one to treat a large variety of problems, such
as prediction, filtering and smoothing, using the geometry of Hilbert
spaces.
While the literature abounds with a variety of optimal estimation
results based on either the Gaussian assumption or the specification of
second-order properties, time series workers have not always believed
in the literal truth of either the Gaussian or second-order specifica
tion. They have none-the-less stressed the importance of such optimali
ty results, probably for two main reasons: First, the results come
from a rich and very workable theory. Second, the researchers often
relied on a vague belief in a kind of continuity principle according
to which the results of time series inference would change only a small
amount if the actual model deviated only a small amount from the assum
ed model. Somehow second-order theory was held in particular esteem
because in such theory one does not assume a Gaussian distribution.
Unfortunately blind faith in the existence of such a continuity
principle turns out to be totally unfounded. This fact has been made
clear with regard to the behavior of maximum-likelihood estimates based
on the Gaussian assumption in the classical setting of independent ob
servations. The seminal work of Hampel, Huber and Tukey formed the
foundation for over a decade of work on robust methods for the indepen
dent observations setting. These methods have been designed to perform
well in a neighborhood of a nominal parametric model. The most frequent
ly treated case is where the nominal model is Gaussian and the neighbor
hood contains outlier-producing heavy-tailed distributions.
"Doing well" in a neighborhood can be described in terms of three
distinct concepts of robustness, namely: effiaienay robustness, min
max robustness, and qualitative robustness. These are in historical
order of inception. Qualitative robustness, introduced by Hampel (1971) ,
is a fundamental continuity condition, which is in fact a basic prin
ciple of statistics. Efficiency robustness is a simple and accessible
concept, whose importance was made quite clear by J.W. Tukey (1960).
An estimate is efficieny robust if it has high efficiency both at the
nominal model and at a strategically-selected finite set of "nearby"
distributions. Min-ma~ robustness was introduced by Huber (1964) in the
problem of estimating location, with asymptotic variance as the loss
function. In general, a min-max robust estimate is one for which the
maximum mean-squared error of an estimate, either for a finite sample
size or asymptotically, is minimized over a neighborhood of the nominal
model. For further details on robustness see Huber (1981).
In general an estimate which has all three robustness properties
is preferred. For some problems, typically in the asymptotic framework
such estimates exist. However, this is not always possible, and often
one will have to settle for qualitative robustness (or its data-orient
ed analogue, resistanae, a term coined by Tukey - see Mosteller and
Tukey, 1977), along with some sort of a verification of efficiency
robustness. The latter is obviously required since some rather ridicu
lous estimates, e.g., identically constant estimates, are qualitatively
robust.
It is only in relatively recent years that attention has turned
to robustness in the context of the time-series setting. As in the in
dependent observations setting, classical time-series procedures based
on normality assumptions lack robustness, and the same is true of most
optimal linear procedures for time series based on second-order speci
fications. For example, the least-squares estimates of autoregressive
parameters are quite non-robust, as are smoothed periodogram spectrum
estimates.
In the time-series setting it is important to realize that there
are a variety of outlier types which can occur and which have different
consequences. Two particular broad types of outliers stand out:
(i) innovation outZiers. which are associated with heavy-tailed distri
butions for the innovations in perfectly-observed linear processes, and
(ii) additive outZiers. See, for example, Fox (1972) and Martin (1981).
Additive outliers, or more general forms of replacement type contami
nation (see Martin and Yohai, 1984 b), cause by far the most serious
problems for the classical time-series estimates. A striking
example of how extremely small outliers can spoil spectrum esti-
mates is provided by Kleiner, Martin and Thomson (1979), who show how
to obtain robust spectrum estimates via robust prewhitening techniques.
v
Since qualitative robustness is a fundamental concept, it should
be noted that Hampel's definition of qualitative robustness does not
really cover the time-series setting, where a variety of definitions
are possible. Fortunately, this issue has recently been dealt with by
a number of authors: Papantoni-Kazakos and Gray (1979), Cox (1981),
Bustos (1981), Boente, Fraiman and Yohai (1982), and by Papantoni
Kazakos in this volume. Boente, Fraiman and Yohai (1982) not only pro
vide results which relate. the various definitions of qualitative ro
bustness, they also provide an intuitively appealing new definition
which is based on Tukey's data-oriented notion of resistance.
In addition to the issue of qualitative robustness for time series,
a variety of other problems concerning robust estimation for time series
have begun to receive attention. For a recent survey of results on
robustness in time series and robust estimates of ARMA models, see Mar
tin and Yohai (1984 a).
Heavy-tailed distributions, and other diverse mechanisms for model
failure (e.g., local nonstationarity, interventions - see Box and Tiao,
1975), which give rise to outliers are not the only kind of deviation
from a nominal model having a severe adverse impact on time-series esti
mates. DeViations from an assumed second-order specification, either in
terms of covariance sequence or spectrum, can also cause problems. This
issue is important, for example, when constructing linear filters,
smoothers and predictors. Some details may be found in the article in
this volume by Franke and poor, and in the many references therein.
With regard to deviations from an assumed second-order model, a
particularly potent kind of second-order deviation is that where one
assumes exponentially decaying correlations (e.g., as in the case of
stationary ARMA processes), and in fact the correlations decay at a
slower rate.Here we have unsuspected long-tailed 'correlations as a
natural counterpart of long-tailed distribution functions. In this re
gard, the article by Graf and Hampel in this volume is highly pertinent.
In both the time series and the non-time series settings non
parametric procedures fODn a natural complement to robust procedures.
One reason for this, in the nonparametric regression setting with inde
pendent observations for example, is the following. Many data sets have
a "practical" identification problem: there are a few unusual data points
which on the one hand may be regarded as outliers, and on the other hand
may be due to nonlinearity. The data mayor may not be sufficient to
say which is the case. We shall of~en wish to use both nonparametric
and robust regression. Then if the sample size is large enough, we
VI
may be able to decide whether or not nonlinearity is present, and
whether or not a few data points are outliers.
In the time-series setting the situation is similar. It is known
that many times series arising in practice exhibit outliers, either in
isolation or in patches, and that certain nonlinear models generate
sample paths which manifest patches of outliers (Subba Rao, 1979; Nemec;
1984). Thus, we will need to have both good robust procedures for esti
mating linear time-series models, and good nonparametric procedures
for estimating nonlinear time-series models (see Watson, 1964, for ear
ly work in this area). And of course, we may need to combine the two
approaches, as does Robinson in his article in this volume, and Collomb
and H~rdle, 1984).
It may also be noted that robust estimates which guard against out
liers are always nonlinear, and that many Gaussian maximum-likelihood
estimates of time-series models are also nonlinear. Thus nonlinearity
is also a common theme which bridges both robust and nonparametric
approaches to modelling and analysis of time series.
In order to provide a forum for discussions and exchange of ideas
for researchers working on robust and nonlinear methods in time-series
analysis, or on related topics in robust regression, a workshop on
"Robust and Nonlinear Methods in Time Series Analysis" took place at
the University of Heidelberg in September 1983, organized under the
auspices of the SFB 123. This volume contains refereed papers, most of
which have been presented at the workshop. A few particularly appropri
ate contributions were specifically invited from authors who could
not attend the meeting.
We take the opportunity to thank Dr. H. Dinges and the speaker of
the SFB 123, Dr. W. Jager, for encouraging and supporting this workshop.
The meeting could not have taken place without the generous support of
the Deutsche Forschungsgemeinschaft. Finally, we would like to thank
all participants for their enthusiastic cooperation, which made the
workshop a lively and successful meeting.
The EcU;toM
References
Boente, G., Fraiman, R. and Yohai, V.J. (1982). ~Qualitative robustness
for general stochastic processes," Technical Report No. 26, Department
of Statistics, University of Washington, Seattle, WA.
Box, G.E.P. and Tiao, G.C. (1975). "Intervention analysis with applica
tions to economic and environmental problems," J. Amer. Stat. Assoc.
70, 70-79.
VII
Bustos, O.H. (1981). ~Qualitative robustness for general processes",
Informes de Matematica, Serie B-002/81, Instituto de Matematica .Pura
e Aplicada, Brazil.
Collomb, G. and H1irdle, W. (1984). "Strong uniform convergence rates in
robust nonparametric time series analysis: kernel regression estimation
from dependent observations", submitted to Annals of Stat.
cox, D. (1981). "Metrics on stochastic processes and qualitative robust
ness", Technical Report No.3, Department of Statistics, University of
Washington, Seattle, WA.
Hampel, F.R. (1971). ~A general qualitative definition of robustness",
Ann. Math. Stat. 42, 1887-1896.
Huber, P.J. (1964). "Robust estimation of a location parameter", Ann.
Math. Stat. 35, 73-101.
Huber, P.J. (1981). Robust Statistics. Wiley, New York, NY.
Kleiner, B., Martin, R.D. and Thompson, D.J. (1979). "Robust estimation
of power spectra", J. Royal Stat. Soc. B41, 313-351.
Martin, R.D. (1981). "Robust Methods for time series", in Applied Time
Series II, D.F. Findley, ed. Academic Press, New York, NY.
Martin, R.D. and Yohai, V.J. (1984 a). "Robustness for Time Series and
Estimating ARMA models", in Handbook of Statistics, Vol. 4, edited by
Brillinger and Krishnaiah, Academic Press, New York, NY.
Martin, R.D. and Yohai, V.J. (1984 b). "Influence function for time
series", Technical Repo~:No. 51, D?partment of Statistics, Uni versi ty of
Washington, Seattle, WA.
Mosteller, F. and Tukey, J.W. (1977). Data Analysis and Regression,
Addison-Wesley, Reading, MA.
Nemec, A.F.L. (1984). "Conditionally heteroscedastic autoregressions",
Technical Report No. 43, Department of Statistics, University of Washing
ton, Seattle, WA.
Papantoni-Kazakos, P. and Gray, R.M. (1979). "Robustness of estimators
on stationary observations", Ann. Probab. 7, 989-1002.
Subba Rao, T. (1979). Discussion of "Robust estimation of power spectra"
by Kleiner, Martin and Thomson, J. Royal Stat. Soc., B, 41, 346-347.
Tukey, J.W. (1960). "A survey of sampling from contaminated distributions"
in Contributions in Probability and Statistics, I. Olkin, ed., Stanford
University Press, Stanford, CA.
Watson, G.S. (1964). "Smooth regression analysis", Sankhya, 26, A,
359-372 •
CONTENTS
H. AKAIKE: On the Use of Bayesian Models in Time 1
Series Analysis •
R.J. BHANSALI: Order Determination for Processes 17
with Infinite Variance .
O. BUSTOS, R. FRAIMAN, V.J. YOHAI: Asymptotic Be- 26
haviour of the Estimates Based on Residual Auto
covariances for ARMA Models .
R. DAHLHAUS: Parameter Estimation of Stationary Pro- 50
cesses with Spectra Containing Strong Peaks .
M. DEISTLER: Linear Error-in-Variables Models . 68
J. FRANKE, H.V. POOR: Minimax-Robust Filtering and 87
Finite-Length Robust Predictors •
H. GRAF, F.R. HAMPEL, J.-D. TACIER: The Problem of 127
Unsuspected Serial Correlations .
E.J. HANNAN: The Estimation of ARMA Processes. 146
W. HXRDLE: How to Determine the-Bandwidth of some 163
Nonlinear Smoothers in Practice •
K.S. LII,'M. ROSENBLATT: Remarks on NonGaussian 185
Linear Processes with Additive Gaussian Noise .
R.D. MARTIN, V.J. YOHAI: Gross-Error Sensitivies 198
of GM and RA-Estimates .
P. PAPANTONI-KAZAKOS: Some Aspects of Qualitative 218
Robustness in Time Series .
S. PORTNOY: Tightness of the Sequence of Empiric 231
C.D.F. Processes Defined from Regression Frac
tiles .
P.M. ROBINSON: Robust Nonparametric Autoregression • 247
P. ROUSSEEUW, V. YOHAI: Robust Regression by Means 256
of S-Estimators .
P.D. TUAN: On Robust Estimation of Parameters for 273
Autoregressive Moving Average Models .
ON THE USE OF BAYESIAN !!ODELS IN TINE SERIES ANALYSIS
Hirotugu Akaike
The Institute of Statistical r1athem.atics
4-6-7 Minami-Azabu
~-1ina to-ku
Tokyo 106
Japan
Abstract
The Bayesian modeling allows very flexible handling of
time series data. This is realized by the explicit repre
sentation of possible alternative situations by the model.
The negative psychological reaction to the use of Bayesian
models can be eliminated once we knOltl how to handle the
models.
In this paper performances of several Bayesian models de
veloped for the purpose of time series analysis are demon
strated with numerical examples. These include m.odels for
the smoothing of partial autocorrelation coefficients,
smooth impulse response function estimation and seasonal
adjustment. ~he concept of the likelihood of a Bayesian
model is playing a fundamental role for the development
of these applications.
Introduction
The basic characteristic of the Bayesian approach is the
explicit representation of the range of possible stochastic
structures through which we look at the data. 'l'he concept of
robustness can be developed only by considering the possibili
ties different to the one that is represented by the basic
model. In this sense the Bayesian modeling is directly related