ARMA Model Identification

210 Pages·1992·3.33 MB·English

by  ByoungSeon Choi (auth.)

Springer Series in Statistics
Probability and its Applications
A Series of the Applied Probability Trust
Editors -Probability and its Applications
J. Gani, c.c. Heyde
Editors -Springer Series in Statistics
J. Berger, S. Fienberg, J. Gani, K. Krickeberg,
I. Olkin, B. Singer
Springer Series in Statistics
Anderson: Continuous-Time Markov Chains: An Applications-Oriented
Approach.
Andrews/Herzberg: Data: A Collection of Problems from Many Fields for the
Student and Research Worker.
Anscombe: Computing in Statistical Science through APL.
Berger: Statistical Decision Theory and Bayesian Analysis, 2nd edition.
Bo/farine/Zacks: Prediction Theory for Finite Populations.
Bremaud: Point Processes and Queues: Martingale Dynamics.
BrockwelllDavis: Time Series: Theory and Methods, 2nd edition.
Choi: ARMA Model Identification
Daley/Vere-Jones: An Introduction to the Theory of Point Processes.
Dzhaparidze: Parameter Estimation and Hypothesis Testing in Spectral Analysis of
Stationary Time Series.
Farrell: Multivariate Calculation.
Fienberg/Hoaglin/Kruskal/Tanur (Eds.): A Statistical Model:
Frederick Mosteller's Contributions to Statistics, Science, and
Public Policy.
Goodman/Kruskal: Measures of Association for Cross Classifications.
Grandell: Aspects of Risk Theory.
Hall: The Bootstrap and Edgeworth Expansion.
Hurdle: Smoothing Techniques: With Implementation in S.
Hartigan: Bayes Theory.
Heyer: Theory of Statistical Experiments.
Jolliffe: Principal Component Analysis.
Kotz/Johnson (Eds.): Breakthroughs in Statistics Volume I.
Kotz/Jollllson (Eds.): Breakthroughs in Statistics Volume 11.
Kres: Statistical Tables for Multivariate Analysis.
Leadbetter/Lindgren/Rootzen: Extremes and Related Properties of Random
Sequences and Processes.
Le Cam: Asymptotic Methods in Statistical Decision Theory.
Le Cam/Yang: Asymptotics in Statistics: Some Basic Concepts.
Manoukiall: Modern Concepts and Theorems of Mathematical Statistics.
Miller, Jr.: Simultaneous Statistical Inference, 2nd edition.
Mosteller/Wallace: Applied Bayesian and Classical Inference: The Case of The
Federalist Papers.
Pollard: Convergence of Stochastic Processes.
Pratt/Gibbons: Concepts of Nonparametric Theory.
Read/Cressie: Goodness-of-Fit Statistics for Discrete Multivariate Data.
Reiss: Approximate Distributions of Order Statistics: With Applications to
Nonparametric Statistics.
Ross: Nonlinear Estimation.
Saclls: Applied Statistics: A Handbook of Techniques, 2nd edition.
Salsburg: The Use of Restricted Significance Tests in Clinical Trials.
Siirndal/Swensson/Wretmall: Model Assisted Survey Sampling.
Selleta: Non-Negative Matrices and Markov Chains.
(continued after index)
ByoungSeon Choi
ARMA Model
Identification
Springer-Vedag
New York Berlin Heidelberg London Paris
Tokyo Hong Kong Barcelona Budapest
ByoungSeon Choi
Department of Applied Statistics
Yonsei University
Seoul,120-749
Korea
Series Editors
1. Gani C.C.Heyde
Department of Statistics Department of Statistics
University of California Institute of Advanced Studies
Santa Barbara, CA 93106 The Australian National University
USA GPO Box 4, Canberra ACf 2601
Australia
Library of Congress Cataloging-in-Publication Data
Choi, ByoungSeon
ARMA model identification
p. cm. --(Probability and its applications)
Includes bibliographical references and index.
ISBN -13: 978 -1-4613-9747-2 e-ISBN- 13: 978 -1-4613-9745-8
DOl: 10.1007/978 -1-4613-9745-8
1. Autoregression (Statistics) 2. Linear models (Statistics)
I. Title. 11. Series
QA278.2.C555 1992
519.5'36--dc20
for Library of Congress 91-45681
Printed on acid-free paper.
© 1992 by Applied Probability Trust
Softcover reprint of the hardcover 1st edition 1992
All rights reserved. This work may not be translated or copied in whole or in part without the writ
ten permission of the publisher (Springer-VerIag New York, Inc., 175 Fifth Avenue, New York, NY
10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in con
nection with any form of information storage and retrieval, electronic adaptation, computer soft
ware, or by similar or dissimilar methodology now known or hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the
former are not especially identified, is not to be taken as a sign that such names, as understood by
the Trade Marks and Merchandise Act, may accordingly be used freely by anyone.
Production managed by Dimitry L. Loseff; manufacturing supervised by Jacqui Ashri.
Photocomposed pages prepared from the author's LATEX files.
9 8 765 432 1
ISBN-13: 978-1-4613-9747-2
Much of human experience is random,
and almost all of it unexpected ....
J. Gani
Journal of Applied Probability
Special Volume 25A (1988), p. 3.
Preface
During the last two decades, considerable progress has been made in statis
tical time series analysis. In particular, numerous techniques utilizing the
autoregressive moving-average (ARMA) model have been developed. Three
books have played important roles in these developments; they are the texts
by Box and Jenkins (1976), Hannan (1970a), and Anderson (1971).
ARMA modeling is usually classified into three stages. The first, model
identification, aims to determine the orders of the ARMA model. The sec
ond, model estimation, is designed to estimate the ARMA coefficients.
The last, diagnostic checking, examines the goodness-of-fit of the estimated
model. Since the above-mentioned books appeared, various new methods
have been proposed for the three stages, particularly for identification.
Previously, methods based on the testing of hypotheses had been popu
larly employed as ARMA model identification procedures. In their book,
Box and Jenkins (1976) used the auto correlation function and the partial
autocorrelation function. In the early 1970s, Akaike presented two penalty
function identification methods known as the FPE and the AIC. To circum
vent the inconsistency problem of the criteria, the BIC and Hannan and
Quinn's method were proposed. At the same time, Cleveland suggested the
use of the inverse autocorrelation function. In 1982, Hannan and Rissa
nen employed the instrumental regression technique as well as the penalty
functions for ARMA modeling. This method has resulted in consistent es
timates of the orders. Similar methods have been proposed by Koreisha
and Pukkila. Throughout the 1980s, identification methods using the pat
terns of some functions of the auto correlations have been studied. These
are called pattern identification methods and include Woodside's method,
the R and S method of Gray, Kelley, and McIntire, the Corner method of
Beguin, Gourieroux and Monfort, the three GPAC methods of Woodward
and Gray, of Glasbey, and of Takemura, the ESACF method of Tsay and
Tiao, the SCAN method of Tsay and Tiao, and the 3-pattern method of
Choi.
Although some of the identification methods are mentioned in the litera
ture, there is no book that brings all of them together. The aim of this book
is to give an account of all the identification methods mentioned above. Of
course, it is impossible to examine all of these in detail. However, I have
tried to survey as many identification methods as possible. Because my
purpose is to explain all the methods in this short book, what I have em-
phasized is not the mathematical details of the methods but rather their
fundamental ideas. Appropriate references are provided for further illus
trative explorations and mathematical investiga~ions. At the end of each
chapter, additional references are given; these are basically for graduate
students starting research in time series analysis, and for readers who want
to apply recently developed time series analysis techniques to their research.
Although this book may at first sight appear to be too difficult for graduate
students, I hope that it can be used as an auxiliary textbook for graduate
courses. It is, in fact, based on my lecture notes for a time series analysis
course taught at Yonsei University in Seoul.
This monograph was written during my sabbatical leave at the Depart
ment of Statistics at the University of California, Santa Barbara (UCSB).
I must thank the faculty, staff, and graduate students for their friendliness
and help during my stay; I wish to express my gratitude to Professor Joe
Gani, who has led me to understand more deeply the values of human and
scholarly life. His warmth and consideration have helped to ease the burden
of the death of one very close to me in 1990.
I am greatly indebted to H. Akaike, E. J. Hannan, J. R. M. Hosking, S. G.
Koreisha, E. Parzen, T. M. Pukkila, J. Rissanen, G. C. Tiao, and R. S. Tsay,
whose comments on an early version of the manuscript led to substantial
improvements. My friends Seong-Cheol Cho, KiYoung Choi, MooYoung
Choi, Hay Y. Chung, Myung-Hoe Huh, SeJung Oh, and Sukgoo Pak read
drafts carefully, and I appreciate their contributions. I would also like to
express my gratitude to current and former graduate students Wonkyung
Lee, Hyun-Ju Noh, GoungJoo Shin, and SeongBack Yi at Yonsei University
and to Benny Cheng, Jie Huang, Aaron Gross, and Jeffrey Stein at UCSB
as well as to Mr. F. Schubert at the UCSB Humanities Computing Facility.
Finally, I wish to thank my friend HoYoun Kim for his support.
CBS
viii
Contents
1 Introduction 1
1.1 ARMA Model. 1
1.2 History .... 2
1.3 Algorithms .. 3
1.3.1 AR Parameters 3
1.3.2 MA Parameters . 9
1.4 Estimation ...... . 13
1.4.1 Extended Yule-Walker Estimates 14
1.4.2 Maximum Likelihood Estimates . 17
1.5 Nonstationary Processes ........ . 19
1.5.1 Sample ACRF of a Nonstationary Process. 21
1.5.2 Iterated Least Squares Estimates 23
1.6 Additional References ............... . 25
2 The Autocorrelation Methods 29
2.1 Box and Jenkins' Method ....... . 29
2.2 The Inverse Autocorrelation Method .. 31
2.2.1 Inverse Autocorrelation Function 31
2.2.2 Estimates of the Spectral Density. 33
2.2.3 Estimates of the IACF . . . . . 37
2.2.4 Identification Using the IACF . 39
2.3 Additional References ........ . 41
3 Penalty Function Methods 43
3.1 The Final Prediction Error Method ..... . 44
3.2 Akaike's Information Criterion ....... . 47
3.2.1 Kullback-Leibler Information Number 47
3.2.2 Akaike's Information Criterion 49
3.3 Generalizations............ 51
3.4 Parzen's Method . . . . . . . . . . . 55
3.5 The Bayesian Information Criterion 58
3.5.1 Schwarz' Derivation ... . 58
3.5.2 Kashyap's Derivation .. . 60
3.5.3 Shortest Data Description . 60
3.5.4 Some Comments . . . . . . 64
3.6 Hannan and Quinn's Criterion 65
3.7 Consistency . . . . . . . . . . . 67
3.8 Some Relations . . . . . . . . . 68
3.8.1 A Bayesian Interpretation 68
3.8.2 The BIC and Prediction Errors 69
3.8.3 The AIC and Cross-Validations . 71
3.9 Additional References · .... . . . . . 72
4 Innovation Regression Methods 75
4.1 AR and MA Approximations 75
4.2 Hannan and Rissanen's Method. 77
4.2.1 A Three-Stage Procedure 77
4.2.2 Block Toeplitz Matrices 79
4.2.3 A Modification of the Whittle Algorithm 84
4.2.4 Some Modifications ... 86
4.3 Koreisha and Pukkila's Method . 91
4.4 The KL Spectral Density 94
4.5 Additional References · .... 98
5 Pattern Identification Methods 101
5.1 The 3-Pattern Method . . . . . 102
5.1.1 The Three Functions . . 102
5.1.2 Asymptotic Distributions 106
5.1.3 Two Chi-Squared Statistics 111
5.2 The R and S Array Method . . . 113
5.2.1 The R and S Patterns . . 113
5.2.2 Asymptotic Distributions 117
5.2.3 The RS Array · ..... 118
5.3 The Corner Method ....... 119
5.3.1 Correlation Determinants 119
5.3.2 Asymptotic Distribution . 120
5.4 The GPAC Methods ....... 121
5.4.1 Woodward and Gray's GPAC 121
5.4.2 Glasbey's GPAC 124
5.4.3 Takemura's GPAC 125
5.5 The ESACF Method . 126
5.6 The SCAN Method. . . . 129
5.6.1 Eigen-analysis. . . 129
5.6.2 The SCAN Method 131
5.7 Woodside's Method. . . . . 132
5.8 Three Systems of Equations 133
5.9 Additional References · .. 136
x
6 Testing Hypothesis Methods 139
6.1 Three Asymptotic Test Procedures 139
6.2 Some Test Statistics . . . . . . 141
6.3 The Portmanteau Statistic. . . 145
6.4 Sequential Testing Procedures . 147
6.5 Additional References ..... 148
Bibliography 149
Index 197
xi