Elements of Multivariate Time Series Analysis

277 Pages·1993·5.47 MB·English

by  Gregory C. Reinsel (auth.)

Springer Series in Statistics
Advisors:
S. Fienberg, J. Gani, K. Krickeberg,
I. Olkin, B. Singer, N. Wermuth
Springer Series in Statistics
Andersen/Borgan/Gill/Keiding: Statistical Models Based on Counting Processes.
Anderson: Continuous-Time Markov Chains: An Applications-Oriented Approach.
Andrews/Herzberg: Data: A Collection of Problems from Many Fields for the Student
and Research Worker.
Anscombe: Computing in Statistical Science through APL.
Berger: Statistical Decision Theory and Bayesian Analysis, 2nd edition.
Bolfarille/Zacks: Prediction Theory for Finite Populations.
Bremaud: Point Processes and Queues: Martingale Dynamics.
Brockwell/Davis: Time Series: Theory and Methods, 2nd edition.
Choi: ARMA Model Identification.
Daley/Vere-Jones: An Introduction to the Theory of Point Processes.
Dzhaparidze: Parameter Estimation and Hypothesis Testing in Spectral Analysis of
Stationary Time Series.
Farrell: Multivariate Calculation.
Federer: Statistical Design and Analysis for Intercropping Experiments.
Fienberg/Hoaglill/Kruskal/Tanur (Eds.): A Statistical Model: Frederick Mosteller's
Contributions to Statistics, Science and Public Policy.
Goodman/Kruskal: Measures of Association for Cross Classifications.
Grandell: Aspects of Risk Theory.
Hall: The Bootstrap and Edgeworth Expansion.
Hardie: Smoothing Techniques: With Implementation in S.
Hartigan: Bayes Theory.
Heyer: Theory of Statistical Experiments.
Jolliffe: Principal Component Analysis.
Kotz/Jolmson (Eds.): Breakthroughs in Statistics Volume I.
Kotz/Jolmson (Eds.): Breakthroughs in Statistics Volume II.
Kres: Statistical Tables for Multivariate Analysis.
Leadbetter/Lilldgren/Rootzell: Extremes and Related Properties of Random Sequences
and Processes.
Le Cam: Asymptotic Methods in Statistical Decision Theory.
Le CamfYang: Asymptotics in Statistics: Some Basic Concepts.
Manoukiall: Modern Concepts and Theorems of Mathematical Statistics.
Mantoll/Sillger/Suzman: Forecasting the Health of Elderly Populations.
Miller, Jr.: Simultaneous Statistical Inference, 2nd edition.
Mosteller/Wallace: Applied Bayesian and Classical Inference: The Case of The
Federalist Papers.
Pollard: Convergence of Stochastic Processes.
Pratt/Gibbons: Concepts of Nonparametric Theory.
Read/Cressie: Goodness-of-Fit Statistics for Discrete Multivariate Data.
Reillsel: Elements of Multivariate Time Series Analysis.
Reiss: A Course on Point Processes.
Reiss: Approximate Distributions of Order Statistics: With Applications to Non
parametric Statistics.
Ross: Nonlinear Estimation.
(continued aftu index)
Gregory C. Reinsel
Elements of Multivariate
Time Series Analysis
With 11 Illustrations
Springer-Verlag
New York Berlin Heidelberg London Paris
Tokyo Hong Kong Barcelona Budapest
Gregory C. Reinsel
Department of Statistics
University of Wisconsin, Madison
Madison, WI 53706-1693 USA
Mathematics Subject Classifications (1991): 62-01, 62MlO, 62M20, 62H20
Library of Congress Cataloging-in-Publication Data
Reinsel, Gregory C.
Elements of multivariate time series analysis / Gregory C.
Reinsel.
p. cm. - (Springer series in statistics)
Includes bibliographical references and index.
ISBN-13: 978-1-4684-0200-1 e-ISBN-13: 978-1-4684-0198-1
DOl: 10.1007/978-1-4684-0198-1
1. Time-series analysis. 2. Multivariate analysis. I. Title.
II. Series.
QA280.R45 1993
519.5'5-dc20 93-13954
Printed on acid-free paper.
© 1993 Springer-Verlag New York, Inc.
Softcover reprint of the hardcover 1s t edition 1993
All rights reserved. This work may not be translated or copied in whole or in part without the
written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New
York, NY 10010, USA), except for brief excerpts in connection with reviews or scholarly
analysis. Use in connection with any form of information storage and retrieval, electronic
adaptation, computer software, or by similar or dissimilar methodology now known or hereaf
ter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication, even
if the former are not especially identified, is not to be taken as a sign that such names, as
understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely
by anyone.
Production managed by Henry Krell; manufacturing supervised by Vincent Scelta.
Photocomposed copy prepared from the author's Troff files.
987654321
To Sandy,
and our children, Chris and Sarah
Preface
The use of methods of time series analysis in the study of multivariate time
series has become of increased interest in recent years. Although the methods
are rather well developed and understood for univarjate time series analysis, the
situation is not so complete for the multivariate case. This book is designed to
introduce the basic concepts and methods that are useful in the analysis and
modeling of multivariate time series, with illustrations of these basic ideas. The
development includes both traditional topics such as autocovariance and auto
correlation matrices of stationary processes, properties of vector ARMA models,
forecasting ARMA processes, least squares and maximum likelihood estimation
techniques for vector AR and ARMA models, and model checking diagnostics
for residuals, as well as topics of more recent interest for vector ARMA models
such as reduced rank structure, structural indices, scalar component models,
canonical correlation analyses for vector time series, multivariate unit-root
models and cointegration structure, and state-space models and Kalman filtering
techniques and applications. This book concentrates on the time-domain
analysis of multivariate time series, and the important subject of spectral
analysis is not considered here. For that topic, the reader is referred to the
excellent books by Jenkins and Watts (1968), Hannan (1970), Priestley (1981),
and others.
The intention of this book is to introduce topics of multivariate time series in
a useful way to readers who have some background in univariate time series
methods of the sort available in the book by Box and Jenkins (1976). It is also
necessary for the reader to have some knowledge of matrix algebra techniques
and results to completely follow all developments of the topics in the book.
Appendices at the end of Chapters 1 and 4 are provided which summarize and
review some basic results on matrices and the multivariate normal distribution,
and results on the multivariate linear model, respectively. It is hoped that these
will provide the necessary background on these topics for the reader. The book
is intended to provide the basic concepts needed for an adequate understanding
of the material, but elaborate and detailed mathematical developments and argu
ments are generally not emphasized, although substantial references are usually
viii Preface
provided for further mathematical details. Hence, the book will be accessible to
a wider audience who have a working background in univariate time series
analysis and some knowledge of matrix algebra methods and who wish to
become familiar with and use multivariate time series modeling techniques in
applications. The book could serve as a graduate-level textbook on "multivari
ate time series" for a second course in time series as well as a reference book for
researchers and practitioners in the area of multiple time series analysis. A set
of exercise problems are included at the end of the book, which it is hoped will
make the book more valuable for textbook use. Listings of the data sets used in
the numerical examples in the book are also included in the Appendix on Data
Sets.
I am indebted to George Tiao and Ruey Tsay for many useful and interesting
discussions, in general, on the subject of multivariate time series analysis. I
would like to thank Sung Abn, Sabyasachi Basu, Sophie Yap, and others, for
their helpful comments on earlier drafts of this material. I would also like to
thank Lisa Ying and Eric Tam for their assistance in preparing the final figures
which appear in this book. I would also like to extend my gratitude to Martin
Gilchrist and others on the staff of Springer-Verlag for their interest in this book
and their assistance with its preparation for publication. Finally, I express my
special appreciation to my wife, Sandy, and my children, Chris and Sarah, for
their help and understanding throughout this project.
Gregory Reinsel
April 1993
Contents
Preface vii
1. Vector Time Series and Model Representations
1.1 Stationary Multivariate Time Series and Their Properties 2
1.1.1 Covariance and Correlation Matrices for a Stationary
Vector Process ........................................................................ 2
1.1.2 Some Spectral Characteristics for a Stationary Vector
Process ................................................................................... 4
1.1.3 Some Relations for Linear Filtering of a Stationary Vector
Process .................................................... _............................. 5
1.2 Linear Model Representations for a Stationary Vector Process.. 7
1.2.1 Infinite Moving Average (Wold) Representation of a
Stationary Vector Process ...................................................... 7
1.2.2 Vector Autoregressive Moving Average (ARMA) Model
Representations ...................................................................... 7
A 1 Appendix: Review of Multivariate Normal Distribution and
Related Topics ............................................................................ 12
Al.l Review of Some Basic Matrix Theory Results ...................... 12
A1.2 Expected Values and Covariance Matrices of Random
Vectors ................................................................................... 13
A1.3 The Multivariate Normal Distribution ................................... 14
A 1.4 Some Basic Results on Stochastic Convergence 18
2. Vector ARMA Time Series Models and Forecasting 21
2.1 Vector Moving Average Models .................................... :............ 21
2.1.1 Invertibility of the Vector Moving Average Model............... 21
2.1.2 Covariance Matrices of the Vector Moving Average
Model ..................................................................................... 22
2.1.3 Features of the Vector MAO) Model.................................... 23
x Contents
2.1.4 Model Structure for Subset of Components in the Vector
MAModel.............................................................................. 24
2.2 Vector Autoregressive Models .................................................... 26
2.2.1 Stationarity of the Vector Autoregressive Model... ............... 26
2.2.2 Yule-Walker Relations for Covariance Matrices of a
Vector AR Process ................................................................. 28
2.2.3 Covariance Features of the Vector AR( 1) Model.................. 28
2.2.4 Univariate Model Structure Implied by Vector AR Model... 29
2.3 Vector Mixed Autoregressive Moving Average Models ............. 33
2.3.1 Stationarity and Invertibility of the Vector ARMA Model.... 33
2.3.2 Relations for the Covariance Matrices of the Vector
ARMA Model ........................................................................ 34
2.3.3 Some Features of the Vector ARMA(l,l) Model.................. 35
2.3.4 Consideration of Parameter Identifiability for Vector
ARMA Models ....................................................................... 36
2.3.5 Further Aspects of Nonuniqueness of Vector ARMA
Model Representations ...... ........... .•........ ................ ...... .......... 39
2.4 Nonstationary Vector ARMA Models ......................................... 40
2.4.1 Vector ARIMA Models for Nonstationary Processes ........... 41
2.4.2 Cointegration in Nonstationary Vector Processes ................. 42
2.4.3 The Vector IMA(I,I) Process or Exponential Smoothing
Model ..................................................................................... 43
2.5 Prediction for Vector ARMA Models .......................................... 45
2.5.1 Minimum Mean Squared Error Prediction ............................. 46
2.5.2 Forecasting for Vector ARMA Processes and Covariance
Matrices of Forecast Errors .................... ................................ 46
2.5.3 Computation of Forecasts for Vector ARMA Processes ....... 48
2.5.4 Some Examples of Forecast Functions for Vector ARMA
Models .................................................................................... 49
3. Canonical Structure of Vector ARMA Models 52
3.1 Consideration of Kronecker Structure for Vector ARMA
Models ........................................................................................ 52
3.1.1 Kronecker Indices and McMillan Degree of Vector
ARMA Process ...................................................................... 53
3.1.2 Echelon Form Structure of Vector ARMA Model Implied
by Kronecker Indices ...... ... ........... .......... ....... ........................ 54
3.1.3 Reduced-Rank Form of Vector ARMA Model Implied by
Kronecker Indices .......................................... ........... ............. 56
3.2 Canonical Correlation Structure for ARMA Time Series ............ 58
3.2.1 Canonical Correlations for Vector ARMA Processes ........... 60
3.2.2 Relation to Scalar Component Model Structure .................... 61
Contents xi
3.3 Partial Autoregressive and Partial Correlation Matrices ............. 64
3.3.1 Vector Autoregressive Model Approximations and Partial
Autoregression Matrices ....... ....... ......................... ....... .......... 64
3.3.2 Recursive Fitting of Vector AR Model Approximations ....... 66
3.3.3 Partial Cross-Correlation Matrices for a Stationary Vector
Process ................................................................................... 69
3.3.4 Partial Canonical Correlations for a Stationary Vector
Process ................................................................................... 71
4. Initial Model Building and Least Squares Estimation for Vector AR
Models 74
4.1 Sample Cross-Covariance and Correlation Matrices and Their
Properties ............................... ..................................... ............... 74
4.1.1 Sample Estimates of Mean Vector and of Covariance and
Correlation Matrices .............................................................. 74
4.1.2 Asymptotic Properties of Sample Correlations ...................... 76
4.2 Sample Partial AR and Partial Correlation Matrices and Their
Properties ................................................................................... 78
4.2.1 Test for Order of AR Model Based on Sample Partial
Autoregression Matrices ........................................................ 78
4.2.2 Equivalent Test Statistics Based on Sample Partial
Correlation Matrices .............................................................. 79
4.3 Conditional Least Squares Estimation of Vector AR Models ...... 80
4.3.1 Least Squares Estimation for the Vector AR(1) Model......... 81
4.3.2 Least Squares Estimation for the Vector AR Model of
General Order ......................................................................... 83
4.3.3 Likelihood Ratio Testing for the Order of the AR Model...... 85
4.3.4 Derivation of the Wald Statistic for Testing the Order of
the AR Model ......................................................................... 85
4.4 Relation of LSE to Yule-Walker Estimate for Vector AR
Models ........................................................................................ 89
4.5 Additional Techniques for Specification of Vector ARMA
Models ........................................................................................ 91
4.5.1 Use of Order Selection Criteria for Model Specification ....... 92
4.5.2 Sample Canonical Correlation Analysis Methods ................. 93
4.5.3 Order Determination Using Linear LSE Methods for the
Vector ARMA Model ............................................................ 96
A4 Appendix: Review of the General Multivariate Linear
Regression Model ................. ... ....... ............. ......... .............. ....... 105
A4.1 Properties of the Maximum Likelihood Estimator of the
Regression Matrix .................................................................. 105
A4.2 Likelihood Ratio Test of Linear Hypothesis About
Regression Coefficients ... .......... ....... ........ ....... ..... .................. 107