Springer Series in Statistics Advisors: S. Fienberg, J. Gani, K. Krickeberg, I. Olkin, B. Singer, N. Wermuth Springer Series in Statistics Andersen/Borgan/Gill/Keiding: Statistical Models Based on Counting Processes. Anderson: Continuous-Time Markov Chains: An Applications-Oriented Approach. Andrews/Herzberg: Data: A Collection of Problems from Many Fields for the Student and Research Worker. Anscombe: Computing in Statistical Science through APL. Berger: Statistical Decision Theory and Bayesian Analysis, 2nd edition. Bolfarille/Zacks: Prediction Theory for Finite Populations. Bremaud: Point Processes and Queues: Martingale Dynamics. Brockwell/Davis: Time Series: Theory and Methods, 2nd edition. Choi: ARMA Model Identification. Daley/Vere-Jones: An Introduction to the Theory of Point Processes. Dzhaparidze: Parameter Estimation and Hypothesis Testing in Spectral Analysis of Stationary Time Series. Farrell: Multivariate Calculation. Federer: Statistical Design and Analysis for Intercropping Experiments. Fienberg/Hoaglill/Kruskal/Tanur (Eds.): A Statistical Model: Frederick Mosteller's Contributions to Statistics, Science and Public Policy. Goodman/Kruskal: Measures of Association for Cross Classifications. Grandell: Aspects of Risk Theory. Hall: The Bootstrap and Edgeworth Expansion. Hardie: Smoothing Techniques: With Implementation in S. Hartigan: Bayes Theory. Heyer: Theory of Statistical Experiments. Jolliffe: Principal Component Analysis. Kotz/Jolmson (Eds.): Breakthroughs in Statistics Volume I. Kotz/Jolmson (Eds.): Breakthroughs in Statistics Volume II. Kres: Statistical Tables for Multivariate Analysis. Leadbetter/Lilldgren/Rootzell: Extremes and Related Properties of Random Sequences and Processes. Le Cam: Asymptotic Methods in Statistical Decision Theory. Le CamfYang: Asymptotics in Statistics: Some Basic Concepts. Manoukiall: Modern Concepts and Theorems of Mathematical Statistics. Mantoll/Sillger/Suzman: Forecasting the Health of Elderly Populations. Miller, Jr.: Simultaneous Statistical Inference, 2nd edition. Mosteller/Wallace: Applied Bayesian and Classical Inference: The Case of The Federalist Papers. Pollard: Convergence of Stochastic Processes. Pratt/Gibbons: Concepts of Nonparametric Theory. Read/Cressie: Goodness-of-Fit Statistics for Discrete Multivariate Data. Reillsel: Elements of Multivariate Time Series Analysis. Reiss: A Course on Point Processes. Reiss: Approximate Distributions of Order Statistics: With Applications to Non parametric Statistics. Ross: Nonlinear Estimation. (continued aftu index) Gregory C. Reinsel Elements of Multivariate Time Series Analysis With 11 Illustrations Springer-Verlag New York Berlin Heidelberg London Paris Tokyo Hong Kong Barcelona Budapest Gregory C. Reinsel Department of Statistics University of Wisconsin, Madison Madison, WI 53706-1693 USA Mathematics Subject Classifications (1991): 62-01, 62MlO, 62M20, 62H20 Library of Congress Cataloging-in-Publication Data Reinsel, Gregory C. Elements of multivariate time series analysis / Gregory C. Reinsel. p. cm. - (Springer series in statistics) Includes bibliographical references and index. ISBN-13: 978-1-4684-0200-1 e-ISBN-13: 978-1-4684-0198-1 DOl: 10.1007/978-1-4684-0198-1 1. Time-series analysis. 2. Multivariate analysis. I. Title. II. Series. QA280.R45 1993 519.5'5-dc20 93-13954 Printed on acid-free paper. © 1993 Springer-Verlag New York, Inc. Softcover reprint of the hardcover 1s t edition 1993 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereaf ter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone. Production managed by Henry Krell; manufacturing supervised by Vincent Scelta. Photocomposed copy prepared from the author's Troff files. 987654321 To Sandy, and our children, Chris and Sarah Preface The use of methods of time series analysis in the study of multivariate time series has become of increased interest in recent years. Although the methods are rather well developed and understood for univarjate time series analysis, the situation is not so complete for the multivariate case. This book is designed to introduce the basic concepts and methods that are useful in the analysis and modeling of multivariate time series, with illustrations of these basic ideas. The development includes both traditional topics such as autocovariance and auto correlation matrices of stationary processes, properties of vector ARMA models, forecasting ARMA processes, least squares and maximum likelihood estimation techniques for vector AR and ARMA models, and model checking diagnostics for residuals, as well as topics of more recent interest for vector ARMA models such as reduced rank structure, structural indices, scalar component models, canonical correlation analyses for vector time series, multivariate unit-root models and cointegration structure, and state-space models and Kalman filtering techniques and applications. This book concentrates on the time-domain analysis of multivariate time series, and the important subject of spectral analysis is not considered here. For that topic, the reader is referred to the excellent books by Jenkins and Watts (1968), Hannan (1970), Priestley (1981), and others. The intention of this book is to introduce topics of multivariate time series in a useful way to readers who have some background in univariate time series methods of the sort available in the book by Box and Jenkins (1976). It is also necessary for the reader to have some knowledge of matrix algebra techniques and results to completely follow all developments of the topics in the book. Appendices at the end of Chapters 1 and 4 are provided which summarize and review some basic results on matrices and the multivariate normal distribution, and results on the multivariate linear model, respectively. It is hoped that these will provide the necessary background on these topics for the reader. The book is intended to provide the basic concepts needed for an adequate understanding of the material, but elaborate and detailed mathematical developments and argu ments are generally not emphasized, although substantial references are usually viii Preface provided for further mathematical details. Hence, the book will be accessible to a wider audience who have a working background in univariate time series analysis and some knowledge of matrix algebra methods and who wish to become familiar with and use multivariate time series modeling techniques in applications. The book could serve as a graduate-level textbook on "multivari ate time series" for a second course in time series as well as a reference book for researchers and practitioners in the area of multiple time series analysis. A set of exercise problems are included at the end of the book, which it is hoped will make the book more valuable for textbook use. Listings of the data sets used in the numerical examples in the book are also included in the Appendix on Data Sets. I am indebted to George Tiao and Ruey Tsay for many useful and interesting discussions, in general, on the subject of multivariate time series analysis. I would like to thank Sung Abn, Sabyasachi Basu, Sophie Yap, and others, for their helpful comments on earlier drafts of this material. I would also like to thank Lisa Ying and Eric Tam for their assistance in preparing the final figures which appear in this book. I would also like to extend my gratitude to Martin Gilchrist and others on the staff of Springer-Verlag for their interest in this book and their assistance with its preparation for publication. Finally, I express my special appreciation to my wife, Sandy, and my children, Chris and Sarah, for their help and understanding throughout this project. Gregory Reinsel April 1993 Contents Preface vii 1. Vector Time Series and Model Representations 1.1 Stationary Multivariate Time Series and Their Properties 2 1.1.1 Covariance and Correlation Matrices for a Stationary Vector Process ........................................................................ 2 1.1.2 Some Spectral Characteristics for a Stationary Vector Process ................................................................................... 4 1.1.3 Some Relations for Linear Filtering of a Stationary Vector Process .................................................... _............................. 5 1.2 Linear Model Representations for a Stationary Vector Process.. 7 1.2.1 Infinite Moving Average (Wold) Representation of a Stationary Vector Process ...................................................... 7 1.2.2 Vector Autoregressive Moving Average (ARMA) Model Representations ...................................................................... 7 A 1 Appendix: Review of Multivariate Normal Distribution and Related Topics ............................................................................ 12 Al.l Review of Some Basic Matrix Theory Results ...................... 12 A1.2 Expected Values and Covariance Matrices of Random Vectors ................................................................................... 13 A1.3 The Multivariate Normal Distribution ................................... 14 A 1.4 Some Basic Results on Stochastic Convergence 18 2. Vector ARMA Time Series Models and Forecasting 21 2.1 Vector Moving Average Models .................................... :............ 21 2.1.1 Invertibility of the Vector Moving Average Model............... 21 2.1.2 Covariance Matrices of the Vector Moving Average Model ..................................................................................... 22 2.1.3 Features of the Vector MAO) Model.................................... 23 x Contents 2.1.4 Model Structure for Subset of Components in the Vector MAModel.............................................................................. 24 2.2 Vector Autoregressive Models .................................................... 26 2.2.1 Stationarity of the Vector Autoregressive Model... ............... 26 2.2.2 Yule-Walker Relations for Covariance Matrices of a Vector AR Process ................................................................. 28 2.2.3 Covariance Features of the Vector AR( 1) Model.................. 28 2.2.4 Univariate Model Structure Implied by Vector AR Model... 29 2.3 Vector Mixed Autoregressive Moving Average Models ............. 33 2.3.1 Stationarity and Invertibility of the Vector ARMA Model.... 33 2.3.2 Relations for the Covariance Matrices of the Vector ARMA Model ........................................................................ 34 2.3.3 Some Features of the Vector ARMA(l,l) Model.................. 35 2.3.4 Consideration of Parameter Identifiability for Vector ARMA Models ....................................................................... 36 2.3.5 Further Aspects of Nonuniqueness of Vector ARMA Model Representations ...... ........... .•........ ................ ...... .......... 39 2.4 Nonstationary Vector ARMA Models ......................................... 40 2.4.1 Vector ARIMA Models for Nonstationary Processes ........... 41 2.4.2 Cointegration in Nonstationary Vector Processes ................. 42 2.4.3 The Vector IMA(I,I) Process or Exponential Smoothing Model ..................................................................................... 43 2.5 Prediction for Vector ARMA Models .......................................... 45 2.5.1 Minimum Mean Squared Error Prediction ............................. 46 2.5.2 Forecasting for Vector ARMA Processes and Covariance Matrices of Forecast Errors .................... ................................ 46 2.5.3 Computation of Forecasts for Vector ARMA Processes ....... 48 2.5.4 Some Examples of Forecast Functions for Vector ARMA Models .................................................................................... 49 3. Canonical Structure of Vector ARMA Models 52 3.1 Consideration of Kronecker Structure for Vector ARMA Models ........................................................................................ 52 3.1.1 Kronecker Indices and McMillan Degree of Vector ARMA Process ...................................................................... 53 3.1.2 Echelon Form Structure of Vector ARMA Model Implied by Kronecker Indices ...... ... ........... .......... ....... ........................ 54 3.1.3 Reduced-Rank Form of Vector ARMA Model Implied by Kronecker Indices .......................................... ........... ............. 56 3.2 Canonical Correlation Structure for ARMA Time Series ............ 58 3.2.1 Canonical Correlations for Vector ARMA Processes ........... 60 3.2.2 Relation to Scalar Component Model Structure .................... 61 Contents xi 3.3 Partial Autoregressive and Partial Correlation Matrices ............. 64 3.3.1 Vector Autoregressive Model Approximations and Partial Autoregression Matrices ....... ....... ......................... ....... .......... 64 3.3.2 Recursive Fitting of Vector AR Model Approximations ....... 66 3.3.3 Partial Cross-Correlation Matrices for a Stationary Vector Process ................................................................................... 69 3.3.4 Partial Canonical Correlations for a Stationary Vector Process ................................................................................... 71 4. Initial Model Building and Least Squares Estimation for Vector AR Models 74 4.1 Sample Cross-Covariance and Correlation Matrices and Their Properties ............................... ..................................... ............... 74 4.1.1 Sample Estimates of Mean Vector and of Covariance and Correlation Matrices .............................................................. 74 4.1.2 Asymptotic Properties of Sample Correlations ...................... 76 4.2 Sample Partial AR and Partial Correlation Matrices and Their Properties ................................................................................... 78 4.2.1 Test for Order of AR Model Based on Sample Partial Autoregression Matrices ........................................................ 78 4.2.2 Equivalent Test Statistics Based on Sample Partial Correlation Matrices .............................................................. 79 4.3 Conditional Least Squares Estimation of Vector AR Models ...... 80 4.3.1 Least Squares Estimation for the Vector AR(1) Model......... 81 4.3.2 Least Squares Estimation for the Vector AR Model of General Order ......................................................................... 83 4.3.3 Likelihood Ratio Testing for the Order of the AR Model...... 85 4.3.4 Derivation of the Wald Statistic for Testing the Order of the AR Model ......................................................................... 85 4.4 Relation of LSE to Yule-Walker Estimate for Vector AR Models ........................................................................................ 89 4.5 Additional Techniques for Specification of Vector ARMA Models ........................................................................................ 91 4.5.1 Use of Order Selection Criteria for Model Specification ....... 92 4.5.2 Sample Canonical Correlation Analysis Methods ................. 93 4.5.3 Order Determination Using Linear LSE Methods for the Vector ARMA Model ............................................................ 96 A4 Appendix: Review of the General Multivariate Linear Regression Model ................. ... ....... ............. ......... .............. ....... 105 A4.1 Properties of the Maximum Likelihood Estimator of the Regression Matrix .................................................................. 105 A4.2 Likelihood Ratio Test of Linear Hypothesis About Regression Coefficients ... .......... ....... ........ ....... ..... .................. 107
Description: