Time Series Analysis and Its Application with R examples

562 Pages·2015·7.53 MB·English

(cid:105) (cid:105)
“tsa3” — 2015/11/16 — 16:04 — page 1 — #1
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“tsa3” — 2015/11/16 — 16:04 — page 2 — #2
(cid:105) (cid:105)
Robert H. Shumway
David S. Stoffer
Time Series Analysis and
Its Applications
With R Examples
Third Edition
BluePrinting2015.11.16
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“tsa3” — 2015/11/16 — 16:04 — page v — #3
(cid:105) (cid:105)
Prof. Robert H. Shumway Prof. David S. Stoffer
Department of Statistics Department of Statistics
University of California University of Pittsburgh
Davis, California Pittsburgh, Pennsylvania
USA USA

ISSN 1431 -875X
ISBN 978-1-4419-7864-6 e-ISBN 978-1-4419-7865-3
DOI 10.1007/978-1-4419-7865-3
Springer New York Dordrecht Heidelberg London
© Springer Science+Business Media, LLC 2011
All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York,
NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in
connection with any form of information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they
are not identified as such, is not to be taken as an expression of opinion as to whether or not they are
subject to proprietary rights.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“tsa3” — 2015/11/16 — 16:04 — page vi — #4
(cid:105) (cid:105)
Preface to the Blue Printing
Considertheblueversionasanewprintingofthethirdeditionthatwillbeupdated
periodically; the printing date is on the title page. No errata will be kept for this
version.Ifyouseeatypo,sendme(DSS)anemailandI’llfixitatthenextupdate.
Theblueeditionfollowstheoriginalyellowversionbutwithinternallinks,various
fixes, additions, and updates. Hopefully you got this from the website for the text
www.stat.pitt.edu/stoffer/tsa3andyoudidn’tpaymoneyforthisversion.Allthecode
isincludedhereandprobablywon’tbepostedonthewebsite.
Thechangessofarare:
Chapter1: Minorchangestosomeexamples.Addeddefinitionoftrendstationarity.
Changedafewgraphics.Addedtheconceptofprewhiteningwhenusing
CCF.Explainednon-negativedefiniteoftheACFinmoredetail(usedto
onlybeinaproblem).
Chapter2: ChangedsmoothingexamplesfromlamemortalitydataexampletoSOI
andfindingElNiño(someexamplesremoved).Expandedonthelagged
regressionexampleandcarrieditontoChapter3.
Chapter3: RemovediidnormalfromdefinitionofARMAmodels.Addedsectionon
regression with ARMA errors and the corresponding problems (moved
fromChapter5).Changed/addedsomeexamplesintheseasonalARMA
section. Added an example on why you should not use auto.arima
fromtheforecastpackage.Carriedlaggedregressionthroughtotheend
(laggedregressionwithautocorrelatederrors).
Chapter4: Minor changes to some examples. Added star magnitude example and
discussed modulated series. Moved some of the filtering section for-
wardforeasieraccesstoinformationwhenneeded.Removedrelianceon
spec.pgram (from stats package) to mvspec (from astsa package) so
we don’t have to spend pages explaining the quirks of stats. Removed
sectiononwaveletsbecausetherearesomanyaccessibletextsavailable.
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“tsa3” — 2015/11/16 — 16:04 — page vii — #5
(cid:105) (cid:105)
PrefacetotheBluePrinting vii
Chapter5: MovedregressionwithARMAerrorstoChapter3.Someminorchanges
toexamples.
Chapter6: Same basic layout, but some sections have been moved. The Bayesian
section (now §6.11) is completely rewritten and is on linear Gaussian
statespacemodelsonly.Thenonlinearstuffisremovedbecauseit’sold
andthenewerstuffisinDouc,Moulines,andStoffer(2014).Alsoadded
a smoothing splines section (now §6.5). Polished and fixed some other
examples.
Chapter7: Nochange,it’sperfect.
Problems: Moved regression with ARMA errors problems to Chapter 3. Removed
Chapter4waveletproblems.Tweaked,rewrote,improved,revisedsome
otherproblems,buttheoverallorderingandcoverageisroughlythesame.
Appendices: Fixedsometypos,butotherwiseunchanged.
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“tsa3” — 2015/11/16 — 16:04 — page viii — #6
(cid:105) (cid:105)
Preface to the Third Edition
Thegoalsofthisbookaretodevelopanappreciationfortherichnessandversatility
of modern time series analysis as a tool for analyzing data, and still maintain a
commitmenttotheoreticalintegrity,asexemplifiedbytheseminalworksofBrillinger
(1975)andHannan(1970)andthetextsbyBrockwellandDavis(1991)andFuller
(1995). Theadvent of inexpensive powerfulcomputing has provided bothreal data
and new software that can take one considerably beyond the fitting of simple time
domainmodels,suchashavebeenelegantlydescribedinthelandmarkworkofBox
andJenkins(1970).Thisbookisdesignedtobeusefulasatextforcoursesintime
seriesonseveraldifferentlevelsandasareferenceworkforpractitionersfacingthe
analysisoftime-correlateddatainthephysical,biological,andsocialsciences.
Wehaveusedearlierversionsofthetextatboththeundergraduateandgraduate
levelsoverthepastdecade.Thereisnowan“EZEdition”onthewebsitehttp://www.
stat.pitt.edu/stoffer/tsa3/ that is more accessible to students with only basic
skills.Ourexperienceisthatanundergraduatecoursecanbeaccessibletostudents
withabackgroundinregressionanalysisandmayinclude§1.1–§1.6,§2.1–§2.3,the
resultsandnumericalpartsof§3.1–§3.9,andbrieflytheresultsandnumericalparts
of §4.1–§4.6. At the advanced undergraduate or master’s level, where the students
have some mathematical statistics background, more detailed coverage of the same
sections, with the inclusion of §2.4 and extra topics from Chapter 5 or Chapter 6
can be used as a one-semester course. Often, the extra topics are chosen by the
students according to their interests. Finally, a two-semester upper-level graduate
courseformathematics,statistics,andengineeringgraduatestudentscanbecrafted
by adding selected theoretical appendices. For the upper-level graduate course, we
shouldmentionthatwearestrivingforabroaderbutlessrigorouslevelofcoverage
than that which is attained by Brockwell and Davis (1991), the classic entry at this
level.
Themajordifferencebetweenthisthirdeditionofthetextandthesecondedition
is that we provide R code for almost all of the numerical examples. An R package
calledastsaisprovidedforusewiththetext;see§R.2fordetails.Rcodeisprovided
simplytoenhancetheexpositionbymakingthenumericalexamplesreproducible.
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“tsa3” — 2015/11/16 — 16:04 — page ix — #7
(cid:105) (cid:105)
PrefacetotheThirdEdition ix
We have tried, where possible, to keep the problem sets in order so that an
instructormayhaveaneasytimemovingfromthesecondeditiontothethirdedition.
However, some of the old problems have been revised and there are some new
problems. Also, some of the data sets have been updated. We added one section
in Chapter 5 on unit roots and enhanced some of the presentations throughout the
text. The exposition on state-space modeling, ARMAX models, and (multivariate)
regressionwithautocorrelatederrorsinChapter6havebeenexpanded.Inthisedition,
weusestandardRfunctionsasmuchaspossible,butweuseourownscripts(included
inastsa)whenwefeelitisnecessarytoavoidproblemswithaparticularRfunction;
theseproblemsarediscussedindetailonthewebsiteforthetextunderRIssues.
We thank John Kimmel, Executive Editor, Springer Statistics, for his guidance
inthepreparationandproductionofthiseditionofthetext.WearegratefultoDon
Percival,UniversityofWashington,fornumeroussuggestionsthatledtosubstantial
improvement to the presentation in the second edition, and consequently in this
edition. We thank Doug Wiens, University of Alberta, for help with some of the
R code in Chapters 4 and 7, and for his many suggestions for improvement of the
exposition. We are grateful for the continued help and advice of Pierre Duchesne,
UniversityofMontreal,andAlexanderAue,UniversityofCalifornia,Davis.Wealso
thankthemanystudentsandotherreaderswhotookthetimetomentiontypographical
errorsandothercorrectionstothefirstandsecondeditions.Finally,workonthethis
editionwassupportedbytheNationalScienceFoundationwhileoneofus(D.S.S.)
wasworkingattheFoundationundertheIntergovernmentalPersonnelAct.
Davis,CA RobertH.Shumway
Pittsburgh,PA DavidS.Stoffer
September2010
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“tsa3” — 2015/11/16 — 16:04 — page x — #8
(cid:105) (cid:105)
Contents
PrefacetotheBluePrinting ......................................... vi
PrefacetotheThirdEdition ......................................... viii
1 CharacteristicsofTimeSeries ................................... 1
1.1 Introduction ............................................... 1
1.2 TheNatureofTimeSeriesData .............................. 3
1.3 TimeSeriesStatisticalModels................................ 10
1.4 MeasuresofDependence:Autocorrelationand
Cross-Correlation .......................................... 16
1.5 StationaryTimeSeries ...................................... 20
1.6 EstimationofCorrelation.................................... 27
1.7 Vector-ValuedandMultidimensionalSeries..................... 33
Problems ...................................................... 38
2 TimeSeriesRegressionandExploratoryDataAnalysis............. 46
2.1 Introduction ............................................... 46
2.2 ClassicalRegressionintheTimeSeriesContext................. 47
2.3 ExploratoryDataAnalysis ................................... 56
2.4 SmoothingintheTimeSeriesContext ......................... 69
Problems ...................................................... 73
3 ARIMAModels................................................ 79
3.1 Introduction ............................................... 79
3.2 AutoregressiveMovingAverageModels ....................... 79
3.3 DifferenceEquations........................................ 93
3.4 AutocorrelationandPartialAutocorrelation..................... 98
3.5 Forecasting ............................................... 105
3.6 Estimation ................................................ 117
3.7 IntegratedModelsforNonstationaryData ...................... 136
3.8 BuildingARIMAModels ................................... 140
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“tsa3” — 2015/11/16 — 16:04 — page xi — #9
(cid:105) (cid:105)
Contents xi
3.9 RegressionwithAutocorrelatedErrors ........................ 148
3.10 MultiplicativeSeasonalARIMAModels ....................... 151
Problems ...................................................... 161
4 SpectralAnalysisandFiltering .................................. 170
4.1 Introduction ............................................... 170
4.2 CyclicalBehaviorandPeriodicity............................. 172
4.3 TheSpectralDensity........................................ 178
4.4 PeriodogramandDiscreteFourierTransform ................... 185
4.5 NonparametricSpectralEstimation............................ 193
4.6 ParametricSpectralEstimation ............................... 208
4.7 MultipleSeriesandCross-Spectra ............................ 212
4.8 LinearFilters .............................................. 217
4.9 LaggedRegressionModels .................................. 223
4.10 SignalExtractionandOptimumFiltering....................... 227
4.11 SpectralAnalysisofMultidimensionalSeries ................... 232
Problems ...................................................... 235
5 AdditionalTimeDomainTopics ................................. 245
5.1 Introduction ............................................... 245
5.2 LongMemoryARMAandFractionalDifferencing .............. 245
5.3 UnitRootTesting........................................... 254
5.4 GARCHModels ........................................... 257
5.5 ThresholdModels .......................................... 265
5.6 LaggedRegression:TransferFunctionModeling ................ 269
5.7 MultivariateARMAXModels................................ 274
Problems ...................................................... 287
6 State-SpaceModels............................................. 290
6.1 Introduction ............................................... 290
6.2 Filtering,Smoothing,andForecasting ......................... 295
6.3 MaximumLikelihoodEstimation ............................. 305
6.4 MissingDataModifications ................................. 314
6.5 SmoothingSplinesandtheKalmanSmoother................... 319
6.6 StructuralModels:SignalExtractionandForecasting ............ 322
6.7 State-SpaceModelswithCorrelatedErrors ..................... 325
6.7.1 ARMAXModels .................................... 327
6.7.2 MultivariateRegressionwithAutocorrelatedErrors ....... 328
6.8 BootstrappingState-SpaceModels ............................ 331
6.9 DynamicLinearModelswithSwitching ....................... 337
6.10 StochasticVolatility ........................................ 349
6.11 BayesianAnalysisofStateSpaceModels ...................... 357
Problems ...................................................... 367
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
“tsa3” — 2015/11/16 — 16:04 — page xii — #10
(cid:105) (cid:105)
xii Contents
7 StatisticalMethodsintheFrequencyDomain ..................... 374
7.1 Introduction ............................................... 374
7.2 SpectralMatricesandLikelihoodFunctions .................... 378
7.3 RegressionforJointlyStationarySeries ....................... 379
7.4 RegressionwithDeterministicInputs ......................... 388
7.5 RandomCoefficientRegression .............................. 397
7.6 AnalysisofDesignedExperiments ............................ 402
7.7 DiscriminantandClusterAnalysis ............................ 416
7.8 PrincipalComponentsandFactorAnalysis ..................... 433
7.9 TheSpectralEnvelope ...................................... 448
Problems ...................................................... 463
AppendixA LargeSampleTheory ................................. 470
A.1 ConvergenceModes ........................................ 470
A.2 CentralLimitTheorems ..................................... 477
A.3 TheMeanandAutocorrelationFunctions ...................... 481
AppendixB TimeDomainTheory ................................. 490
B.1 HilbertSpacesandtheProjectionTheorem ..................... 490
B.2 CausalConditionsforARMAModels ......................... 494
B.3 LargeSampleDistributionoftheAR(p)ConditionalLeastSquares
Estimators ................................................ 496
B.4 TheWoldDecomposition.................................... 499
AppendixC SpectralDomainTheory............................... 501
C.1 SpectralRepresentationTheorem ............................. 501
C.2 LargeSampleDistributionoftheDFTandSmoothedPeriodogram. 505
C.3 TheComplexMultivariateNormalDistribution ................. 515
AppendixR RSupplement ........................................ 521
R.1 FirstThingsFirst ........................................... 521
R.2 astsa ..................................................... 521
R.3 GettingStarted............................................. 522
R.3.1 BasicStatistics ...................................... 525
R.3.2 Packages ........................................... 526
R.3.3 Word .............................................. 527
R.4 TimeSeriesPrimer ......................................... 527
References......................................................... 535
Index ............................................................. 547
(cid:105) (cid:105)
(cid:105) (cid:105)