Linear models and time-series analysis : regression, ANOVA, ARMA and GARCH

881 Pages·2019·21.38 MB·English

by  Marc S

LinearModelsandTime-SeriesAnalysis
TheWileySeriesinProbabilityandStatisticsiswellestablishedandauthoritative.Itcoversmanytopicsofcurrentresearch
interestinbothpureandappliedstatisticsandprobabilitytheory.Writtenbyleadingstatisticiansandinstitutions,thetitles
spanbothstate-of-the-artdevelopmentsinthefieldandclassicalmethods.
Reflectingthewiderangeofcurrentresearchinstatistics,theseriesencompassesapplied,methodologicalandtheoretical
statistics,rangingfromapplicationsandnewtechniquesmadepossiblebyadvancesincomputerizedpracticetorigorous
treatmentoftheoreticalapproaches.
Thisseriesprovidesessentialandinvaluablereadingforallstatisticians,whetherinacademia,industry,government,or
research.
SeriesEditors:
DavidJ.Balding,UniversityCollegeLondon,UK
NoelA.Cressie,UniversityofWollongong,Australia
GarrettFitzmaurice,HavardSchoolofPublicHealth,USA
HarveyGoldstein,UniversityofBristol,UK
GeofGivens,ColoradoStateUniversity,USA
GeertMolenberghs,KatholiekeUniversiteitLeuven,Belgium
DavidW.Scott,RiceUniversity,USA
RueyS.Tsay,UniversityofChicago,USA
AdrianF.M.Smith,UniversityofLondon,UK
RelatedTitles
QuantileRegression:EstimationandSimulation,Volume2byMarilenaFurno,DomenicoVistocco
NonparametricFinancebyJussiKlemelaFebruary2018
MachineLearning:TopicsandTechniquesbyStevenW.KnoxFebruary2018
MeasuringAgreement:Models,Methods,andApplicationsbyPankajK.Choudhary,HaikadyN.NagarajaNovember2017
EngineeringBiostatistics:AnIntroductionusingMATLABandWinBUGSbyBraniVidakovicOctober2017
FundamentalsofQueueingTheory,5thEditionbyJohnF.Shortle,JamesM.Thompson,DonaldGross,CarlM.Harris
October2017
Reinsurance:ActuarialandStatisticalAspectsbyHansjoergAlbrecher,JanBeirlant,JozefL.TeugelsSeptember2017
ClinicalTrials:AMethodologicPerspective,3rdEditionbyStevenPiantadosiAugust2017
AdvancedAnalysisofVariancebyChihiroHirotsuAugust2017
MatrixAlgebraUsefulforStatistics,2ndEditionbyShayleR.Searle,AndreI.KhuriApril2017
StatisticalIntervals:AGuideforPractitionersandResearchers,2ndEditionbyWilliamQ.Meeker,GeraldJ.Hahn,LuisA.
EscobarMarch2017
TimeSeriesAnalysis:NonstationaryandNoninvertibleDistributionTheory,2ndEditionbyKatsutoTanakaMarch2017
ProbabilityandConditionalExpectation:FundamentalsfortheEmpiricalSciencesbyRolfSteyer,WernerNagelMarch2017
TheoryofProbability:AcriticalintroductorytreatmentbyBrunodeFinettiFebruary2017
SimulationandtheMonteCarloMethod,3rdEditionbyReuvenY.Rubinstein,DirkP.KroeseOctober2016
LinearModels,2ndEditionbyShayleR.Searle,MarvinH.J.GruberOctober2016
RobustCorrelation:TheoryandApplicationsbyGeorgyL.Shevlyakov,HannuOjaAugust2016
StatisticalShapeAnalysis:WithApplicationsinR,2ndEditionbyIanL.Dryden,KantiV.MardiaJuly2016
MatrixAnalysisforStatistics,3rdEditionbyJamesR.SchottJune2016
StatisticsandCausality:MethodsforAppliedEmpiricalResearchbyWolfgangWiedermann(Editor),AlexandervonEye
(Editor)May2016
TimeSeriesAnalysisbyWilfredoPalmaFebruary2016
Linear Models and Time-Series Analysis
Regression,ANOVA,ARMAandGARCH
MarcS.Paolella
DepartmentofBankingandFinance
UniversityofZurich
Switzerland
Thiseditionfirstpublished2019
©2019JohnWiley&SonsLtd
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted,inanyformor
byanymeans,electronic,mechanical,photocopying,recordingorotherwise,exceptaspermittedbylaw.Adviceonhowto
obtainpermissiontoreusematerialfromthistitleisavailableathttp://www.wiley.com/go/permissions.
TherightofDrMarcS.Paolellatobeidentifiedastheauthorofthisworkhasbeenassertedinaccordancewithlaw.
RegisteredOffices
JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,USA
JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,WestSussex,PO198SQ,UK
EditorialOffice
9600GarsingtonRoad,Oxford,OX42DQ,UK
Fordetailsofourglobaleditorialoffices,customerservices,andmoreinformationaboutWileyproductsvisitusat
www.wiley.com.
Wileyalsopublishesitsbooksinavarietyofelectronicformatsandbyprint-on-demand.Somecontentthatappearsin
standardprintversionsofthisbookmaynotbeavailableinotherformats.
LimitofLiability/DisclaimerofWarranty
Whilethepublisherandauthorshaveusedtheirbesteffortsinpreparingthiswork,theymakenorepresentationsor
warrantieswithrespecttotheaccuracyorcompletenessofthecontentsofthisworkandspecificallydisclaimallwarranties,
includingwithoutlimitationanyimpliedwarrantiesofmerchantabilityorfitnessforaparticularpurpose.Nowarrantymay
becreatedorextendedbysalesrepresentatives,writtensalesmaterialsorpromotionalstatementsforthiswork.Thefactthat
anorganization,website,orproductisreferredtointhisworkasacitationand/orpotentialsourceoffurtherinformation
doesnotmeanthatthepublisherandauthorsendorsetheinformationorservicestheorganization,website,orproductmay
provideorrecommendationsitmaymake.Thisworkissoldwiththeunderstandingthatthepublisherisnotengagedin
renderingprofessionalservices.Theadviceandstrategiescontainedhereinmaynotbesuitableforyoursituation.Youshould
consultwithaspecialistwhereappropriate.Further,readersshouldbeawarethatwebsiteslistedinthisworkmayhave
changedordisappearedbetweenwhenthisworkwaswrittenandwhenitisread.Neitherthepublishernorauthorsshallbe
liableforanylossofprofitoranyothercommercialdamages,includingbutnotlimitedtospecial,incidental,consequential,
orotherdamages.
®
MATLAB isatrademarkofTheMathWorks,Inc.andisusedwithpermission.TheMathWorksdoesnotwarrantthe
®
accuracyofthetextorexercisesinthisbook.Thiswork’suseordiscussionofMATLAB softwareorrelatedproductsdoes
notconstituteendorsementorsponsorshipbyTheMathWorksofaparticularpedagogicalapproachorparticularuseofthe
®
MATLAB software.
LibraryofCongressCataloging-in-PublicationData
Names:Paolella,MarcS.,author.
Title:Linearmodelsandtime-seriesanalysis:regression,ANOVA,ARMAand
GARCH/Dr.MarcS.Paolella.
Description:Hoboken,NJ:JohnWiley&Sons,2019.|Series:Wileyseriesin
probabilityandstatistics|
Identifiers:LCCN2018023718(print)|LCCN2018032640(ebook)|ISBN
9781119431855(AdobePDF)|ISBN9781119431985(ePub)|ISBN9781119431909
(hardcover)
Subjects:LCSH:Time-seriesanalysis.|Linearmodels(Statistics)
Classification:LCCQA280(ebook)|LCCQA280.P3732018(print)|DDC
515.5/5–dc23
LCrecordavailableathttps://lccn.loc.gov/2018023718
CoverDesign:Wiley
CoverImages:ImagescourtesyofMarcS.Paolella
Setin10/12ptWarnockProbySPiGlobal,Chennai,India
10 9 8 7 6 5 4 3 2 1
v
Contents
Preface xiii
PartI LinearModels:RegressionandANOVA 1
1 TheLinearModel 3
1.1 Regression,Correlation,andCausality 3
1.2 OrdinaryandGeneralizedLeastSquares 7
1.2.1 OrdinaryLeastSquaresEstimation 7
1.2.2 FurtherAspectsofRegressionandOLS 8
1.2.3 GeneralizedLeastSquares 12
1.3 TheGeometricApproachtoLeastSquares 17
1.3.1 Projection 17
1.3.2 Implementation 22
1.4 LinearParameterRestrictions 26
1.4.1 FormulationandEstimation 27
1.4.2 EstimabilityandIdentifiability 30
1.4.3 MomentsandtheRestrictedGLSEstimator 32
1.4.4 TestingWithh=0 34
1.4.5 TestingWithNonzeroh 37
1.4.6 Examples 37
1.4.7 ConfidenceIntervals 42
1.5 AlternativeResidualCalculation 47
1.6 FurtherTopics 51
1.7 Problems 56
1.A Appendix:DerivationoftheBLUSResidualVector 60
1.B Appendix:TheRecursiveResiduals 64
1.C Appendix:Solutions 66
2 FixedEffectsANOVAModels 77
2.1 Introduction:Fixed,Random,andMixedEffectsModels 77
2.2 TwoSamplet-TestsforDifferencesinMeans 78
2.3 TheTwoSamplet-TestwithIgnoredBlockEffects 84
vi Contents
2.4 One-WayANOVAwithFixedEffects 87
2.4.1 TheModel 87
2.4.2 EstimationandTesting 88
2.4.3 DeterminationofSampleSize 91
2.4.4 TheANOVATable 93
2.4.5 ComputingConfidenceIntervals 97
2.4.6 AWordonModelAssumptions 103
2.5 Two-WayBalancedFixedEffectsANOVA 107
2.5.1 TheModelandUseoftheInteractionTerms 107
2.5.2 SumsofSquaresDecompositionWithoutInteraction 108
2.5.3 SumsofSquaresDecompositionWithInteraction 113
2.5.4 ExampleandCodes 117
3 IntroductiontoRandomandMixedEffectsModels 127
3.1 One-FactorBalancedRandomEffectsModel 128
3.1.1 ModelandMaximumLikelihoodEstimation 128
3.1.2 DistributionTheoryandANOVATable 131
3.1.3 PointEstimation,IntervalEstimation,andSignificanceTesting 137
3.1.4 Satterthwaite’sMethod 139
3.1.5 UseofSAS 142
3.1.6 ApproximateInferenceintheUnbalancedCase 143
3.1.6.1 PointEstimationintheUnbalancedCase 144
3.1.6.2 IntervalEstimationintheUnbalancedCase 150
3.2 CrossedRandomEffectsModels 152
3.2.1 TwoFactors 154
3.2.1.1 WithInteractionTerm 154
3.2.1.2 WithoutInteractionTerm 157
3.2.2 ThreeFactors 157
3.3 NestedRandomEffectsModels 162
3.3.1 TwoFactors 162
3.3.1.1 BothEffectsRandom:ModelandParameterEstimation 162
3.3.1.2 BothEffectsRandom:ExactandApproximateConfidenceIntervals 167
3.3.1.3 MixedModelCase 170
3.3.2 ThreeFactors 174
3.3.2.1 AllEffectsRandom 174
3.3.2.2 Mixed:ClassesFixed 176
3.3.2.3 Mixed:ClassesandSubclassesFixed 177
3.4 Problems 177
3.A Appendix:Solutions 178
PartII Time-SeriesAnalysis:ARMAXProcesses 185
4 TheAR(1)Model 187
4.1 MomentsandStationarity 188
4.2 OrderofIntegrationandLong-RunVariance 195
4.3 LeastSquaresandMLEstimation 196
Contents vii
4.3.1 OLSEstimatorofa 196
4.3.2 LikelihoodDerivationI 196
4.3.3 LikelihoodDerivationII 198
4.3.4 LikelihoodDerivationIII 198
4.3.5 AsymptoticDistribution 199
4.4 Forecasting 200
4.5 SmallSampleDistributionoftheOLSandMLPointEstimators 204
4.6 AlternativePointEstimatorsofa 208
4.6.1 UseoftheJackknifeforBiasReduction 208
4.6.2 UseoftheBootstrapforBiasReduction 209
4.6.3 Median-UnbiasedEstimator 211
4.6.4 Mean-BiasAdjustedEstimator 211
4.6.5 Mode-AdjustedEstimator 212
4.6.6 Comparison 213
4.7 ConfidenceIntervalsfora 215
4.8 Problems 219
5 RegressionExtensions:AR(1)ErrorsandTime-varyingParameters 223
5.1 TheAR(1)RegressionModelandtheLikelihood 223
5.2 OLSPointandIntervalEstimationofa 225
5.3 Testinga=0intheARX(1)Model 229
5.3.1 UseofConfidenceIntervals 229
5.3.2 TheDurbin–WatsonTest 229
5.3.3 OtherTestsforFirst-orderAutocorrelation 231
5.3.4 FurtherDetailsontheDurbin–WatsonTest 236
5.3.4.1 TheBoundsTest,andCritiqueofUseofp-Values 236
5.3.4.2 LimitingPowerasa→±1 239
5.4 Bias-AdjustedPointEstimation 243
5.5 UnitRootTestingintheARX(1)Model 246
5.5.1 Nullisa=1 248
5.5.2 Nullisa<1 256
5.6 Time-VaryingParameterRegression 259
5.6.1 MotivationandIntroductoryRemarks 260
5.6.2 TheHildreth–HouckRandomCoefficientModel 261
5.6.3 TheTVPRandomWalkModel 269
5.6.3.1 CovarianceStructureandEstimation 271
5.6.3.2 TestingforParameterConstancy 274
5.6.4 RosenbergReturntoNormalcyModel 277
6 AutoregressiveandMovingAverageProcesses 281
6.1 AR(p)Processes 281
6.1.1 StationarityandUnitRootProcesses 282
6.1.2 Moments 284
6.1.3 Estimation 287
6.1.3.1 WithoutMeanTerm 287
6.1.3.2 StartingValues 290
viii Contents
6.1.3.3 WithMeanTerm 292
6.1.3.4 ApproximateStandardErrors 293
6.2 MovingAverageProcesses 294
6.2.1 MA(1)Process 294
6.2.2 MA(q)Processes 299
6.3 Problems 301
6.A Appendix:Solutions 302
7 ARMAProcesses 311
7.1 BasicsofARMAModels 311
7.1.1 TheModel 311
7.1.2 ZeroPoleCancellation 312
7.1.3 Simulation 313
7.1.4 TheARIMA(p,d,q)Model 314
7.2 InfiniteARandMARepresentations 315
7.3 InitialParameterEstimation 317
7.3.1 ViatheInfiniteARRepresentation 318
7.3.2 ViaInfiniteARandOrdinaryLeastSquares 318
7.4 Likelihood-BasedEstimation 322
7.4.1 CovarianceStructure 322
7.4.2 PointEstimation 324
7.4.3 IntervalEstimation 328
7.4.4 ModelMis-specification 330
7.5 Forecasting 331
7.5.1 AR(p)Model 331
7.5.2 MA(q)andARMA(p,q)Models 335
7.5.3 ARIMA(p,d,q)Models 339
7.6 Bias-AdjustedPointEstimation:ExtensiontotheARMAX(1,q)model 339
7.7 SomeARIMAXModelExtensions 343
7.7.1 StochasticUnitRoot 344
7.7.2 ThresholdAutoregressiveModels 346
7.7.3 FractionallyIntegratedARMA(ARFIMA) 347
7.8 Problems 349
7.A Appendix:GeneralizedLeastSquaresforARMAEstimation 351
7.B Appendix:MultivariateAR(p)ProcessesandStationarity,andGeneralBlockToeplitz
MatrixInversion 357
8 Correlograms 359
8.1 TheoreticalandSampleAutocorrelationFunction 359
8.1.1 Definitions 359
8.1.2 MarginalDistributions 365
8.1.3 JointDistribution 371
8.1.3.1 Support 371
8.1.3.2 AsymptoticDistribution 372
8.1.3.3 Small-SampleJointDistributionApproximation 375
Contents ix
8.1.4 ConditionalDistributionApproximation 381
8.2 TheoreticalandSamplePartialAutocorrelationFunction 384
8.2.1 PartialCorrelation 384
8.2.2 PartialAutocorrelationFunction 389
8.2.2.1 TPACF:FirstDefinition 389
8.2.2.2 TPACF:SecondDefinition 390
8.2.2.3 SamplePartialAutocorrelationFunction 392
8.3 Problems 396
8.A Appendix:Solutions 397
9 ARMAModelIdentification 405
9.1 Introduction 405
9.2 VisualCorrelogramAnalysis 407
9.3 SignificanceTests 412
9.4 PenaltyCriteria 417
9.5 UseoftheConditionalSACFforSequentialTesting 421
9.6 UseoftheSingularValueDecomposition 436
9.7 FurtherMethods:PatternIdentification 439
PartIII ModelingFinancialAssetReturns 443
10 UnivariateGARCHModeling 445
10.1 Introduction 445
10.2 GaussianGARCHandEstimation 450
10.2.1 BasicProperties 451
10.2.2 IntegratedGARCH 452
10.2.3 MaximumLikelihoodEstimation 453
10.2.4 VarianceTargetingEstimator 459
10.3 Non-GaussianARMA-APARCH,QMLE,andForecasting 459
10.3.1 ExtendingtheVolatility,Distribution,andMeanEquations 459
10.3.2 ModelMis-specificationandQMLE 464
10.3.3 Forecasting 467
10.4 Near-InstantaneousEstimationofNCT-APARCH(1,1) 468
10.5 S -APARCHandTestingtheIIDStableHypothesis 473
𝛼,𝛽
10.6 MixedNormalGARCH 477
10.6.1 Introduction 477
10.6.2 TheMixN(k)-GARCH(r,s)Model 478
10.6.3 ParameterEstimationandModelFeatures 479
10.6.4 Time-VaryingWeights 482
10.6.5 MarkovSwitchingExtension 484
10.6.6 MultivariateExtensions 484
11 RiskPredictionandPortfolioOptimization 487
11.1 ValueatRiskandExpectedShortfallPrediction 487