SINGULAR SPECTRUM ANALYSIS FOR TIME SERIES

156 Pages·2020·4.377 MB·English

by  NINA ZHIGLJAVSKY ANATOLY GOLYANDINA

SPRINGER BRIEFS IN STATISTICS
Nina Golyandina
Anatoly Zhigljavsky
Singular
Spectrum
Analysis
for Time Series
Second Edition
123
SpringerBriefs in Statistics
More information about this series at http://www.springer.com/series/8921
Nina Golyandina Anatoly Zhigljavsky
(cid:129)
Singular Spectrum Analysis
for Time Series
Second Edition
123
NinaGolyandina AnatolyZhigljavsky
Faculty of Mathematics andMechanics Schoolof Mathematics
St.Petersburg State University Cardiff University
St.Petersburg, Russia Cardiff, UK
ISSN 2191-544X ISSN 2191-5458 (electronic)
SpringerBriefs inStatistics
ISBN978-3-662-62435-7 ISBN978-3-662-62436-4 (eBook)
https://doi.org/10.1007/978-3-662-62436-4
1stedition:©TheAuthor(s)2013
2ndedition:©TheAuthor(s),underexclusivelicensetoSpringer-VerlagGmbH,DE,
partofSpringerNature2020
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseof
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
hereinorforanyerrorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregard
tojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
This Springer imprint is published by the registered company Springer-Verlag GmbH, DE part of
SpringerNature.
Theregisteredcompanyaddressis:HeidelbergerPlatz3,14197Berlin,Germany
Preface to the Second Edition
Singular spectrum analysis (SSA) is a technique of time series analysis and fore-
casting. It combines elements of classical time series analysis, multivariate statis-
tics, multivariate geometry, dynamical systems and signal processing. SSA can be
very useful in solving a variety of problems such as forecasting, imputation of
missing values, decomposition of the original time series into a sum of a small
number of interpretable components such as a slowly varying trend, oscillatory
components and a ‘structureless’ noise. For applying the core SSA algorithms,
neither a parametric model nor stationarity-type conditions have to be assumed.
This makes some versions of SSA model-free, which enables SSA to have a very
wide range of applicability. The rapidly increasing number of new applications of
SSAisaconsequenceofnewfundamentalresearchonSSAandtherecentprogress
incomputingandsoftwareengineering,whichmadeitpossibletouseSSAforvery
complicated tasks that were unthinkable 20 years ago, see Sect. 1.3.
The subject of time series analysis and forecasting is very well developed.
Despite its undisputable usefulness, SSA has not yet reached the popularity it
deserves. We see the following four main reasons why SSA is still not recognized
bycertaingroupsofstatisticians.Thefirstreasonistradition:SSAisnotaclassical
statistical method and many statisticians are simply not aware of it. Second,
mainstreamstatisticiansoftenprefermodel-basedtechniqueswherecalculationsare
automatic and do not require the computer-analyst interaction. Third, in some
instancesSSArequiressubstantialcomputingpower.Finally,SSAissometimestoo
flexible (especially when analyzing multivariate time series), and therefore has too
many options which are difficult to formalize.
The first edition of this book has been published in 2013 and has attracted
considerable attention of specialists and those interested in the analysis of com-
plicatedtimeseriesdata.Webelievethisisrelatedtothespecialstatusofthisbook
asthesolepublicationwherethemethodologyofSSAisconciselybutatthesame
time comprehensively explained. The bookexhibitsthehugepotential ofSSAand
shows how to use SSA both safely and with maximum effect.
v
vi PrefacetotheSecondEdition
A mutual agreement between the publisher and the authors concerning the
second edition of the book has been easily reached. A permission of volume
increasebyalmost15%hasbeengranted.Wehaveusedthispossibilitytobroaden
the range of topics covered; we have also revised parts of the first edition of the
book in view of recent publications on SSA and our deeper understanding of the
subject. The two main additions to the previous version of the book are:
(a) Sect. 1.2, where we discuss the place of SSA among other methods, and
(b) Sects. 2.6and3.10 devoted tomultivariateandmultidimensionalextensions of
Basic SSA. Also, we have made significant changes in all sections of Chap. 1 as
well as in Sects. 2.4.4, 2.5.2, 2.5.3 and 3.8.4. Finally, we have added Sect. 2.5.5,
wherewediscussthewaysofdealingwithoutliers.Allothersectionsofthepresent
edition are very similar to the corresponding sections in the first edition; we just
corrected a few typos, improved certain explanations and slightly modified some
text to make it consistent with the new material.
Potential readers of the book are: (a) professional statisticians and econometri-
cians; (b) specialists in any discipline where problems of time series analysis and
forecasting occur; (c) specialists in signal processing and those needed to extract
signalsfromnoisydata;(d)Ph.D.studentsworkingontopicsrelatedtotimeseries
analysis; (e) students taking appropriate M.Sc. courses on applied time series
analysis;(f)anyoneinterestedintheinterdisciplinarityofstatisticsandmathematics.
AcknowledgementsThe authorsareverymuch indebtedtoVladimirNekrutkin,a
coauthoroftheirfirstmonographonSSA.Hiscontributiontothemethodologyand
especially theory of SSA cannot be underestimated. We especially want to thank
Anton Korobeynikov, who is the original author and the maintainer of the
R-package Rssa with fast computer implementation of SSA and also our coauthor
of the book devoted to the SSA-family methods and their implementation in Rssa.
The recent development of SSA for long time series and multidimensional exten-
sions would be impossible without effective implementation. The authors very
much acknowledge many useful comments made by Jon Gillard. The authors are
also grateful to former and current Ph.D. students and collaborators of Nina
Golyandina: Konstantin Usevich, a specialist in algebraic approach to linear
recurrence relations (2D-SSA), Theodore Alexandrov (automatic SSA), Andrey
Pepelyshev (SSA for density estimation), Maxim Lomtev (SSA-ICA), Eugene
Osipov and Marina Zhukova (missing data imputation), Alex Shlemov (SSA fil-
tering, versions of SSA for improving separability). The help of Alex Shlemov in
preparation offigures is very much appreciated.
St. Petersburg, Russia Nina Golyandina
Cardiff, UK Anatoly Zhigljavsky
August 2020
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Overview of SSA Methodology and the Structure
of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 SSA-Related Topics Outside the Scope of This Book. . . . . . . . . 5
1.3 SSA and Other Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.1 Origins of SSA and Similar Techniques . . . . . . . . . . . . 6
1.3.2 Is SSA a Linear Method? . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.3 SSA and Autoregression. . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.4 SSA and Linear Regression for Trend Estimation. . . . . . 10
1.3.5 SSA and DFT, EMD, DWT . . . . . . . . . . . . . . . . . . . . . 11
1.3.6 SSA and Signal Detection; Monte Carlo SSA . . . . . . . . 12
1.4 Computer Implementation of SSA . . . . . . . . . . . . . . . . . . . . . . . 13
1.5 Historical and Bibliographical Remarks . . . . . . . . . . . . . . . . . . . 15
1.6 Common Symbols and Acronyms . . . . . . . . . . . . . . . . . . . . . . . 16
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2 Basic SSA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.1 The Main Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.1.1 Description of the Algorithm . . . . . . . . . . . . . . . . . . . . 21
2.1.2 Analysis of the Four Steps in Basic SSA. . . . . . . . . . . . 23
2.2 Potential of Basic SSA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2.1 Extraction of Trends and Smoothing . . . . . . . . . . . . . . . 29
2.2.2 Extraction of Periodic Components . . . . . . . . . . . . . . . . 31
2.2.3 Complex Trends and Periodicities with Varying
Amplitudes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.2.4 Finding Structure in Short Time Series . . . . . . . . . . . . . 33
2.2.5 Envelopes of Oscillating Signals and Estimation
of Volatility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
vii
viii Contents
2.3 Models of Time Series and SSA Objectives. . . . . . . . . . . . . . . . 36
2.3.1 SSA and Models of Time Series. . . . . . . . . . . . . . . . . . 36
2.3.2 Classification of the Main SSA Tasks . . . . . . . . . . . . . . 47
2.3.3 Separability of Components of Time Series . . . . . . . . . . 48
2.4 Choice of Parameters in Basic SSA. . . . . . . . . . . . . . . . . . . . . . 51
2.4.1 General Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.4.2 Grouping for Given Window Length. . . . . . . . . . . . . . . 55
2.4.3 Window Length. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.4.4 Signal Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2.4.5 Automatic Identification of SSA Components . . . . . . . . 67
2.5 Some Variations of Basic SSA . . . . . . . . . . . . . . . . . . . . . . . . . 70
2.5.1 Preprocessing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.5.2 Prior and Posterior Information in SSA . . . . . . . . . . . . . 72
2.5.3 Rotations for Separability . . . . . . . . . . . . . . . . . . . . . . . 75
2.5.4 Sequential SSA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
2.5.5 SSA and Outliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
2.5.6 Replacing the SVD with Other Procedures . . . . . . . . . . 81
2.5.7 Complex SSA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
2.6 Multidimensional and Multivariate Extensions of SSA . . . . . . . . 83
2.6.1 MSSA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
2.6.2 2D-SSA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
2.6.3 Shaped SSA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3 SSA for Forecasting, Interpolation, Filtering and Estimation . . . . . . 91
3.1 SSA Forecasting Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.1.1 Main Ideas and Notation . . . . . . . . . . . . . . . . . . . . . . . 91
3.1.2 Formal Description of the Algorithms . . . . . . . . . . . . . . 93
3.1.3 SSA Forecasting Algorithms: Similarities
and Dissimilarities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.1.4 Appendix: Vectors in a Subspace . . . . . . . . . . . . . . . . . 96
3.2 LRR and Associated Characteristic Polynomials. . . . . . . . . . . . . 98
3.2.1 Roots of the Characteristic Polynomials. . . . . . . . . . . . . 99
3.2.2 Min-Norm LRR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
3.3 Recurrent Forecasting as Approximate Continuation . . . . . . . . . . 102
3.3.1 Approximate Separability and Forecasting Errors. . . . . . 103
3.3.2 Approximate Continuation and Characteristic
Polynomials. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.4 Confidence Bounds for the Forecasts. . . . . . . . . . . . . . . . . . . . . 106
3.4.1 Monte Carlo and Bootstrap Confidence Intervals . . . . . . 107
3.4.2 Confidence Intervals: Comparison of Forecasting
Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.5 Summary and Recommendations on Forecasting Parameters . . . . 110
3.6 Case Study: ‘Fortified Wine’. . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Contents ix
3.7 Imputation of Missing Values . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.8 Subspace-Based Methods and Estimation of Signal
Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.8.1 Basic Facts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
3.8.2 ESPRIT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
3.8.3 Overview of Other Subspace-Based Methods. . . . . . . . . 126
3.8.4 Hankel SLRA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
3.9 SSA and Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
3.9.1 Linear Filters and Their Characteristics . . . . . . . . . . . . . 131
3.9.2 SSA Reconstruction as a Linear Filter. . . . . . . . . . . . . . 132
3.9.3 Middle Point Filter. . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
3.9.4 Last Point Filter and Forecasting. . . . . . . . . . . . . . . . . . 135
3.9.5 Causal SSA (Last-Point SSA). . . . . . . . . . . . . . . . . . . . 136
3.10 Multidimensional/Multivariate SSA . . . . . . . . . . . . . . . . . . . . . . 138
3.10.1 MSSA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
3.10.2 2D-SSA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144