Elements of Nonlinear Time Series Analysis and Forecasting

626 Pages·2017·8.011 MB·English

by  Jan G. De Gooijer

Springer Series in Statistics
Jan G. De Gooijer
Elements of
Nonlinear Time
Series Analysis
and Forecasting
Springer Series in Statistics
Series editors
Peter Bickel, CA, USA
Peter Diggle, Lancaster, UK
Stephen E. Fienberg, Pittsburgh, PA, USA
Ursula Gather, Dortmund, Germany
Ingram Olkin, Stanford, CA, USA
Scott Zeger, Baltimore, MD, USA
More information about this series at http://www.spring er.com/series/692
Jan G. De Gooijer
Elements of Nonlinear
Time Series Analysis
and Forecasting
123
Jan G. De Gooijer
University of Amsterdam
Amsterdam, The Netherlands
ISSN 0172-7397 ISSN 2197-568X (electronic)
Springer Series in Statistics
ISBN 978-3-319-43251-9 ISBN 978-3-319-43252-6 (ebook )

DOI 10.1007/978-3-319-43252-6

Library of Congress Control Number: 2017935720

© Springer International Publishing Switzerland 2017
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or
part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way,
and transmission or information storage and retrieval, electronic adaptation, computer software,
or by similar or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are
exempt from the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in
this book are believed to be true and accurate at the date of publication. Neither the publisher
nor the authors or the editors give a warranty, express or implied, with respect to the material
contained herein or for any errors or omissions that may have been made. The publisher remains
neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Printed on acid-free paper

This Springer imprint is published by Springer Nature
The registered company is Springer International Publishing AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
To Jeanne
Preface
Empiricaltimeseriesanalysisandmodelinghasbeendeviating,overthelast40years
orso, fromthelinearparadigmwiththeaimofincorporatingnonlinearfeatures. In-
deed, there are various occasions when subject-matter, theory or data suggests that
a time series is generated by a nonlinear stochastic process. If theory could provide
some understanding of the nonlinear phenomena underlying the data, the modeling
process would be relatively easy, with estimation of the model parameters being all
that is required. However, this option is rarely available in practice. Alternatively,
a particular nonlinear model may be selected, fitted to the data and subjected to a
battery of diagnostic tests to check for features that the model has failed adequately
to approximate. Although this approach corresponds to the usual model selection
strategy in linear time series analysis, it may involve rather more problems than in
the linear case.
One immediate problem is the selection of an appropriate nonlinear model or
method. However, given the wealth of nonlinear time series models now available,
this is a far from easy task. For practical use a good nonlinear model should at least
fulfill the requirement that it is general enough to capture some of the nonlinear
phenomena in the data and, moreover, should have some intuitive appeal. This
implies a systematic account of various aspects of these models and methods.
The Hungarian mathematician John von Neumann once said that the study of
nonlinear functions is akin to the study of non-elephants.1 This remark illustrates
a common problem with nonlinear theory, which in our case is equivalent to non-
linear models/methods: the subject is so vast that it is difficult to develop general
approachesandtheoriessimilartothoseexistingforlinearfunctions/models. Fortu-
nately,overthelasttwotothreedecades,thetheoryandpracticeof“non-elephants”
has made enormous progress. Indeed, several advancements have taken place in the
nonlinear model development process in order to capture specific nonlinear features
of the underlying data generating process. These features include symptoms such as
1AsimilarremarkiscreditedtothePolishmathematicianStanislawM.Ulamsayingthatusing
a term like nonlinear science is like referring to the bulk of zoology as the study of non-elephant
animals; Campbell, Farmer, Crutchfield, and Jen (1985), “Experimental mathematics: The role of
computation in nonlinear science”. Communications of the ACM, 28(4), 374–384.
vii
viii Preface
non-Gaussianity, aperiodicity, asymmetric cycles, multi-modality, nonlinear causal
relationships, nonstationarity, and time-irreversibility, among others. Additionally,
considerable progress has been made in the development of methods for real, out-
of-sample, nonlinear time series forecasting.2
Unsurprisingly, the mass of research and applications of nonlinear time series
analysis and forecasting methods is scattered over a wide range of scientific discip-
lines and numerous journal articles. This does not ensure easy access to the sub-
ject. Moreover, different papers tend to use different notations making it difficult to
conceptualize, compare, and contrast new ideas and developments across different
scientific fields. This book is my attempt to bring together, organize, extend many
of the important ideas and works in nonlinear time series analysis and forecasting,
and explain them in a comprehensive and systematic statistical framework.
While some mathematical details are needed, the main intent of the book is
to provide an overview of the current state-of-the-art of the subject, focusing on
practical issues rather than discussing technical details. To reach this goal, the
text offers a large number of examples, pseudo-algorithms, empirical exercises, and
real-world illustrations, as well as other supporting additions and features. In this
respect, I hope that the many empirical examples will testify to the breadth of the
subject matter that the book addresses. Some of the material presented in the
book is my own or developed with co-authors, but a very large part is based on the
contributions made by others. Extensive credit for such previously published work
is given throughout the book, and additional bibliographic notes are given at the
end of every chapter.
Who is this book for?
The text is designed to be used with a course in Nonlinear Time Series Analysis,
Statistical System Processing orwithacoursein Nonlinear Model Identification that
would typically be offered to graduate students in system engineering, mathematics,
statistics, and econometrics. At the same time, the book will appeal to researchers,
postgraduates, and practitioners in a wide range of other fields. Finally, the book
should be of interest to more advanced readers who would like to brush up on
their present knowledge of the subject. Thus, the book is not written toward a
singleprototypicalreaderwithaspecificbackground,anditislargelyself-contained.
Nevertheless, it is assumed that the reader has some familiarity with basic linear
time series ideas. Also, a bit of knowledge about Markov chains and Monte Carlo
simulation methods is more than welcome.
The book is selective in its coverage of subjects, although this does not imply
that a particular topic is unimportant if it is not included. For instance, Bayesian
approaches – that can relax many assumptions commonly made on the type and
nature of nonlinearity – can be applied to all models. Of course, the extensive list of
2Throughout the book, I will use the terms forecast and prediction interchangeably, although
notquiteprecisely. Thatis,predictionconcernsstatementsaboutthelikelyoutcomeofunobserved
events, not necessarily those in the future.
Preface ix
references allows readers to follow up on original sources for more technical details
on different methods. As a further help to facilitate reading, each chapter concludes
with a set of key terms and concepts, and a summary of the main findings.
What are the main features?
Here are some main features of the book.
• The book shows concrete applications of “modern” nonlinear time series ana-
lysis on a variety of empirical time series. It avoids a “theorem-proof” format.
• Thebookpresentsatoolboxofdiscrete-timenonlinearmodels,methods,tests,
andconcepts. Thereisusually,butnotinallcases,adirectfocusonthe“best”
available procedure. Alternative procedures that boast sufficient theoretical
and practical underpinning are introduced as well.
• The book uses graphs to explore and summarize real-world data, analyze the
validity of the nonlinear models fitted and present the forecasting results.
• The book covers time-domain and frequency-domain methods both for the
analysis of univariate and multivariate (vector) time series. In addition, the
book makes a clear distinction between parametric models on the one hand,
and semi- and nonparametric models/methods on the other. This offers the
reader the possibility to concentrate exclusively on one of these ways of time
series analysis.
• One additional feature of the book are the numerous algorithms in pseudo
codeformwhichstreamlinemanyideasandmaterialinasystematicway. Thus
readerscanrapidlyobtainthegeneralgistofamethodortechnique. Moreover,
it is relatively easy to convert a pseudocode to programming language.
Real data
It is well known that real data analysis can reduce the gap between theory and
practice. Hence,throughoutthebookabroadsetofempiricaltimeseries,originating
from many different scientific fields, will be used to illustrate the main points of the
text. ThisalreadystartsoffinChapter1whereIintroducefiveempiricaltimeseries
which will be used as “running” examples throughout the book. In later chapters,
other concrete examples of nonlinear time series analysis will appear. In each case,
I provide some background information about the data so that the general context
becomes clear. It may also help the reader to get a better understanding of specific
nonlinear features in the underlying data generating mechanism.
About the chapters
The text is organized as follows. Chapter 1 introduces some important terms and
concepts from linear and nonlinear time series analysis. In addition, this chapter
offers some basic tools for initial data analysis and visualization. Next, the book is
structured into two tracks.
x Preface
The first track (Chapters 2, 3, 5 – 8, and 10) mainly includes parametric non-
linear models and techniques for univariate time series analysis. Here, the overall
outline basically follows the iterative cycle of model identification, parameter es-
timation, and model verification by diagnostic checking. In particular, Chapter 2
concentrates on some important nonlinear model classes. Chapter 3 introduces the
concepts of stationarity and invertibility. The material on time-domain linearity
testing (Chapter 5), model estimation and selection (Chapter 6), tests for serial
dependence (Chapter 7), and time-reversibility (Chapter 8) relates to Chapter 2.
Although Chapter 7 is clearly based on nonparametric methods, the proposed test
statisticstrytodetectstructurein“residuals”obtainedfromfittedparametricmod-
els, and hence its inclusion in this track. If forecasting from parametric univariate
timeseriesmodelsistheobjective,Chapter10providesahostofmethods. Asapart
of the entire forecasting process, the chapter also includes methods for the construc-
tion of forecast intervals/regions, and methods for the evaluation and combination
of forecasts.
When sufficient data is available, the flexibility offered by many of the semi-
and nonparametric techniques in the second track may be preferred over parametric
models/methods. A possible starting point of this track is to test for linearity and
Gaussianity through spectral density estimation methods first (Chapter 4). In some
situations, however, a reader can jump directly to specific sections in Chapter 9
which contain extensive material on analyzing nonlinear time series by semi- and
nonparametric methods. Also some sections in Chapter 9 discuss forecasting in a
semi-andnonparametricsetting. Finally, bothtrackscontainchaptersonmultivari-
ate nonlinear time series analysis (Chapters 11 and 12). The following exhibit gives
a rough depiction of how the two tracks are interrelated.
Univariate Multivariate
8
Parametric 2 5 6 7 10 11
3
1
Semi- and nonparametric
4 9 12
Each solid directed line, denoted by a → b, represents a suggestion that Chapter
a be read before Chapter b. The medium-dashed lines indicate that some specific
chapters can be read independently. Chapters 2, 7, and 9 are somewhat lengthy,
but the dependence among sections is not very strong.
At the end of each chapter, the book contains two types of exercises. Theory
exercises illustrate and reinforce the theory at a more advanced level, and provide
results that are not available in the main text. The chapter also includes empir-