Stationary Stochastic Processes for Scientists and Engineers

316 Pages·2013·65.354 MB·English

by  Lindgren, Georg, Rootzen, Holger, Sandsten, Maria

Statistics
S SSTTAATTIIOONNAARRYY
T
“This book is a lucid and well-paced introduction to stationary
F A
stochastic processes, superbly motivated and illustrated through a
O T
wealth of convincing applications in science and engineering. It offers SSTTOOCCHHAASSTTIICC
R I
O
a clear guide to the formulation and mathematical properties of these

S
N
processes and to some non-stationary processes too … The reader
C
will find tools for analysis and calculation and also—importantly— I A PPRROOCCEESSSSEESS FFOORR
E R
material to deepen understanding and generate enthusiasm and
N Y
confidence. An outstanding text.”
T
—Clive Anderson, University of Sheffield S SSCCIIEENNTTIISSTTSS AANNDD
I
T
S
Carefully balancing mathematical rigor and ease of exposition, T O
Stationary Stochastic Processes for Scientists and Engineers S C
EENNGGIINNEEEERRSS
H
teaches you how to use these processes efficiently. The book A
provides you with a sufficient understanding of the theory and a N A
S
practical appreciation of how it is used in real-life situations. Special D
T
emphasis is on the interpretation of various statistical models and
E I
C
concepts as well as the types of questions statistical analysis can N

answer. G P
R
I
Features N O
• Explains the relationship between a covariance function and E C
spectral density E E
• Illustrates the difference between Fourier analysis of data and R S
S S
Fourier transformation of a covariance function
E
• Covers AR, MA, ARMA, and GARCH models
S
• Details covariance and spectral estimation
• Shows how stochastic processes act in linear filters, including GGeeoorrgg LLiinnddggrreenn
the matched, Wiener, and Kalman filters
• Describes Monte Carlo simulations of different types of
HHoollggeerr RRoooottzzéénn
S L
processes a R i
n o n
• Includes many examples from applied fields as well as exercises d o d
s tz g MMaarriiaa SSaannddsstteenn
that highlight both the theory and practical situations in discrete t é r
e e
n
and continuous time n n
K20279
K20279_Cover.indd 1 9/12/13 9:13 AM
STATIONARY
STOCHASTIC
PROCESSES FOR
SCIENTISTS AND
ENGINEERS
3
9
7
5
3
3
5
3
4
1
8|
4
7
2
1
5
n|
ut
STATIONARY
STOCHASTIC
PROCESSES FOR
SCIENTISTS AND
ENGINEERS
Georg Lindgren
Lund University
Lund, Sweden
Holger Rootzén
Chalmers University of Technology
Gothenburg, Sweden
Maria Sandsten
Lund University
Lund, Sweden
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2014 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20130830
International Standard Book Number-13: 978-1-4665-8619-2 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information stor-
age or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copy-
right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222
Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that pro-
vides licenses and registration for a variety of users. For organizations that have been granted a pho-
tocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are
used only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Preface xi
Asurveyofthecontents xiii
1 Stochasticprocesses 1
1.1 Somestochastic models 1
1.2 Definitionofastochastic process 10
1.3 Distribution functions 14
1.3.1 TheCDFfamily 15
1.3.2 Gaussianprocesses 17
2 Stationaryprocesses 19
2.1 Introduction 19
2.2 Momentfunctions 19
2.2.1 Themomentfunctions 21
2.2.2 Simpleproperties andrules 24
2.2.3 Interpretation ofmomentsandmomentfunctions 26
2.3 Stationary processes 28
2.3.1 Strictlystationary processes 30
2.3.2 Weaklystationary processes 32
2.3.3 Importantproperties ofthecovariance function 33
2.4 Randomphaseandamplitude 35
2.4.1 Arandom harmonicoscillation 35
2.4.2 Superposition ofrandom harmonicoscillations 38
2.4.3 Powerandaveragepower 39
2.5 Estimationofmeanvalueandcovariance function 41
2.5.1 Estimationofthemeanvaluefunction 42
2.5.2 Ergodicity 47
2.5.3 Estimatingthecovariancefunction 49
2.5.4 Ergodicity asecondtime 53
2.5.5 Processeswithcontinuous time 54
v
vi CONTENTS
2.6 Stationary processes andthenon-stationary reality 54
2.7 MonteCarlosimulation fromcovariance function 55
2.7.1 Whysimulateastochastic process? 55
2.7.2 Simulation of Gaussian process from covariance
function 56
Exercises 57
3 ThePoissonprocessanditsrelatives 63
3.1 Introduction 63
3.2 ThePoissonprocess 63
3.2.1 Definitionandsimpleproperties 64
3.2.2 Interarrival timeproperties 66
3.2.3 Somedistribution properties 68
3.3 Stationary independent increments 70
3.3.1 Ageneralclasswithinteresting properties 70
3.3.2 Covarianceproperties 71
3.4 Thecovariance intensity function 74
3.4.1 Correlation intensity 74
3.4.2 Countingprocesses withcorrelated increments 75
3.5 SpatialPoissonprocess 76
3.6 Inhomogeneous Poissonprocess 76
3.7 MonteCarlosimulation ofPoissonprocesses 77
3.7.1 Homogeneous Poissonprocesses inRandRn 77
3.7.2 Inhomogeneous Poissonprocesses 78
Exercises 79
4 Spectralrepresentations 81
4.1 Introduction 81
4.2 Spectrum incontinuous time 82
4.2.1 Definitionandgeneralproperties 82
4.2.2 Continuous spectrum 84
4.2.3 Someexamples 85
4.2.4 Discretespectrum 90
4.3 Spectrum indiscretetime 92
4.3.1 Definition 92
4.3.2 Fouriertransformation ofdata 94
4.4 Samplingandthealiasing effect 96
4.5 Afewmoreremarksanddifficulties 101
4.5.1 Thesamplingtheorem 101
4.5.2 Fourierinversion 103
CONTENTS vii
4.5.3 Spectral representation ofthe second-moment func-
tionb(τ) 105
4.5.4 Beyondspectrum andcovariance function 106
4.6 MonteCarlosimulation fromspectrum 106
Exercises 107
5 Gaussianprocesses 111
5.1 Introduction 111
5.2 Gaussianprocesses 112
5.2.1 Stationary Gaussianprocesses 113
5.2.2 Gaussianprocesswithdiscretespectrum 115
5.3 TheWienerprocess 117
5.3.1 Theone-dimensional Wienerprocess 117
5.3.2 Selfsimilarity 118
5.3.3 Brownianmotion 119
5.4 RelativesoftheGaussianprocess 121
5.5 TheLévyprocessandshotnoiseprocess 123
5.5.1 TheLévyprocess 123
5.5.2 Shotnoise 123
5.6 MonteCarlosimulationofaGaussianprocessfromspectrum 126
5.6.1 Simulationbyrandomcosines 128
5.6.2 Simulationviacovariance function 130
Exercises 131
6 Linearfilters–generaltheory 133
6.1 Introduction 133
6.2 Linearsystemsandlinearfilters 135
6.2.1 Filterwithrandom input 135
6.2.2 Impulseresponse 137
6.2.3 Momentrelations 138
6.2.4 Frequencyfunction andspectralrelations 139
6.3 Continuity, differentiation, integration 143
6.3.1 Quadraticmeanconvergence instochastic processes 144
6.3.2 Differentiation 145
6.3.3 Integration 150
6.4 Whitenoiseincontinuous time 151
6.5 Cross-covariance andcross-spectrum 153
6.5.1 Definitionsandgeneralproperties 153
6.5.2 Input-output relations 156
6.5.3 Interpretation ofthecross-spectral density 158
viii CONTENTS
Exercises 161
7 AR-,MA-,andARMA-models 165
7.1 Introduction 165
7.2 Auto-regression andmovingaverage 166
7.2.1 Auto-regressive process, AR(p) 166
7.2.2 Movingaverage,MA(q) 174
7.2.3 Mixedmodel,ARMA(p,q) 175
7.3 EstimationofAR-parameters 180
7.4 Prediction inAR-andARMA-models 182
7.4.1 Prediction ofAR-processes 182
7.4.2 Prediction ofARMA-processes 183
7.4.3 Thecovariance function foranARMA-process 186
7.4.4 Theorthogonality principle 187
7.5 Asimplenon-linear model–theGARCH-process 189
7.6 MonteCarlosimulation ofARMAprocesses 192
Exercises 192
8 Linearfilters–applications 197
8.1 Introduction 197
8.2 Differential equations withrandom input 197
8.2.1 Lineardifferential equations withrandom input 197
8.2.2 Differential equations drivenbywhitenoise 201
8.3 Theenvelope 205
8.3.1 TheHilberttransform 206
8.3.2 Acomplexrepresentation 208
8.3.3 Theenvelope ofanarrow-banded process 209
8.4 Matchedfilter 212
8.4.1 Digitalcommunication 213
8.4.2 Astatistical decisionproblem 214
8.4.3 Theoptimalmatchedfilter 216
8.5 Wienerfilter 220
8.5.1 Reconstruction ofastochastic process 221
8.5.2 Optimalcleaning filter;frequency formulation 222
3
5 8.5.3 Generalsolution; discretetimeformulation 225
7
5
3 8.6 Kalmanfilter 226
3
5
43 8.6.1 Theprocess andmeasurementmodels 226
1
48| 8.6.2 Updatingaconditional normaldistribution 227
7
2
1 8.6.3 TheKalmanfilter 228
5
n|
ut 8.7 Anexamplefromstructural dynamics 229
CONTENTS ix
8.7.1 Aone-wheeled car 230
8.7.2 Astochastic roadmodel 231
8.7.3 Optimization 232
8.8 MonteCarlosimulation incontinuous time 233
Exercises 233
9 Frequencyanalysisandspectralestimation 237
9.1 Introduction 237
9.2 Theperiodogram 238
9.2.1 Definition 238
9.2.2 Expectedvalue 240
9.2.3 “Pre-whitening” 242
9.2.4 Variance 243
9.3 Thediscrete Fouriertransform andtheFFT 244
9.4 Biasreduction –datawindowing 247
9.5 Reduction ofvariance 251
9.5.1 Lagwindowing 252
9.5.2 Averagingofspectra 253
9.5.3 Multiplewindows 255
9.5.4 Estimationofcross-andcoherence spectrum 256
Exercises 259
A Someprobabilityandstatistics 261
A.1 Probabilities, random variablesandtheirdistribution 261
A.2 Multidimensional normaldistribution 263
A.2.1 Conditional normaldistribution 267
A.2.2 Complexnormalvariables 270
A.3 Convergence inquadratic mean 271
A.4 Somestatistical theory 273
B Deltafunctions,generalized functions,andStieltjesintegrals 275
B.1 Introduction 275
B.2 Thedeltafunction 275
B.3 Formalrulesfordensity functions 277
B.4 Distibutions ofmixedtype 278
B.5 Generalized functions 279
B.6 Stieltjesintegrals 280
B.6.1 Definition 280
B.6.2 Somesimpleproperties 281
x CONTENTS
C Kolmogorov’s existencetheorem 285
C.1 Theprobability axioms 285
C.2 Randomvariables 286
C.3 Stochastic processes 286
D Covariance/spectral densitypairs 289
E Ahistoricalbackground 291
References 299