Spectral analysis and its applications PDF
Preview Spectral analysis and its applications
HOLDEN-DAY SERIES IN TIME SERIES ANALYSIS Gwilym M. Jenkins and Emanuel Parzen, Editors SPECTRAL ANALYSIS and its applications GWILYM M. JENKINS University of Lancaster, U.K. and DONALD G. WATTS University of IVisconsin, U.S.A. RH HOLDEN-DAY San Francisco, Cambridge, London, Amsterdam < Copyright 1968 by Holden-Day, Inc., 500 Sansome Street, San Francisco, California. All rights reserved. No part of this book may be reproduced, by mimeograph, or any other means, without the permission in writing of the publisher. Library of Congress Catalog Card Number: 67-13840. Printed in the United States of America. Preface Time series analysis is now widely used in many branches of engineering, the physical sciences and economics. One important aspect of time series analysis is spectral analysis, which is concerned with the splitting up of time series into different frequency components. Applications of spectral analysis cover a wide range of problems, for example, the effect of wave oscillations on the vibration of ships and the influence of disturbances or noise on the per formance of electrical guidance systems and chemical reactors. This book has been designed primarily for post-graduate engineers, since most of the applications of spectral analysis have, in fact, been made by engineers and physicists. One of the difficulties faced by users or potential users of spectral analysis is that most of the theory has been developed by statisticians during the last fifteen years. Unfortunately, much of this litera ture is difficult to read. Hence it is felt that a book directed mainly toward engineers is long overdue. However, we hope this book will appeal to a much wider audience, including mathematicians, statisticians, economists, physicists and biologists. One of the difficulties we have encountered in writing this book is that, whereas spectral analysis involves the use of sophisticated statistical tech niques, many engineers lack knowledge of elementary statistics. This is true even of electrical engineers, some of whom possess considerable knowledge of probability theory. For example, the Wiener theory of prediction and control shows that an optimum filter or control system can be designed if various spectra associated with the signal and noise in the system are known. However, little attention is paid in books on control theory to the very important practical question of how to estimate these spectra from finite lengths of record. It is with such problems that we shall be concerned in this book. To provide a gradual introduction to time series estimation problems, we have been forced in the earlier chapters to deal with elementary statistical problems. This may distract mathematical and statistical readers, but in view of our experience in expounding these ideas to engineers, we feel that VI Preface a self-contained introduction, which includes most of the statistical ideas needed later on in the book, is necessary. Those readers who are familiar with the material of Chapters 2, 3 and 4 can, of course, start at Chapter 5. Chapter 1 is devoted to a brief outline of the territory covered and to a description of the kind of problems which can be solved using spectral analysis. Chapter 2 deals with the important ideas of Fourier analysis and is basic to what follows. Most of this is well known to engineers but is brought together here in a form oriented toward spectral analysis. In Chapter 3 we introduce some basic notions in probability theory which are fundamental to subsequent chapters. Chapter 4 consists of an introduction to many important ideas in statistical inference and includes a discussion of the sampling distribution approach to estimation theory, the theory of least squares and a brief reference to likelihood inference. Not all of this material is necessary for an understanding of the spectral techniques discussed later in the book, and engineering readers may wish to omit the latter part of this chapter at first reading. The most relevant parts of this chapter, as far as spectral analysis is concerned, are the sections on the sampling distribution approach to estima tion theory and the theory of least squares. The latter is one of the most important weapons in the statistician’s armory, and in our experience it is widely misunderstood among engineers. Chapter 5 contains some of the simpler ideas in the theory of stochastic processes, for example, stationarity, the autocorrelation function and moving average-autoregressive processes. Methods for estimating auto correlation functions and parameters in linear processes are described and illustrated by examples. In Chapter 6, the ideas of Fourier analysis and stochastic processes are brought together to provide a description of a stationary stochastic process by means of its spectrum. It is shown how Fourier methods need to be tailored to estimate the spectrum from finite lengths of record. The sampling properties of spectral estimators are then derived, and the important notion of smoothing of these estimators is intro duced. Chapter 7 contains many simulated and practical examples of spectral estimation and gives a systematic method, called window closing, for de ciding the amount of smoothing required. In Chapter 8 the ideas of Chapters 5-7 are extended to pairs of time series, leading to the definition of the cross correlation function, the cross spectrum and the squared coherency spectrum. Chapter 9 is devoted to estimating the cross spectrum and the notion of aligning two series. Cross spectral analysis is applied in Chapter 10 to estimating the frequency response function of a linear system. Finally, we consider in Chapter 11 the spectral analysis of a vector of several time series and the estimation of the frequency response matrix of a linear system. This book has been written at a time when there is much active work in this area and when much experience has still to be gained in the application Preface vn of spectral methods. Nevertheless, it is felt that enough has been achieved already to warrant an attempt. It is hoped that the book will provide applied scientists and engineers with a comprehensive and useful handbook for the application of spectral analysis to practical time series problems, as well as proving useful as a post-graduate textbook. We are greatly indebted to Professor K. N. Stanton of the School of Engineering, Purdue University, for making available the power-station data used in later chapters and to Professor H. J. Wertz of the University of Wisconsin for helpful suggestions regarding computer programs. We are very grateful to Mr. A. J. A. MacCormick of the Statistics Department of the University of Wisconsin and also Mr. M. J. McClellan of the U.S. Army Mathematics Research Center, University of Wisconsin, for writing and running some of the computer programs. We also thank Mr. MacCormick, and Mr. A. S. Alavi of the University of Lancaster, for checking through the manuscript. Lancaster, U.K. Gwilym M. Jenkins Madison, PTis., U.S.A. Donald G. Watts Contents Preface . v Chapter 1. AIMS AND MEANS IN TIME SERIES ANALYSIS 1.1 Time series and stochastic processes ... 1 1.1.1 Deterministic and non-deterministic functions 1 1.1.2 Stochastic processes.. 2 1.1.3 Experimental and non-experimental data 3 1.2 Time-domain and frequency-domain descriptions of time series . 3 1.2.1 Stationarity....................... 4 1.2.2 The autocovariance function 4 1.2.3 The spectrum .... 6 1.2.4 Parametric time series models 9 1.3 Objectives of time series analysis . 10 1.3.1 Model building ... 10 1.3.2 Uses of time series models 12 1.3.3 Frequency response studies 14 1.4 Scope of the present book 14 Chapter 2. FOURIER ANALYSIS 2.1 Introduction.......................................................................... 16 2.1.1 The role of Fourier analysis in applied mathematics and engineering ... 16 2.1.2 Finite Fourier series 17 2.1.3 Fourier series 23 2.1.4 Fourier integrals............................ 24 2.2 Fourier transforms and their properties . 25 2.2.1 Well-behaved functions. 25 2.2.2 Generalized functions .... 28 2.2.3 Fourier series as Fourier transforms . 33 IX Contents 2.3 Linear systems and convolution 34 2.3.1 Linear differential equations 34 2.3.2 Step and impulse functions 36 2.3.3 Frequency response functions 39 2.3.4 Response to an arbitrary input 44 2.3.5 Linear difference equations 46 2.4 Applications to time series analysis 48 2.4.1 Finite-length records 48 2.4.2 Time sampling and aliasing 51 Appendix A2.1 Operational properties of Fourier transforms 54 Chapter 3. PROBABILITY THEORY 3.1 Frequency and probability distributions . 57 3.1.1 Discrete random variables and distributions. 57 3.1.2 Continuous random variables and distributions. 61 3.1.3 Estimation of probability density functions . 64 3.1.4 Bivariate distributions . 64 3.1.5 Multivariate distributions . 67 3.2 Moments of random variables 68 3.2.1 Univariate moments 68 3.2.2 Multivariate moments...................... 71 3.2.3 Moments of linear functions of random variables . 72 3.2.4 The correlation coefficient............................................. 73 3.2.5 Moments of non-linear functions of random variables 75 3.3 Sampling distributions............................................. 77 3.3.1 Sampling distribution of the mean, variance known 78 3.3.2 Sampling distribution of the variance .... 79 3.3.3 Sampling distribution of the mean, variance unknown . 83 3.3.4 Sampling distribution of the ratio of two variances 85 3.3.5 Two properties of the chi-squared distribution 87 Appendix A3.1 Moments of linear functions of random variables 88 Chapter 4. INTRODUCTION TO STATISTICAL INFERENCE 4.1 Historical development of statistical inference . . . . 91 4.2 The sampling distribution approach to statistical inference 92 4.2.1 The basic method 92 4.2.2 Confidence intervals 95 4.2.3 Properties of estimators 97 4.2.4 Maximum likelihood estimators 99 4.2.5 Tests of significance 103 4.3 Least squares estimation . 105 4.3.1 The principle of least squares . 105 4.3.2 Confidence intervals for one parameter 109 Contents XI 4.3.3 Confidence regions for many parameters. 112 4.3.4 Orthogonality 114 4.4 Likelihood inference 116 4.4.1 The basic method .... 116 4.4.2 Properties of likelihood functions 117 4.4.3 Examples of likelihood functions . 119 4.4.4 Least squares and likelihood estimation . 120 4.4.5 Methods of summarizing likelihood functions 122 4.4.6 Estimation of a Normal mean and variance 127 4.5 Summary ... .... 130 Appendix A4.1 Linear least squares theory 132 Chapter 5. INTRODUCTION TO TIME SERIES ANALYSIS 5.1 Stationary and non-stationary stochastic processes . 140 5.1 Definition and classification of time series 140 5.1.2 Description of a stochastic process 143 5.1.3 Stationarity and the autocovariance function 147 5.1.4 Classification of time series occurring in practice 150 5.1.5 Minimum mean square error system analysis 152 5.2 The autocorrelation and autocovariance functions . 155 5.2.1 Basic properties............................................. 155 5.2.2 The linear process and its autocovariance function . 157 5.2.3 Finite moving average processes . 161 5.2.4 Autoregressive processes.................................. 162 5.2.5 General autoregressive-moving average processes . 166 5.2.6 Interpretation of the autocorrelation function 170 5.3 Estimation of autocovariance functions . 171 5.3.1 Least squares system analysis . 172 5.3.2 Sample autocovariance functions....................... 174 5.3.3 Properties of autocovariance function estimators 174 5.3.4 Discrete time autocovariance estimates....................... 180 5.3.5 Practical aspects of autocovariance function estimation 183 5.4 Estimation of the parameters of a linear process 189 5.4.1 Maximum likelihood estimation of autoregressive parameters.................................................................... 189 5.4.2 Mean likelihood estimates for autoregressive parameters . 193 5.4.3 Determining the order of the autoregressive process 197 5.4.4 Estimation of the parameters of a moving average process 200 5.4.5 Estimation of the parameters of a mixed autoregressive- moving average process .... . . 202 Appendix A5.1 The calculus of variations . 204 Appendix A5.2 Moments of linear processes . 205 Appendix A5.3 Flow chart for covariance program 207
The list of books you might like
