Statistical Signal Processing: Modelling and Estimation

333 Pages·2002·10.402 MB·English

by  Dr Thierry Chonavel PhD (auth.)

Advanced Textbooks in Control and Signal Processing
Springer-Verlag London Ltd.
Series Editors
Professor Michael J. Grimble, Professor of Industrial Systems and Director
Professor Michael A. Johnson, Professor of Control Systems and Deputy Director
Industrial Control Centre, Department of Electronic and Electrical Engineering,
University of Strathclyde, Graham Hills Building, 50 George Street, Glasgow GIIQE, U.K.
Other titles published in this se ries:
Genetic Algorithms: Concepts and Designs
K.F. Man, K.S. Tang and S. Kwong
Model Predictive Control
E. F. Camacho and C. Bordons
Introduction to Optimal Estimation
E.W. Kamen and J. Su
Discrete-Time Signal Processing
D. Williamson
Neural Networks for Modelling and Control ofD ynamic Systems
M. N0rgaard, O. Ravn, N.K. Poulsen and 1.K. Hansen
Modelling and Control ofR obot Manipulators (2nd Edition)
1. Sciavicco and B. Siciliano
Fault Detection and Diagnosis in Industrial Systems
1.H. Chiang, E.1. Russell and R.D. Braatz
Soft Computing
1. Fortuna, G. Rizzotto, M. Lavorgna, G. Nunnari, M.G. Xibilia and R. Caponetto
Parallel Processing for Real-time Signal Processing and Control
M.O. Tokhi, M.A. Hossain and M.H. Shaheed
Publication due January 2003
T. Chonavel
Translated by Janet Ormrod
Statistical Signal
Processing
Modelling and Estimation
, Springer
Dr Thierry Chonavel, PhD
ENST de Bretagne,
Technopöle de Brest Iroise 29285, Brest Cedex, France
British Library Cataloguing in Publication Data
Chonavel, T.
Statistical signal processing : modelling and estimation. -
(Advanced textbooks in control and signal processing)
1.Signal processing -Mathematical models 2.Signal
processing -Statistical methods
I.Title
621.3'822
ISBN 978-1-85233-385-0
Library of Congress Cataloging-in-Publication Data
Chonavel, T. (Thierry), 1963-
Statistical signal processing : modelling and estimation 1 T. Chonave!.
p. cm. --(Advanced textbooks in control and signal processing)
Inc1udes bibliographical references and indeL
ISBN 978-1-85233-385-0 ISBN 978-1-4471-0139-0 (eBook)
DOI 10.1007/978-1-4471-0139-0
1. Signal processing--Mathematics. 2. Statistics. I. Title. 11. Series.
TK5102.9 .C4835 2001
621.382'2--dc21 2001020769
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as
permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced,
stored or transmitted, in any form or by any means, with the prior permission in writing ofthe publishers,
or in the case of reprographie reproduction in accordance with the terms of licences issued by the
Copyright Licensing Agency. Enquiries conceming reproduction outside those terms should be sent to
the publishers.
Additional material to this book can be downloaded from http://extras.springer.com.
ISSN 1439-2232
ISBN 978-1-85233-385-0
http://www.springer.co.uk
C Springer-Verlag London 2002
Originally published by Springer-Verlag London Limited in 2002
Tbe use of registered names, trademarks ete. in this publication does not imply, even in the absence of a
specific statement, that such names are exempt from the relevant laws and regulations and therefore free
for general use.
Tbe publisher makes no representation, express or implied. with regard to the accuracy of the
information contained in this book and cannot accept any legal responsibility or liability for any errors or
omissions that may be made.
Typesetting: Blectronic text files prepared by author
69/3830-543210 Printed on acid-free paper SPIN 10783163
Series Editors' Foreword
The topics ofcontrol engineering and signal processing continue to flourish and
develop. Incornmonwith general scientific investigation, new ideas, concepts and
interpretations emerge quite spontaneously and these are then discussed, used,
discarded or subsumed into the prevailing subject paradigm. Sometimes these
innovative concepts coalesce into a new sub-discipline within the broad subject
tapestryofcontrol and signal processing. This preliminarybattlebetween old and
new usually takes placeatconferences, throughthe Internet and in thejournalsof
the discipline. Aftera little more maturityhas been acquired bythe newconcepts
thenarchivalpublicationasascientificorengineeringmonographmayoccur.
Anewconceptincontrolandsignalprocessingis knownto have arrived when
sufficientmaterial hasdevelopedfor the topic to betaughtasaspecialisedtutorial
workshop oras a course to undergraduates, graduates or industrial engineers. The
Advanced Textbooks in Control and Signal Processing series is designed as a
vehicle for the systematic presentation ofcourse material for both popular and
innovative topics in the discipline. It is hoped that prospective authors will
welcome the opportunity to publish a structured presentation of either existing
subject areas or some of the newer emerging control and signal processing
technologies.
In communications, control engineering and related disciplines measured
signals are almost always corrupted by noise or subject to limited random
distortion. This means that ifanexperimentis repeatedthe samemeasured signals
will not be obtained, and there is an uncertainty present in the signal. The main
tools used to analyse and understand the mechanisms operating to produce this
uncertainty are those ofstatistical signal processing. However, it is only over the
lasttwentyyearsorsothatawell-defineddisciplineofstatisticalsignalprocessing
has really emerged. The tools of this new discipline are built on foundations
comprising probability theory, statistics, random processes and measure theory.
Thierry Chonavel has written a book with strong roots in the basics of the
discipline but which deals with fundamental problems in statistical signal
processing. These important core problems ofpractical signal processing include
Kalman and Wiener filtering, prediction, spectral identification, and non
parametric andparametric estimation. The approach to these standardproblems is
mathematical and rigorous: the reader is first led through chapters on random
processes, power spectra and spectral representations before the core chapters of
the book are reached. The closing chapters of the book generalise some ofthe
vi SeriesEditors'Foreword
methods and presents a broadening of the material, for example, higher-order
statisticalprocessesandadaptiveestimation.
The mathematical approach ofthe text yields benefits in clarity and precision
in definitions. Graduate students on Masters courses or studying for doctoral
qualifications will find this text invaluable for communications, signalprocessing,
control engineering courses and research. Engineers, and research workers who
use statistical signal processing concepts are likely to find the book a good tutor
for specificquestionsandanup-to-datereferencesource.
MJ. GrimbleandM.A. Johnson
IndustrialControlCentre
Glasgow, Scotland,U.K.
January2002
Contents
List of Notation and Symbols xv
List of Abbreviations XIX
1. Introduction. ............................................. 1
1.1 Foreword.............................................. 1
1.2 Motivation for a Book in Signal Processing 1
1.3 A Few Classical Problems in Statistical Signal Processing. ... 2
1.4 Why This Book? ....................................... 4
1.5 Book Contents ......................................... 6
1.6 Acknowledgment....................................... 8
2. Random Processes. ....................................... 9
2.1 Basic Definitions. .......................... ............ 9
2.1.1 Definition....................................... 9
2.1.2 Probability Distribution ofa Random Process. ....... 10
2.1.3 Kolmogorov's Consistency Theorem 10
2.2 Second Order Processes 11
2.3 Classical Operations in £2([1,A,dP) ...................... 12
2.3.1 Mean Square Convergence. ........................ 13
2.3.2 Mean Square Continuity 14
2.3.3 Mean Square Derivative. .......................... 14
2.3.4 Mean Square Integration. ......................... 15
2.4 Stationarity and Ergodicity .............................. 17
2.4.1 Stationary Processes , 17
2.4.2 Ergodic Processes 17
Exercises 20
3. Power Spectrum ofWSS Processes ....................... 23
3.1 Spectra with a Density: Power Spectral Density (PSD) ...... 23
3.2 Spectral Measure. ...................................... 24
Exercises 26
VIII Contents
4. Spectral Representation ofWSS Processes. ............... 31
4.1 Stochastic Measures and Stochastic Integrals. .............. 31
4.1.1 Definition....................................... 31
4.1.2 Measure J.Lt Associated with Z. .... ............ .... 31
4.1.3 Principle ofthe Method. .......................... 32
4.1.4 Construction ofStochastic Integrals fJW.¢J(f)dZ(f) .... 32
4.2 Kolmogorov's Isomorphism. ............................. 34
4.3 Spectral Representation ................................. 34
4.4 Sampling.............................................. 35
Exercises 36
5. Filtering ofWSS Processes ............................... 41
5.1 Elements ofDeterministic Signal Filtering. ................ 41
5.2 Filtering ofWSS Processes 42
5.3 Comparison ofthe Deterministic with the Stochastic Case. " 44
5.4 Examples.............................................. 45
5.4.1 Bandpass Filters. ................................ 45
5.4.2 Differentiators................................... 45
5.4.3 Linear Partial Differential Equations. ............... 46
Exercises 47
6. Important Particular Processes. .......................... 51
6.1 Gaussian Processes. .................................... 51
6.2 Poisson Processes. ...................................... 53
6.3 White Noise " 54
6.3.1 Generalised Processes. ............................ 55
6.3.2 Brownian Motion. ................................ 59
6.4 Cyclostationary Processes 60
6.5 Circular Processes. ..................................... 62
6.6 Multivariate Processes 64
Exercises 65
7. Non-linear Transforms of Processes " 69
7.1 Square Law Detector and Hard Limiter 69
7.1.1 Square Law Detector 69
7.1.2 Hard Limiter " 70
7.1.3 Bussgang's Theorem. ............................. 71
7.2 Amplitude Modulation. ................................. 72
7.2.1 Phase and Quadrature Modulation. ................ 73
7.2.2 Analytic Representation and SSB Modulation. ....... 73
7.2.3 Rice's Representation. ............................ 74
7.2.4 Demodulation in the Presence ofNoise. ............. 75
Exercises 76
Contents IX
8. Linear Prediction of WSS Processes .. .................... 79
8.1 Definitions............................................. 79
8.1.1 Conditional Expectation and Linear Prediction. ...... 79
8.1.2 Innovation Process 80
8.1.3 Regular and Singular Processes 81
8.1.4 Examples....................................... 81
8.2 Wold's Decomposition Theorem. ......................... 82
8.3 Finite Past Linear Prediction 84
8.4 Causal Factorisation ofa PSD 87
8.4.1 Causal Factorisation .............................. 87
8.4.2 Minimum-phase Causal Factorisation. .............. 88
8.5 The Continuous Case. .................................. 91
Exercises 91
9. Particular Filtering Techniques. .......................... 95
9.1 Wiener Filter. ......... .................... .... ........ 95
9.2 Kalman Filter. ......................................... 97
9.3 Generalisation ofKalman Filter 101
9.4 Matched Filter 105
Exercises 107
10. Rational Spectral Densities 111
10.1 Difference Equations and Rational Spectral Densities 111
10.2 Spectral Factorisation ofRational Spectra 113
10.3 State Space Representation of ARMA Models 114
Exercises 115
11. Spectral Identification of WSS Processes 119
11.1 Spectral Identification ofARMA Processes 119
11.1.1 Identification ofthe AR Part 119
11.1.2 Identification of the MA Part 120
11.1.3 Identification of the State Space Representation 120
11.2 The Trigonometric Moment Problem 122
11.2.1 Condition of Existence ofSolutions 122
11.2.2 Orthogonal Polynomials on the Unit Circle 123
11.2.3 Particular Classes ofHolomorphic Functions 126
11.2.4 General Solution to the Problem 127
11.2.5 Maximum Entropy Spectrum 128
11.3 Line Spectra 129
11.4 Lattice Filters 132
Exercises 134
x Contents
12. Non-parametric Spectral Estimation 139
12.1 Definitions 139
12.2 Elements ofNon-parametric Estimation 140
12.2.1 Independent Data Sequences 140
12.2.2 Ergodic Processes 142
12.3 Empirical Mean and Autocovariances 142
12.3.1 Linear Processes 142
12.3.2 Empirical Mean 144
12.3.3 Empirical Autocovariance Coefficients 145
12.4 Empirical PSD: the Periodogram 147
12.4.1 The White Noise Case 148
12.4.2 The Periodogram of Linear Processes 149
12.4.3 The Case of Line Spectra 150
12.5 Smoothed Periodogram 152
12.5.1 Integrated Periodogram 152
12.5.2 Smoothed Periodogram 153
12.5.3 Averaged Periodogram 156
Exercises 156
13. Parametric Spectral Estimation 159
13.1 Introduction 159
13.2 Elements ofParametric Spectral Estimation 160
13.2.1 Cramer-Rao Lower Bound (CRLB) 160
13.2.2 Maximum Likelihood Estimators 161
13.2.3 MinimumVariance Linear Unbiased Estimators 161
13.2.4 Least Squares Estimators 161
13.3 Estimation ofthe Autocovariance Coefficients 162
13.4 Spectrum Estimation ofARMA Models: Mean Square Criteria163
13.4.1 Estimation ofRational Spectra 163
13.4.2 Rational Filter Synthesis 166
13.5 Asymptotic Log-likelihood ofGaussian Processes 167
13.5.1 Gaussian Log-likelihood 168
13.5.2 Asymptotic Behaviour ofthe Log-likelihood 168
13.6 Approximate Maximum Likelihood Estimation 172
13.6.1 Principle ofthe Method 172
13.6.2 Convergence ofthe Estimators 174
13.7 Maximum Likelihood Estimation ofARMA Models 176
13.7.1 The General Case 176
13.7.2 AR Models 177
13.7.3 AR Models Parameterised by Reflection Coefficients .. 179
Exercises 180