Stochastic Approximation and Its Applications

375 Pages·2003·6.38 MB·English

by  Han-Fu Chen (auth.)

Stochastic Approximation and Its Applications
Nonconvex Optimization and Its Applications
Volume 64
Managing Editor:
Panos Pardalos
Advisory Board:
J.R. Birge
Northwestern University, U.S.A.
Ding-Zhu Du
University of Minnesota, U.S.A.
C. A. Floudas
Princeton University, U.S.A.
J. Mockus
Lithuanian Academy of Sciences, Lithuania
H. D. Sherali
Virginia Polytechnic Institute and State University, U.S.A.
G. Stavroulakis
Technical University Braunschweig, Germany
The titles published in this series are listed at the end of this volume.
Stochastic Approximation
and Its Applications
by
Han-Fu Chen
Institute of Systems Science,
Academy of Mathematics and System Science,
Chinese Academy of Sciences,
Beijing, P.R. China
KLUWER ACADEMIC PUBLISHERS
NEW YORK,BOSTON, DORDRECHT, LONDON, MOSCOW
eBookISBN: 0-306-48166-9
Print ISBN: 1-4020-0806-6
©2003 Kluwer Academic Publishers
NewYork, Boston, Dordrecht, London, Moscow
Print ©2002 Kluwer Academic Publishers
Dordrecht
All rights reserved
No part of this eBook maybe reproducedor transmitted inanyform or byanymeans,electronic,
mechanical, recording, or otherwise, without written consent from the Publisher
Created in the United States of America
Visit Kluwer Online at: http://kluweronline.com
and Kluwer's eBookstore at: http://ebooks.kluweronline.com
Contents
Preface ix
Acknowledgments xv
1. ROBBINS-MONRO ALGORITHM 1
1.1 Finding Zeros of a Function. 2
1.2 Probabilistic Method 4
1.3 ODE Method 10
1.4 Truncated RM Algorithm and TS Method 16
1.5 Weak Convergence Method 21
1.6 Notes and References 23
2. STOCHASTIC APPROXIMATION ALGORITHMS WITH
EXPANDING TRUNCATIONS 25
2.1 Motivation 26
2.2 General Convergence Theorems by TS Method 28
2.3 Convergence Under State-Independent Conditions 41
2.4 Necessity of Noise Condition 45
2.5 Non-Additive Noise 49
2.6 Connection Between Trajectory Convergence and Property
of Limit Points 57
2.7 Robustness of StochasticApproximation Algorithms 67
2.8 Dynamic Stochastic Approximation 82
2.9 Notes and References 93
3. ASYMPTOTIC PROPERTIES OF STOCHASTIC
APPROXIMATION ALGORITHMS 95
3.1 Convergence Rate: Nondegenerate Case 96
3.2 Convergence Rate: Degenerate Case 103
3.3 Asymptotic Normality 113
v
vi STOCHASTIC APPROXIMATION AND ITS APPLICATIONS
3.4 AsymptoticEfficiency 130
3.5 Notes and References 149
4. OPTIMIZATION BY STOCHASTIC APPROXIMATION 151
4.1 Kiefer-Wolfowitz Algorithm with Randomized Differences 153
4.2 Asymptotic Properties of KW Algorithm 166
4.3 Global Optimization 172
4.4 Asymptotic Behavior of Global Optimization Algorithm 194
4.5 Application to Model Reduction 210
4.6 Notes and References 218
5. APPLICATION TO SIGNAL PROCESSING 219
5.1 Recursive Blind Identification 220
5.2 Principal Component Analysis 238
5.3 Recursive Blind Identification by PCA 246
5.4 Constrained Adaptive Filtering 265
5.5 Adaptive Filtering by Sign Algorithms 273
5.6 Asynchronous Stochastic Approximation 278
5.7 Notes and References 288
6. APPLICATION TO SYSTEMS AND CONTROL 289
6.1 Application to Identification and Adaptive Control 290
6.2 Application to Adaptive Stabilization 305
6.3 Application to Pole Assignment for Systems with Unknown
Coefficients 316
6.4 Application to Adaptive Regulation 321
6.5 Notes and References 327
Appendices 329
329
A.1 Probability Space 329
A.2 Random Variable and Distribution Function 330
A.3 Expectation 330
A.4 Convergence Theorems and Inequalities 331
A.5 Conditional Expectation 332
A.6 Independence 333
A.7 Ergodicity 333
335
B.1 Convergence Theorems for Martingale 335
B.2 Convergence Theorems for MDS I 339
B.3 Borel-Cantelli-Lévy Lemma 340
Contents vii
B.4 Convergence Criteria for Adapted Sequences 341
B.5 Convergence Theorems for MDS II 343
B.6 Weighted Sum of MDS 344
References 347
Index 355

Preface
Estimating unknown parameters based on observation data contain-
ing information about the parameters is ubiquitous in diverse areas of
both theory and application. For example, in system identification the
unknown system coefficients are estimated on the basis of input-output
data of the control system; in adaptive control systems the adaptive
control gain should be defined based on observation data in such a way
that the gain asymptotically tends to the optimal one; in blind chan-
nel identification the channel coefficients are estimated using the output
data obtained at the receiver; in signal processing the optimal weighting
matrix is estimated on the basis of observations; in pattern classifica-
tion the parameters specifying the partition hyperplane are searched by
learning, and more examples may be added to this list.
All these parameter estimation problems can be transformed to a
root-seeking problem for an unknown function. To see this, let de-
note the observation at time i.e., the information available about the
unknown parameters at time It can be assumed that the parameter
under estimation denoted by is a root of some unknown function
This is not a restriction, because, for example, may
serve as such a function. Let be the estimate for at time Then
the available information at time can formally be written as
where
Therefore, by considering as an observation on at with
observation error the problem has been reduced to seeking the
root of based on
It is clear that for each problem to specify is of crucial importance.
The parameter estimation problem is possible to be solved only if
ix