Statistical Estimation: Asymptotic Theory

410 Pages·1981·14.174 MB·English

by  I. A. Ibragimov, R. Z. Has’minskii (auth.)

Applied Probability Applications of
Control
Mathematics
Economics
Information and Communication
Modeling and Identification 16
Numerical Techniques
Optimization
Edited by A. V. Balakrishnan
Advisory Board E. Dynkin
G. Kallianpur
R. Radner
Applications of Mathematics
Volume 1 Deterministic and Stochastic Optimal Control
By W.H. Fleming and R. W. Rishel
1975. ix, 222p. 4 illus.
Volume 2 Methods of Numerical Mathematics
By G.!. Marchuk
1975. xii, 316p. 10 illus.
Volume 3 Applied Functional Analysis, Second Edition
By A.V. Balakrishnan
1981. xiii, 373p.
Volume 4 Stochastic Processes in Queueing Theory
By A.A. Borovkov
1976. xi, 28Op. 14 illus.
Volume 5 Statistics of Random Processes I: General Theory
By R.S. Lipster and A.N. Shiryayev
1977. x, 394p.
Volume 6 Statistics of Random Processes II: Applications
By R.S. Lipster and A.N. Shiryayev
1978. x, 339p.
Volume 7 Game Theory: Lectures for Economists and Systems Scientists
By N.N. Vorob'ev
1977. xi, 178p. 60 illus.
Volume 8 Optimal Stopping Rules
By A.N. Shiryayev
1978. x, 217p. 7 illus.
Volume 9 Gaussian Random Processes
By I.A. Ibragimov and Y.A. Rosanov
1978. x, 275p.
Volume 10 Linear Multivarlable Control: A Geometric Approach
By W.M. Wonham
1979. xi, 326p. 27 illus.
Volume 11 Brownian Motion
By T. Hida
1980. xvi, 325p. 13 illus.
Volume 12 Conjugate Direction Methods in Optimization
By M. Hestenes
1980. x, 325p. 22 illus.
Volume 13 Stochastic Filtering Theory
By G. Kallianpur
1980. xv, 316p.
Volume 14 Controlled Diffusion Processes
By N.V. Krylov
1980. xii, 308p.
Volume 15 Stochastic Storage Processes: Queues, Insurance Risk, and Dams
By N .R. Prabhu
1980. xii, l4Op.
I. A. Ibragimov
R. Z. Has'minskii
Statistical Estimation
Asymptotic Theory
Translated by Samuel Kotz
Springer Science+Business Media, LLC
I. A. Ibragimov R. Z. Has'ininskii
LOMI Fontanka 25 Doz., Institut Problem Peredaci Inf.
Leningrad, 19011 ul. Aviamotornaja 8
USSR Dorp 2, Moscow
USSR
Editor Translator
A. V. Balakrishnan Samuel Kotz
University of California Department of Management Sciences
Systems Science Department and Statistics
Los Angeles, CA 90024 University of Maryland
USA College Park, MD 20742
USA
AMS Subject Classification 62E20, 62H12, 60E05, 60E99
Library of Congress Cataloging in Publication Data
Ibragimov, Il'dar Abdulovich.
Statistical estimation—asymptotic theory.
(Applications of mathematics ; 16)
Translation of Asimptoticheskaia teoriia otsenivaniia.
Includes index.
1. Asymptotic expansions. II. Has'minskii, Rafail
Zalmanovich, joint author. II. Title. III. series.
QA355.I2613 515 80-28541
Title of the Russian Original Edition : Asimptoticheskaya
teoriya otsenivanya. Nauka, Moscow 1979.
All rights reserved.
No part of this book may be translated or reproduced in
any form without written permission from Springer-Verlag.
© 1981 by Springer Science+Business Media New York
Originally Published by Springer-Verlag New York Inc in 1981
Softeover reprint of the hardcover 1st edition 1981
9 8 7 6 5 4 3 2 1
ISBN 978-1-4899-0029-6 ISBN 978-1-4899-0027-2 (eBook)
DOI 10.1007/978-1-4899-0027-2
Contents
Basic Notation
Introduction 3
Chapter I
The Problem of Statistical Estimation 10
I The Statistical Experiment 10
2 Formulation of the Problem of Statistical Estimation 16
3 Some Examples 24
4 Consistency. Methods for Constructing Consistent Estimators 30
5 Inequalities for Probabilities of Large Deviations 41
6 Lower Bounds on the Risk Function 58
7 Regular Statistical Experiments. The Cramer-Rao Inequality 62
8 Approximating Estimators by Means of Sums of Independent Random
Variables 82
9 Asymptotic Efficiency 90
10 Two Theorems on the Asymptotic Behavior of Estimators 103
Chapter II
Local Asymptotic Normality of Families of Distributions 113
I Independent Identically Distributed Observations 113
2 Local Asymptotic Normality (LAN) 120
3 Independent Nonhomogeneous Observations 123
4 Corollaries for the Model "Signal Plus Noise" l31
5 Observations with an "Almost Smooth" Density 133
6 Multidimensional Parameter Set 139
7 Observations in Gaussian White Noise 143
8 Some Properties of Families of Distributions Admitting the LAN
Condition 147
9 Characterization of Limiting Distributions of Estimators 150
10 Anderson's Lemma 155
11 Asymptotic Efficiency under LAN Conditions 158
12 Asymptotically Minimax Risk Bound 162
13 Some Corollaries. Superefficient Estimators 169
v
VI Contents
Chapter III
Properties of Estimators in the Regular Case 173
I Maximum Likelihood Estimator 173
2 Bayesian Estimators 178
3 Independent Identically Distributed Observations 184
4 Independent Nonhomogeneous Observations 190
5 Gaussian White Noise 199
Chapter IV
Some Applications to Nonparametric Estimation 214
I A Minimax Bound on Risks 214
2 Bounds on Risks for Some Smooth Functionals 220
3 Examples of Asymptotically Efficient Estimators 229
4 Estimation of Unknown Density 232
5 Minimax Bounds on Estimators for Density 237
Chapter V
Independent Identically Distributed Observations.
Densities with Jumps 241
I Basic Assumptions 241
2 Convergence of Marginal Distributions of the Likelihood Ratio 246
3 Convergence in the Space Do 260
4 The Asymptotic Behavior of Estimators 266
5 Locally Asymptotic Exponential Statistical Experiments 276
Chapter VI
Independent Identically Distributed Observations.
Classification of Singularities 281
I Assumptions. Types of Singularities 281
2 Limiting Behavior of the Likelihood Ratio 288
3 Y.'i Processes. Singularities of the First and Third Type 297
4 Y.'i Processes. Singularities of the Second Type 303
5 Proofs of Theorems 2.1-2.3 309
6 Properties of Estimators 312
Chapter VII
Several Estimation Problems in a Gaussian White Noise 321
I Frequency Modulation 321
2 Estimation of Parameters of Discontinuous Signals 329
3 Calculation of Efficiency of Maximum Likelihood Estimators 338
4 Nonparameteric Estimation of an Unknown Signal 345
5 Lower Bounds on Nonparametric Estimators 354
Contents vii
Appendix I
Some Limit Theorems of Probability Theory 363
I Convergence of Random Variables and Distributions in Rk 363
2 Some Limit Theorems for Sums of Independent Random Variables 366
3 Weak Convergence on Function Spaces 369
4 Conditions for the Density of Families of Distributions in C(F) and
CO(Rk) and Criteria for Uniform Convergence 371
5 A Limit Theorem for Integrals of Random Functions 380
Appendix II
Stochastic Integrals and Absolute Continuity of Measures 382
I Stochastic Integrals over b(t) 382
2 Some Definitions and Theorems of Measure Theory 384
3 Stochastic Integrals over Orthogonal Random Measure 387
Remarks 389
Bibliography 395
Index 401
Basic Notation
In this book the ordinary double enumeration of theorems and formulas
separately for each chapter is used. When referring (in a given chapter) to
theorems, subsections, or formulas appearing in another chapter, the number
ofthis chapter is indicated first. When a reference to a theorem and a formula
presented in Appendices I or II is given, the triple enumeration is used; for
example, (l.A.12) indicates a reference to formula 12 in Appendix I.
The symbol 0 indicates the end of a proof.
e.
9'(e) denotes the distribution law of a random element
e
.2'( IP ) denotes the distribution law of a random element with respect
to P.
Yea, R) denotes the Gaussian distribution with mean a and covariance
matrix R.
Rk denotes a k-dimensional Euclidean space.
J denotes the unit matrix.
(.,.) denotes the scalar product.
IAI = sUPI).I=1 I( AA, A)I·
xc-) denotes the indicator function of a set.
Ae denotes the closure of set A (usually in the Euclidean norm).
K denotes a compact set (usually in a parameter set 9).
A denotes the set complementary to A (or the events complementary
to A).
Gothic letters ~, ~, ij are used to denote a-algebras of sets; semi-bold
letters are used to denote classes of functions.
We present below the notation for classes often encountered in this book.
C(A) denotes the space of continuous functions on A with a uniform
metric.
2 Basic Notation
Lp(A) denotes the space offunctions integrable in absolute value together
with their p-th power with respect to the Lebesgue measure (on a set A).
Lp(Jl), Lp(Jl, A) denotes the space of functions absolutely integrable
together with their p-th power with respect to measure Jl (on a set A).
Classes W, W', Wp, We,ex, W~,ex are defined on page 19; the class W is
defined on p. 179.
Classes D, Do are defined at the very beginning of Section 5.3, the class
1:(1, L) on page 233 and the classes l:.(P, L), l:.(P) on page 235. Classes of
functions l:.(P, p, L), l:.'(P, 2, L), l:.(P, 00, L) are defined in Chapter VII on
pages 352, 349, 347, respectively.
Letters b, B, c with indices or without denote positive constants, possibly
different ones even within the course of a single argument; the symbol B(L)
denotes a positive function on L, whose growth is at most polynomial in L.
a x b means that 0 S lim alb s lim alb < 00.
AT denotes the transpose of matrix A.
Whenever the limits of integration are not indicated, the integration is
carried over the whole space (usually over X, X" or Rk).