Robust Statistics: The Approach Based on Influence Functions

526 Pages·2005·9.47 MB·English

by  Frank R. Hampel

Robust Statistics
Robust Statist ics
The Approach Based
on Influence Functions
FRANK R. HAMPEL
ETH, Zurich, Switzerland
ELVEZIO M. RONCHETTI
Princeton University, Princeton, New Jersey
PETER J. ROUSSEEUW
Derft University of Technology, Derft, The Netherlanh
WERNER A. STAHEL
ETH, Zurich, Switzerland
John Wiley & Sons
New York Chichester Brisbane Toronto Singapore
Copyright @ 1986 by John Wiley & Sons, Inc.
All rights rcscrvcd. Published simultaneously in Canada.
Reproduction or translation of any part of this work
beyond that permitted by Section 107 or 108 of the
1976 United States Copyright Act without the permission
of the copyright owner is unlawful. Requests for
permission or further information should be addressed to
the Permissions Department, John Wiley & Sons, Inc.
Lihrary oj Congress Cataloging m Publication Data.
Main entry under title:
Robust statistics.
(Wiley series in probability and mathematical
statistics. probability and mathematical statistics,
ISSN 0271-6356)
Includcs index.
1. Robust statistics. 1. Hampcl, Frank R.,
1941- . 11. Scri~~.
QA276.R618 1985 519.5 R5-9428
ISBN 0-471-73577-9
10 9 8 7 6 5 4 3
To our families and friends
Non omnia possumus omnes
Preface
Statistics is the art and science of extracting useful information from
empirical data. An effective way for conveying the information is to use
parametric stochastic models. After some models had been used for more
than two centuries, R. A. Fisher multiplied the number of useful models and
derived statistical procedures based on them in the 1920s. His work laid the
foundation for what is the most widely used statistical approach in today’s
sciences.
This “classical approach” is founded on stringent stochastic models, and
before long it was noticed that the real world does not behave as nicely as
described by their assumptions. In addition, the good performance and the
valid application of the procedures require strict adherence to the assump-
tions. Consequently, nonparametric statistics emerged as a field of research,
and some of its methods, such as the Wilcoxon test, became widely popular
in applications. The basic principle was to make as few assumptions about
the data as possible and still get the answer to a specific question like “Is
there a difference?” While some problems of this kind did find very
satisfactory solutions, parametric models continued to play an outstanding
role because of their capacity to describe the information contained in a
data set more completely, and because they are useful in a wider range of
applications, especially in more complex situations.
Robust statistics combines the virtues of both approaches. Parametric
models are used as vehicles of information, and procedures that do not
depend critically on the assumptions inherent in these models are imple-
mented.
Most applied statisticians have avoided the worst pitfalls of relying on
untenable assumptions. Almost unavoidably, they have identified grossly
aberrant observations and have corrected or discarded them before analyz-
ing the data by classical statistical procedures. Formal tests for such outliers
have been devised, and the growing field of diagnostics, both formal and
ix
X PREFACE
informal, is based on this approach. While the combination of diagnosis,
corrective action, and classical treatment is a robust procedure, there are
other methods that have better performance.
Such a statement raises the question of how performance should be
characterized. There is clearly a need for theoretical concepts to treat this
problem. More specifically, we will have to discuss what violations of
assumptions might arise in practice, how they can be conceptualized, and
how the sensitivity-or its opposite, the robustness-of statistical proce-
dures to these deviations should be measured. But a new theory may not
only be used to describe the existing procedures under new aspects, it also
suggests new methods that are superior in light of the theory, and we will
extensively describe such new procedures.
The first theoretic approach to robust statistics was introduced by Huber
in his famous paper published in 1964 in the Annals of Mathematical
Statistics. He identified neighborhoods of a stochastic model which are
supposed to contain the “true” distribution that generates the data. Then he
found the estimator that behaves optimal over the whole neighborhood in a
minimax spirit. In the following decade, he extended his basic idea and
found a second approach. His book on the subject (Huber, 1981) made this
fundamental work available to a wider audience.
The present book treats a different theoretic approach to robust statistics,
originated by Hampel (1968, 1971, 1974). In order to describe the basic
idea, let us draw on an analogy with the discussion of analytic functions in
calculus. If you focus on a particular value of the function’s argument, you
can approximate the function by the tangent in this point. You should not
forget, though, that the linearization may be useful at most up to the vicinity
of the nearest singularity. Analogously, we shall introduce the influence
function of an estimator or a test, which corresponds to the first derivative,
and the breakdown point, which takes the place of the distance to the
nearest singularity. We shall then derive optimal procedures with respect to
the “infinitesimal” behavior as characterized by the influence function.
The approach is related to Huber’s minimax approach, but it is mathe-
matically simpler. It is not confined to models with invariance structure, as
the minimax approach is. Instead, it can be used generally whenever
maximum likelihood estimators and likelihood ratio tests make sense.
The intention of the book is to give a rather comprehensive account of
the approach based on influence functions, covering both generalities and
applications to some of the most important statistical models. It both
introduces basic ideas of robust statistics and leads to research areas. It may
serve as a textbook; the exercises range from simple questions to research
problems. The book also addresses an audience interested in devising robust
PREFACE xi
methods for applied problems. The mathematical background required
includes the basics of calculus, linear algebra, and mathematical statistics.
Occasionally, we allude to more high-powered mathematics.
Chapter 1 provides extensive background information and motivation for
robust statistics in general, as well as for the specific approach of this book.
Except for a few paragraphs, it may be read profitably even without the
background just mentioned.
Chapters 2 through 7 deal with the mathematical formulation of the
results. Chapter 2 treats the estimation problem in the case of a single
parameter and introduces the basic concepts and results, and Chapter 3
does the same for the respective testing problem. Chapter 4 extends the
approach to the estimation of a general finite-dimensional parameter and
gives detailed advice as to how it may be applied in any decent particular
problem at hand. Chapters 5 and 6 elaborate on the application of the
approach to the estimation of covariance matrices and regression parame-
ters, respectively, including possible variants and some related work on
these problems. The testing problem in regression is treated in Chapter 7.
Chapter 8 gives an outlook on open problems and some complementary
topics, including common misunderstandings of robust statistics. Most of it
can be read in connection with Chapter 1 after skimming Chapter 2.
Two decades have passed since the fundamental concepts of robust
statistics have been formulated. We feel that it is high time for the new
procedures to find wide practical application. Computer programs are now
available for the most widely used models, regression and covariance matrix
estimation. We hope that by presenting this general and yet quite simple
approach as well as the basic concepts, we help to get everyday’s robust
methods going.
HOW TO READ THE BOOK
The material covered in this book ranges from basic mathematical structures
to refinements and extensions, from philosophical remarks to applied exam-
ples, and from basic concepts to remarks on rather specific work on special
problems. For the reader’s convenience, complementary parts are marked
by an asterisk.
Different readers will direct their main interests to different parts of the
book. A chart given before the table of contents indicates possible paths.
The most important definitions of robustness notions can be found in
Sections 2.lb-c, 2.2, 2Sa, 3.2a, 3.5, and 4.2a-b, and the most important
mathematical results are probably in Sections 2.4a, 2.6c, 4.3a-b, and 6.3b.
xii PREFACE
The applied statistician will be particularly interested in the examples and
methods such as those discussed in Sections Lld, 2.0, 2.ld, 2.le, 2.6c, 3.1,
3.2c, 4.2d, 5.3c, 6.2, 6.3b, 6.4% 7.3d, and 7Sd, and in the techniques
implemented in the computer programs for robust regression and covari-
ance matrices mentioned in Subsection 6.4~.F inally, he or she should
become aware of the largely open problem of long-range dependence
described in Section 8.1.
ACKNOWLEDGMENTS
J. W. Tukey, P. J. Huber, and C. Daniel have had decisive influences,
directly or indirectly, on our view of robust statistics and data analysis.
H. Kiinsch has generously contributed a section.
Many colleagues have supported us by their encouragement and stimulat-
ing comments on the manuscript, in particular R. Carroll, B. Clarke, D.
Donoho, A. Durbin, C. A. Field, I. Olkin, E. Spjnrtvoll, G. S. Watson, and
R. E. Welsch. Very valuable comments have been given by two anonymous
referees.
The book developed as a result of a short course given at the 15th
European Meeting of Statisticians in Palermo, Italy, in 1982; the course was
proposed to us by the Chairman of the Programme Committee, J. Ooster-
hoff.
M. Karrer spent many overtime hours typing portions of the manuscript.
Other portions were typed by P. FIemming and N. Zuidervaart. M. Mtichler
and J. Beran have done computations and searched for references. The
second author has been partially supported by ARO contract #DAAG29-
82-K-0178, and the third author by the Belgian National Science Founda-
tion.
Last but not least we wish to thank our wives for their patience and
support.
FRANKR . HWEL
ELVEZXMO. RONCHETM
PETERJ . ROUSSEEUW
WERNEAR , STAHEL
Ztlrich, Switzerland
Princeton. New Jem9
De& ThC Netherlam&
Zffrich, Switzerlclnd
March 1985
POSSIBLE PATHS THROUGH THE BOOK
Quick overview Chapters 1,2, parts of 8,
and introductions to 3-7
One-parameter models Chapters 1,2, 3
Multiparameter models Chapters 1.2.4, and 5
(
estimation Chapters 1,2,4, and 6
Regression testing
Chapters 1,2,3 ,6, and 7
xiii