Robust Statistics: The Approach Based on Influence Functions PDF
Preview Robust Statistics: The Approach Based on Influence Functions
Robust Statistics Robust Statist ics The Approach Based on Influence Functions FRANK R. HAMPEL ETH, Zurich, Switzerland ELVEZIO M. RONCHETTI Princeton University, Princeton, New Jersey PETER J. ROUSSEEUW Derft University of Technology, Derft, The Netherlanh WERNER A. STAHEL ETH, Zurich, Switzerland John Wiley & Sons New York Chichester Brisbane Toronto Singapore Copyright @ 1986 by John Wiley & Sons, Inc. All rights rcscrvcd. Published simultaneously in Canada. Reproduction or translation of any part of this work beyond that permitted by Section 107 or 108 of the 1976 United States Copyright Act without the permission of the copyright owner is unlawful. Requests for permission or further information should be addressed to the Permissions Department, John Wiley & Sons, Inc. Lihrary oj Congress Cataloging m Publication Data. Main entry under title: Robust statistics. (Wiley series in probability and mathematical statistics. probability and mathematical statistics, ISSN 0271-6356) Includcs index. 1. Robust statistics. 1. Hampcl, Frank R., 1941- . 11. Scri~~. QA276.R618 1985 519.5 R5-9428 ISBN 0-471-73577-9 10 9 8 7 6 5 4 3 To our families and friends Non omnia possumus omnes Preface Statistics is the art and science of extracting useful information from empirical data. An effective way for conveying the information is to use parametric stochastic models. After some models had been used for more than two centuries, R. A. Fisher multiplied the number of useful models and derived statistical procedures based on them in the 1920s. His work laid the foundation for what is the most widely used statistical approach in today’s sciences. This “classical approach” is founded on stringent stochastic models, and before long it was noticed that the real world does not behave as nicely as described by their assumptions. In addition, the good performance and the valid application of the procedures require strict adherence to the assump- tions. Consequently, nonparametric statistics emerged as a field of research, and some of its methods, such as the Wilcoxon test, became widely popular in applications. The basic principle was to make as few assumptions about the data as possible and still get the answer to a specific question like “Is there a difference?” While some problems of this kind did find very satisfactory solutions, parametric models continued to play an outstanding role because of their capacity to describe the information contained in a data set more completely, and because they are useful in a wider range of applications, especially in more complex situations. Robust statistics combines the virtues of both approaches. Parametric models are used as vehicles of information, and procedures that do not depend critically on the assumptions inherent in these models are imple- mented. Most applied statisticians have avoided the worst pitfalls of relying on untenable assumptions. Almost unavoidably, they have identified grossly aberrant observations and have corrected or discarded them before analyz- ing the data by classical statistical procedures. Formal tests for such outliers have been devised, and the growing field of diagnostics, both formal and ix X PREFACE informal, is based on this approach. While the combination of diagnosis, corrective action, and classical treatment is a robust procedure, there are other methods that have better performance. Such a statement raises the question of how performance should be characterized. There is clearly a need for theoretical concepts to treat this problem. More specifically, we will have to discuss what violations of assumptions might arise in practice, how they can be conceptualized, and how the sensitivity-or its opposite, the robustness-of statistical proce- dures to these deviations should be measured. But a new theory may not only be used to describe the existing procedures under new aspects, it also suggests new methods that are superior in light of the theory, and we will extensively describe such new procedures. The first theoretic approach to robust statistics was introduced by Huber in his famous paper published in 1964 in the Annals of Mathematical Statistics. He identified neighborhoods of a stochastic model which are supposed to contain the “true” distribution that generates the data. Then he found the estimator that behaves optimal over the whole neighborhood in a minimax spirit. In the following decade, he extended his basic idea and found a second approach. His book on the subject (Huber, 1981) made this fundamental work available to a wider audience. The present book treats a different theoretic approach to robust statistics, originated by Hampel (1968, 1971, 1974). In order to describe the basic idea, let us draw on an analogy with the discussion of analytic functions in calculus. If you focus on a particular value of the function’s argument, you can approximate the function by the tangent in this point. You should not forget, though, that the linearization may be useful at most up to the vicinity of the nearest singularity. Analogously, we shall introduce the influence function of an estimator or a test, which corresponds to the first derivative, and the breakdown point, which takes the place of the distance to the nearest singularity. We shall then derive optimal procedures with respect to the “infinitesimal” behavior as characterized by the influence function. The approach is related to Huber’s minimax approach, but it is mathe- matically simpler. It is not confined to models with invariance structure, as the minimax approach is. Instead, it can be used generally whenever maximum likelihood estimators and likelihood ratio tests make sense. The intention of the book is to give a rather comprehensive account of the approach based on influence functions, covering both generalities and applications to some of the most important statistical models. It both introduces basic ideas of robust statistics and leads to research areas. It may serve as a textbook; the exercises range from simple questions to research problems. The book also addresses an audience interested in devising robust PREFACE xi methods for applied problems. The mathematical background required includes the basics of calculus, linear algebra, and mathematical statistics. Occasionally, we allude to more high-powered mathematics. Chapter 1 provides extensive background information and motivation for robust statistics in general, as well as for the specific approach of this book. Except for a few paragraphs, it may be read profitably even without the background just mentioned. Chapters 2 through 7 deal with the mathematical formulation of the results. Chapter 2 treats the estimation problem in the case of a single parameter and introduces the basic concepts and results, and Chapter 3 does the same for the respective testing problem. Chapter 4 extends the approach to the estimation of a general finite-dimensional parameter and gives detailed advice as to how it may be applied in any decent particular problem at hand. Chapters 5 and 6 elaborate on the application of the approach to the estimation of covariance matrices and regression parame- ters, respectively, including possible variants and some related work on these problems. The testing problem in regression is treated in Chapter 7. Chapter 8 gives an outlook on open problems and some complementary topics, including common misunderstandings of robust statistics. Most of it can be read in connection with Chapter 1 after skimming Chapter 2. Two decades have passed since the fundamental concepts of robust statistics have been formulated. We feel that it is high time for the new procedures to find wide practical application. Computer programs are now available for the most widely used models, regression and covariance matrix estimation. We hope that by presenting this general and yet quite simple approach as well as the basic concepts, we help to get everyday’s robust methods going. HOW TO READ THE BOOK The material covered in this book ranges from basic mathematical structures to refinements and extensions, from philosophical remarks to applied exam- ples, and from basic concepts to remarks on rather specific work on special problems. For the reader’s convenience, complementary parts are marked by an asterisk. Different readers will direct their main interests to different parts of the book. A chart given before the table of contents indicates possible paths. The most important definitions of robustness notions can be found in Sections 2.lb-c, 2.2, 2Sa, 3.2a, 3.5, and 4.2a-b, and the most important mathematical results are probably in Sections 2.4a, 2.6c, 4.3a-b, and 6.3b. xii PREFACE The applied statistician will be particularly interested in the examples and methods such as those discussed in Sections Lld, 2.0, 2.ld, 2.le, 2.6c, 3.1, 3.2c, 4.2d, 5.3c, 6.2, 6.3b, 6.4% 7.3d, and 7Sd, and in the techniques implemented in the computer programs for robust regression and covari- ance matrices mentioned in Subsection 6.4~.F inally, he or she should become aware of the largely open problem of long-range dependence described in Section 8.1. ACKNOWLEDGMENTS J. W. Tukey, P. J. Huber, and C. Daniel have had decisive influences, directly or indirectly, on our view of robust statistics and data analysis. H. Kiinsch has generously contributed a section. Many colleagues have supported us by their encouragement and stimulat- ing comments on the manuscript, in particular R. Carroll, B. Clarke, D. Donoho, A. Durbin, C. A. Field, I. Olkin, E. Spjnrtvoll, G. S. Watson, and R. E. Welsch. Very valuable comments have been given by two anonymous referees. The book developed as a result of a short course given at the 15th European Meeting of Statisticians in Palermo, Italy, in 1982; the course was proposed to us by the Chairman of the Programme Committee, J. Ooster- hoff. M. Karrer spent many overtime hours typing portions of the manuscript. Other portions were typed by P. FIemming and N. Zuidervaart. M. Mtichler and J. Beran have done computations and searched for references. The second author has been partially supported by ARO contract #DAAG29- 82-K-0178, and the third author by the Belgian National Science Founda- tion. Last but not least we wish to thank our wives for their patience and support. FRANKR . HWEL ELVEZXMO. RONCHETM PETERJ . ROUSSEEUW WERNEAR , STAHEL Ztlrich, Switzerland Princeton. New Jem9 De& ThC Netherlam& Zffrich, Switzerlclnd March 1985 POSSIBLE PATHS THROUGH THE BOOK Quick overview Chapters 1,2, parts of 8, and introductions to 3-7 One-parameter models Chapters 1,2, 3 Multiparameter models Chapters 1.2.4, and 5 ( estimation Chapters 1,2,4, and 6 Regression testing Chapters 1,2,3 ,6, and 7 xiii
The list of books you might like
