Multiple comparison procedures

487 Pages·1987·21.3 MB·English

by  Yosef Hochberg

Multiple Comparison
Procedures
Multiple Comparison
Procedures
YOSEF HOCHBERG
Tel-Aviv University
Ramat-Aviv, Israel
AJIT C. TAMHANE
Northwestern University
Evanston, Illinois
JOHN WILEY & SONS
New York Chichester Brisbane Toronto Singapore
0 0 0 0
This book has electrofiically reproduced
been h
informatioa stared at Wiley & Sons,
digital John h.
We are pleased that the use of thrs new technology
wili enable us to works of endurmg scholarly
aGeep
value in print BS long as there is a reasonable demand
for them. The content of this book is identical to
previous pnnhgs.
Copyright 0 1987 by John Wiley & Sons, Inc.
qll rights reserved. Published simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system or transmitted
in my form 01- by any means, electronic, nlechanical, photucopying, reconling, scanning
or othenvisc, except as permitted under Sections 107 or 108 of the 1976 United States
Copyright Act, without cirher the prior written permission of the Publisher, or
authorization through payment of the appropriate per-copy fee to the Copyright
Clwancc Center, 222 Rosewood Drivc, Danvcrs, MA01923, [phone], fax
(978j 750-4470. Requests to the Publisher for permission should be addressed io the
Permisbions Department. John Wiley & Sons. Inc., 1 I1 River Street, Hoboken. NJ 07030.
(201) 738-601 I, fax [phone]. E-Mail: [email protected].
To order or customer service please. call l(800bCALL-WILEY (225-5945).
books for
Library of Congm~C atslogiag PuMkation Data:
in
Hochberg, Yosef.
Multiple comparison procedures.
(Wiley series in probability and mathematical
statistics. Applied probability and statistics,
ISSN 0271.6356)
Bibliography: p.
Includes index.
1. Muliple comparisons {Statistics) 1. Tamhane,
Ajit C. 11. Title. 111. Series.
OA278.4.H63 1987 519.5'35 87-6226
0-471-82222-1
ISBN
To our parents
Preface
The foundations of the subject of multiple comparisons were laid in the
late 1940s and early 1950s, principaIly by David Duncan, Roy,
S.N.
Henry Scheffi, and John Tukey, although some of the ideas appeared
much earlier in the works of Fisher, Student, and others; see Harter 1980
for a complete historical account. Tukey’s (1953) mimeographed notes on
the subject form a rich source of results (some of which are being
rediscovered even today), but unfortunately they had only a limited
circulation. Miller’s (1966) book helped to popularize the use of multiple
comparison procedures (MCPs) and provided an impetus to new research
in the field. There have been a large number of review articles surveying
the field, most published during the 1970s (e.g., Aitkin 1969, Chew
1976a, Dunnett 1970, Dunnett and Goldsmith 1981, Games 1971, Gill
1973, Miller 1977, 1985, O’Neill and Wetherill 1971, Ryan 1959, Shaffer
1986b, Spj~tvoll1 974, and Thomas 1973). Commonly used MCPs are
discussed in many elementary and advanced texts. However, no unified
comprehensive treatment of the subject that incorporates the develop-
ments in the last two decades and that plays a role similar to Miller’s
monograph is available. The aim of the present is to fulfill this need.
book
Because MCPs have been around for quite some time and because
their use is well accepted (and sometimes even required) in many
disciplines, one might expect that there would be few, if any, unresolved
questions and controversies. Such is not the case, however. Considerable
confusion still exists in regard to what the different MCPs provide, which
ones should be used, and when. There are many statisticians who even
question the very need and appropriateness of any multiple comparison
approach (see, e.g., the discussion following O’Neill and Wetherill 1971,
and the papers by Carmer and Walker 1982, Dawkins 1983, Little 1978,
O’Brien 1983, Perry 1986, and Petersen 1977).
vii
vili PREFACE
Some of the controversial issues cut across the general subject of
statistical inference and are not germane specifically to multiple com-
parisons. These include testing versus confidence estimation, and the
choice of approach to inference (e.g., a classical Fisherian or a Neyman-
Pearsonian approach, a decision-theoretic approach, a Bayesian ap-
proach, or an informal graphical approach). We prefer not to take rigid
stands on these issues. Certainly applications can be found for each basic
approach to inference, although some types of applications are more
common than others. We have tried to present a variety of techniques
based on different approaches, spelling out the pros and cons in each
case. The emphasis, however, is on classical approaches and on confi-
dence estimation.
Another line of criticism stems from the misuse of MCPs in practice. A
common example of this is the application of an MCP (a popular choice
being Duncan’s 1955 stepwise procedure) for making pairwise com-
parisons among all treatments when the treatments have a certain struc-
ture, as is the case when they correspond to multifactorial combinations
or to increasing levels of some quantitative factor (Chew 1976b). In such
situations, comparisons other than pairwise comparisons may be of
interest; for example, orthogonal contrasts, and a specially tailored MCP
for such a family of comparisons may be required. Sometimes procedures
other than MCPs are required, for example, when the researcher’s goal is
to select the “best” treatment or to explore the possible clustering
patterns among the treatments. The key point is that statistical proce-
dures that should be used in a given problem depend on the questions of
interest and the nature of research. Problems of inference encountered in
empirical research are of diverse nature and indiscriminate use of MCPs
or, for that matter, any other statistical technique in all problems is
clearly inappropriate. In Chapter 1 we have addressed the questions of
when and why to use MCPs. This discussion should also help to answer
other basic criticisms voiced against multiple comparisons.
The subject of multiple comparisons forms a part of the broader
subject of simultaneous statistical inference. In this book we focus on
problems involving multiplicity and selection (“data-snooping”) of infer-
ences when comparing treatments based on univariate responses. (Refer
to Krishnaiah, Mudholkar, and Subbaiah 1980 and Krishnaiah and Reis-
ing 1985 for the corresponding multivariate techniques.) Roy’s union-
intersection method forms the unifying theme used to derive the various
classical MCPs for these problems. We do not discuss the problems of
simultaneous point estimation and simultaneous confidence bands in
regression. We also do not discuss the related topic of ranking and
PREFACE ix
selection procedures, on which there are full-length books by Gibbons,
Olkin, and Sobel (1977) and Gupta and Panchapakesan (1979).
The following is a brief outline of the book. In Chapter 1 we e!aborate
on our philosophy and approach to multiple comparison problems, and
discuss the basic notions of families, error rates, and control of error
rates. The remainder of the book is divided into two parts. Part I,
consisting of Chapters 2-6, deals with MCPs based on a classical error
rate control approach for fixed-effects linear models under the usual
normal theory assumptions. Part 11, consisting of Chapters 7-11, deals
with MCPs for other models and problems, and MCPs based on alterna-
tive approaches (e.g., decision-theoretic and Bayesian). Pere are three
appendixes. Appendix 1 gives some general theory of MCPs that is not
restricted to the setting of Part I (for which case the corresponding theory
is given in Chapter 2). Appendix 2 reviews the probability inequalities
that are used in deriving and computing conservative critical points for
various MCPs. Finally, Appendix 3 discusses the probability distributions
that arise in the context of some classical MCPs. Tables of the percentage
points of these distributions are also included in this appendix.
Sections are numbered fresh starting in each chapter. Subsections,
equations, examples, figures, and tables are numbered fresh starting in
each section but with the corresponding section identification. In both
cases there is no chapter identification provided. Thus, for example,
within a given chapter, equation (3.10) is the tenth equation in Section 3
of that chapter. When the same equation is referenced in another
chapter, it is referred to as equation (3.10) of such and such chapter.
This book is intended to serve the needs of researchers as well as
practitioners. The comprehensive review of the published literature until
1986, as well as the many open problems noted throughout the book,
should prove valuable to researchers. (It should be remarked, however,
that the literature on multiple comparisons is vast and it is impossible to
reference each and every publication. We have limited our references to
those publications that are relevant to the discussions of the topics
covered in the book.) Practitioners should find it helpful to have the steps
involved in implementing the various MCPs clearly spelled out and
illustrated by small numerical examples. Many of the examples are taken
from the original papers where the corresponding MCPs first appeared.
As mentioned before, the examples and discussion of Chapter 1 provide
useful guidelines to the perennial problem of when to use MCPs; a need
for such case studies was noted by Anscombe (1985).
We have assumed that the reader has had a course in mathematical
statistics covering the basic concepts of inference and is familiar with
X PREFACE
matrix algebra, linear models, and experimental designs. For the benefit
of those who may not have had training in the last two topics, we have
included a brief review of the necessary ideas in Section 1 of Chapter 2.
This can be used as a text in a special topics course on multiple
book
comparisons or for supplementary reading in a course on linear models
and analysis of variance.
In conclusion, we hope that this book will meet its stated objectives. In
addition, we hope that it will help to dispel some of the confusion and
controversy that have surrounded the subject of MCPs, and encourage
their correct use when appropriate in practice.
YOSEFH WHBERG
AJITC .
TAMHANE
Tel Aviv, Israel
Evanston, Illinois
Acknowledements
We would like to gratefully acknowledge the facilities provided principal-
ly by Northwestern University and also by Tel-Aviv and New York
Universities, which made the work on this book possible. Early drafts of
some of the chapters were written when the second author visited Cornell
University in 1982-1983 on sabbatical leave with partial support from the
U.S. Army Research Durham Contract DAAG-29-81-K-0168.
Office,
We are also indebted to many individuals who contributed in different
ways to this book. Bob Berk, Morton Brown, Tony Hayter, Jason Hsu,
Tom Santner, Juliet Shaffer, Gary Simon, and Yung Liang Tong read
sections of the manuscript and provided comments and suggestions.
G.V.S. Gopal read the entire manuscript and noted several errors. Charlie
Dunnett computed Tables in Appendix 3 as well as some special
4-7
percentage points of the multivariate ?-distribution required in Examples
2.1 and 2.3 of Chapter 5. Bob Bechhofer and Ruben Gabriel gave timely
advice and encouragement. Beatrice Shube, the senior editor of the
statistics series at Wiley, was always patient and cooperative with us. The
other staff at Wiley were also very helpful and efficient in carrying out a
speedy and quality production of the book. Particular mention should be
made of Margaret Comaskey, the editorial supervisor and Diana Cisek,
the senior production supervisor. Finally, it is our pleasure to acknow-
ledge the assistance of June Wayne, Carol Surma and Marianne Delaney
of Northwestern University. June cheerfully typed and retyped numerous
drafts of the manuscript (which required a considerable amount of
patience and perseverance in spite of all the modern word processor
features). Carol graciously pitched in whenever June was away. Marianne
typed the author and subject indexes. We wish to express our most
sincere thanks to them.
xi