Multiple Comparisons Using R

202 Pages·2010·4.115 MB·English

by  Frank Bretz, Torsten Hothorn, Peter Westfall

Multiple
Comparisons
Using R
C5742_FM.indd 1 6/29/10 2:35:24 PM
Multiple
Comparisons
Using R
Frank Bretz
Torsten Hothorn
Peter Westfall
C5742_FM.indd 2 6/29/10 2:35:24 PM
Multiple
Comparisons
Using R
Frank Bretz
Torsten Hothorn
Peter Westfall
C5742_FM.indd 3 6/29/10 2:35:24 PM
Chapman & Hall/CRC
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
Chapman & Hall/CRC is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed in the United States of America on acid-free paper
10 9 8 7 6 5 4 3 2 1
International Standard Book Number: 978-1-58488-574-0 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Library of Congress Cataloging‑in‑Publication Data
Bretz, Frank.
Multiple comparisons using R / Frank Bretz, Torsten Hothorn, Peter Westfall.
p. cm.
Includes bibliographical references and index.
ISBN 978-1-58488-574-0 (hardcover : alk. paper)
1. Multiple comparisons (Statistics) 2. R (Computer program language) I. Hothorn,
Torsten. II. Westfall, Peter H., 1957- III. Title.
QA278.4.B74 2011
519.5--dc22 2010023538
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
C5742_FM.indd 4 6/29/10 2:35:25 PM
To Jiamei and my parents
— F.B.
To Carolin, Elke and Ludwig
— T.H.
To Marilyn and Amy
— P.W.

Contents
List of Figures ix
List of Tables xi
Preface xiii
1 Introduction 1
2 General Concepts 11
2.1 Error rates and general concepts 11
2.1.1 Error rates 11
2.1.2 General concepts 17
2.2 Construction methods 20
2.2.1 Union intersection test 20
2.2.2 Intersection union test 22
2.2.3 Closure principle 23
2.2.4 Partitioning principle 29
2.3 Methods based on Bonferroni’s inequality 31
2.3.1 Bonferroni test 31
2.3.2 Holm procedure 32
2.3.3 Further topics 34
2.4 Methods based on Simes’ inequality 35
3 Multiple Comparisons in Parametric Models 41
3.1 General linear models 41
3.1.1 Multiple comparisons in linear models 41
3.1.2 The linear regression example revisited using R 45
3.2 Extensions to general parametric models 48
3.2.1 Asymptotic results 48
3.2.2 Multiple comparisons in general parametric models 50
3.2.3 Applications 52
3.3 The multcomp package 53
3.3.1 The glht function 53
3.3.2 The summary method 59
3.3.3 The confint method 64
vii
viii CONTENTS
4 Applications 69
4.1 Multiple comparisons with a control 70
4.1.1 Dunnett test 71
4.1.2 Step-down Dunnett test procedure 77
4.2 All pairwise comparisons 82
4.2.1 Tukey test 82
4.2.2 Closed Tukey test procedure 93
4.3 Dose response analyses 99
4.3.1 A dose response study on litter weight in mice 100
4.3.2 Trend tests 103
4.4 Variable selection in regression models 108
4.5 Simultaneous confidence bands 111
4.6 Multiple comparisons under heteroscedasticity 114
4.7 Multiple comparisons in logistic regression models 118
4.8 Multiple comparisons in survival models 124
4.9 Multiple comparisons in mixed-effects models 125
5 Further Topics 127
5.1 Resampling-based multiple comparison procedures 127
5.1.1 Permutation methods 127
5.1.2 Using R for permutation multiple testing 137
5.1.3 Bootstrap testing – Brief overview 140
5.2 Group sequential and adaptive designs 140
5.2.1 Group sequential designs 141
5.2.2 Adaptive designs 150
5.3 Combining multiple comparisons with modeling 156
5.3.1 MCP-Mod: Dose response analyses under model uncer-
tainty 157
5.3.2 A dose response example 162
Bibliography 167
Index 183
List of Figures
1.1 Probability of committing at least one Type I error 2
1.2 Scatter plot of the thuesen data 4
1.3 Impact of multiplicity on selection bias 8
2.1 Visualization of two null hypotheses in the parameter space 24
2.2 Venn-type diagram for two null hypotheses 25
2.3 Schematic diagram of the closure principle for two null
hypotheses 25
2.4 Schematic diagram of the closure principle for three null
hypotheses 27
2.5 Partitioning principle for two null hypotheses 30
2.6 Schematic diagram of the closure principle for three null
hypotheses under the restricted combination condition 33
2.7 Rejection regions for the Bonferroni and Simes tests 36
2.8 Power comparison for the Bonferroni and Simes tests 37
3.1 Boxplots of the warpbreaks data 54
3.2 Two-sided simultaneous confidence intervals for the Tukey
test in the warpbreaks example 65
3.3 Compact letter display for all pairwise comparisons in the
warpbreaks example 67
4.1 Boxplots of the recovery data 70
4.2 One-sided simultaneous confidence intervals for the Dunnett
test in the recovery example 76
4.3 Closed Dunnett test in the recovery example 81
4.4 Two-sided simultaneous confidence intervals for the Tukey
test in the immer example 87
4.5 Compact letter display for all pairwise comparisons in the
immer example 89
4.6 Alternativecompactletterdisplayforallpairwisecomparisons
in the immer example 90
4.7 Mean-mean multiple comparison and tiebreaker plot for all
pairwise comparisons in the immer example 92
4.8 Mean-mean multiple comparison plot for selected orthogonal
contrasts in the immer example 94
ix