Analysis of Variance via Confidence Intervals

239 Pages·2004·1.99 MB·English

by  K D Bird

Analysis of Variance via Confidence Intervals

Analysis of Variance via
Confidence Intervals

Kevin D. Bird

SAGE Publications

London • Thousand Oaks • New Delhi
 Kevin D. Bird 2004

First published 2004

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system,
transmitted or utilized in any form or by any means, electronic, mechanical, photocopying, recording
or otherwise, without permission in writing from the Publishers.

SAGE Publications Ltd
1 Oliver’s Yard

55 City Road
London EC1Y 1SP

SAGE Publications Inc
2455 Teller Road
Thousand Oaks, California 91320

SAGE Publications India Pvt Ltd
Post Box 4109
B-42 Panchsheel Enclave
New Delhi 110 017

British Library Cataloguing in Publication data

A catalogue record for this book is available from the British Library.

ISBN 0 7619 6357 X

Library of Congress Control Number available

Printed in India by Gopsons Papers Ltd, Noida
Contents

Preface ix

1
Comparing Two Means 1
Introduction 1
Organization of this book 3
Confident inference on a single comparison 4
Strength of inference on a comparison 5
Interpreting effect size 8
Practical equivalence inference 10
Constructing a confidence interval on a single comparison 12
Population standard deviation known 12
Population standard deviation unknown 14
Replicating the experiment: a simulation 17
The subjectivist critique of confidence interval inference 22
Further reading 23
Questions and exercises 24

2
One-way Analysis of Variance 27
The ANOVA model 27
Effect size 29
The ANOVA partition of variation 30
Heterogeneity inference 32
Contrasts 34
The scale of contrast coefficients 35
Contrast statistics 37
Simultaneous inference on multiple contrasts 39
Simultaneous confidence interval procedures 40
Heterogeneity inference from computer programs 42
Confidence interval inference on contrasts from computer programs 43
Example 2.1 Planned orthogonal contrasts 44
Example 2.2 Bonferroni-t confidence intervals on contrasts 47
Example 2.3 Scheffé post hoc analysis 48
Alternatives to Bonferroni-t SCIs for restricted analyses 49
Further reading 51
Questions and exercises 51

3
Precision and Power 54
Factors influencing precision 54
vi Contents
Precision of estimation with known error variance 56
Choosing an analysis strategy 57
Precision of estimation with unknown error variance 58
Example 3.1 Controlling precision 60
Power 60
Precision or power? 63
Further reading 64
Questions and exercises 64

4
Simple Factorial designs 66
Factorial effect contrasts defined on cell means 67
The two-factor main effects model 69
The two-factor ANOVA model with interaction 72
Sources of variation in a balanced two-factor design 75
Heterogeneity inference 78
Contrasts on parameters of the two-factor ANOVA model 79
A simple effects model 81
Error rates and critical constants 83
What if all factorial effects are equally important? 85
Example 4.1 Constructing CIs on all factorial contrasts 86
Increasing the complexity of factorial designs 88
Further reading 89
Questions and exercises 89

5
Complex Factorial Designs 91
Partitioning variation, degrees of freedom and the overall error rate 91
The Fee × Treatment data set 93
Factorial contrasts for complex two-factor designs 96
Product contrasts 96
Simplifying the terminology for factorial contrasts 103
Critical constants for CIs on contrasts within families 103
Selecting factorial contrasts on a post hoc basis 104
Example 5.1 An F-based two-way ANOVA 105
The SMR procedure 109
Example 5.2 SMR SCIs on all factorial contrasts 112
Planned contrasts analyses of data from J × K designs 114
Factorial designs with more than two factors 116
Three-factor designs with multiple levels on some factors 118
Further reading 120
Questions and exercises 120

6
Within-subjects Designs 123
The multivariate model for single-factor within-subjects designs 124
Confidence intervals on contrasts in planned analyses 125
Standardized within-subjects contrasts 126
Contents vii

Confidence intervals in post hoc analyses 128
Carrying out a planned analysis with PSY 129
Carrying out a post hoc analysis 130
Two-factor within-subjects designs 132
Analysis options 134
Example 6.1 Two-factor within-subjects planned analysis 135
Example 6.2 Two-factor within-subjects post hoc analysis 139
Within-subjects designs with more than two-factors 143
Further reading 143
Questions and exercises 144

7
Mixed Designs 146
The social anxiety data set 146
The multivariate means model for mixed designs 147
Example 7.1 Two-factor mixed-design planned analysis 150
Confidence interval procedures for post hoc analyses 153
Example 7.2 Two-factor mixed-design post hoc analysis 155
Tests of homogeneity hypotheses 158
Alternative multivariate test statistics 161
Allowing for inferences on simple effect contrasts 162
Example 7.3 GCR-based SCIs on all factorial contrasts 164
Mixed designs with more than two factors 167
Complex mixed designs with multiple levels on some factors 168
Beyond multifactor ANOVA 169
Further reading 170
Questions and exercises 170

Appendix A PSY 173

Appendix B SPSS 184

Appendix C Noncentral confidence intervals 190

Appendix D Trend Analysis 194

Appendix E Solutions 200

Appendix F Statistical Tables 211

References 219

Index 223

Preface

In recent years a great deal of emphasis has been placed on the value of interval
estimates of effect sizes in psychological and biomedical research. The
International Committee of Medical Journal Editors (1997), the APA Task
Force on Statistical Inference (Wilkinson and Task Force on Statistical
Inference, 1999) and the fifth edition of the APA Publication Manual (2001) all
recommend that confidence interval analysis should be preferred to null
hypothesis significance testing.
Standard treatments of analysis of variance (ANOVA), the most widely used
method of data analysis in experimental psychology, provide very little guidance
about how a confidence interval approach can be implemented in the ANOVA
context. When confidence intervals are discussed (usually very briefly), the
purpose is often to illustrate their value in interpreting outcomes that are not
statistically significant, rather than as the primary method of producing
statistical inferences.
My purpose in writing this book has been to provide a treatment of fixed-
effects ANOVA informed by Jason Hsu’s (1996) hierarchy of levels of
inference on comparisons. When applied to analyses based on ANOVA models,
this hierarchy makes it clear that interval estimates of contrasts on effect
parameters are more informative than (and imply) the lower-level inferences
(directional, inequality and homogeneity inferences) produced by traditional
approaches to ANOVA. The relationships between inferences at different levels
in the hierarchy provide a basis for comparison between the analyses
recommended here and the analyses typically reported in the literature.
The standard null hypothesis significance testing approach provides no
satisfactory basis for the interpretation of nonsignificant outcomes. One of the
most important advantages of the confidence interval approach is the basis it
provides for the interpretation of contrasts and other functions of effect
parameters (such as Cohen’s f ) when those parameters cannot be declared non-
zero by a statistical test. If the experimenter is able to specify the smallest non-
trivial value of an effect size parameter, an appropriate confidence interval will
show whether the data can justify the claim that the parameter is trivially small,
whatever the outcome of the corresponding significance test. I have included a
number of examples of practical equivalence inference based on confidence
intervals. The possibility of practical equivalence inference is particularly