SIMULATIONS TO ANALYZE TYPE I ERROR AND POWER IN THE ANOVA F TEST AND ...

80 Pages·2009·0.44 MB·English

SIMULATIONS TO ANALYZE TYPE I ERROR
AND POWER IN THE ANOVA F TEST AND
NONPARAMETRIC ALTERNATIVES

by
Joshua Daniel Patrick
B.S., The University of West Florida, 2007

A thesis submitted to the Department of Mathematics and Statistics
College of Arts and Sciences
The University of West Florida
In partial fulfillment of the requirements for the degree of
Master of Science

2009
The thesis of Joshua Daniel Patrick is approved:

____________________________________________ _________________
Raid W. Amin, Ph.D., Committee Member Date

____________________________________________ _________________
Subhash C. Bagui, Ph.D., Committee Member Date

____________________________________________ _________________
Morris L. Marx, Ph.D., Committee Chair Date

Accepted for the Department/Division:

____________________________________________ _________________
Kuiyuan Li, Ph.D., Chair Date

Accepted for the University:

____________________________________________ _________________
Richard S. Podemski, Ph.D., Dean of Graduate Studies Date
ii
TABLE OF CONTENTS

LIST OF FIGURES ........................................................................................................ iv
ABSTRACT .................................................................................................................... vi
CHAPTER I. INTRODUCTION ........................................................................1
CHAPTER II. PRELIMINARIES ........................................................................3
A. The ANOVA F Test ................................................................3
B. The Kruskal-Wallis Test .........................................................4
C. The Normal Scores Test..........................................................5
D. Type I Error and Power ...........................................................5
E. Distributions ............................................................................6
CHAPTER III. LITERATURE REVIEW .............................................................8
CHAPTER IV. THE SIMULATIONS .................................................................12
CHAPTER V. RESULTS ...................................................................................15
A. Normal Data ..........................................................................15
B. Contaminated-Normal Data ..................................................27
C. Gamma Data .........................................................................39
CHAPTER VI. SUMMARY AND CONCLUSIONS .........................................52
REFERENCES ...............................................................................................................56
APPENDIXES ................................................................................................................58
A. SAS Code for Normal Data ..................................................59
B. SAS Code for Contaminated-Normal Data...........................64
C. SAS Code for Gamma Data ..................................................70

iii
LIST OF FIGURES

1. Type I error for normal data, small equal samples ..................................................16
2. Type I error for normal data, medium equal samples .............................................17
3. Type I error for normal data, large equal samples ..................................................18
4. Type I error for normal data, small unequal samples ..............................................19
5. Type I error for normal data, medium unequal samples .........................................20
6. Type I error for normal data, large unequal samples ..............................................21
7. Power for normal data, small equal samples ...........................................................22
8. Power for normal data, medium equal samples ......................................................23
9. Power for normal data, large equal samples ............................................................24
10. Power for normal data, small unequal samples .......................................................25
11. Power for normal data, medium unequal samples ..................................................26
12. Power for normal data, large unequal samples ........................................................27
13. Type I error for contaminated-normal data, small equal samples ...........................28
14. Type I error for contaminated-normal data, medium equal samples ......................29
15. Type I error for contaminated-normal data, large equal samples ............................30
16. Type I error for contaminated-normal data, small unequal samples .......................31
17. Type I error for contaminated-normal data, medium unequal samples ..................32
18. Type I error for contaminated-normal data, large unequal samples ........................33
19. Power for contaminated-normal data, small equal samples ....................................34
iv
20. Power for contaminated-normal data, medium equal samples ................................35
21. Power for contaminated-normal data, large equal samples .....................................36
22. Power for contaminated-normal data, small unequal samples ................................37
23. Power for contaminated-normal data, medium unequal samples ............................38
24. Power for contaminated-normal data, large unequal samples .................................39
25. Type I error for Gamma data, small equal samples ................................................40
26. Type I error for Gamma data, medium equal samples ............................................41
27. Type I error for Gamma data, large equal samples .................................................42
28. Type I error for Gamma data, small unequal samples.............................................43
29. Type I error for Gamma data, medium unequal samples ........................................44
30. Type I error for Gamma data, large unequal samples .............................................45
31. Power for Gamma data, small equal samples ..........................................................46
32. Power for Gamma data, medium equal samples .....................................................47
33. Power for Gamma data, large equal samples ..........................................................48
34. Power for Gamma data, small unequal samples ......................................................49
35. Power for Gamma data, medium unequal samples .................................................50
36. Power for Gamma data, large unequal samples ......................................................51

v
ABSTRACT
SIMULATIONS TO ANALYZE TYPE I ERROR AND
POWER IN THE ANOVA F TEST AND
NONPARAMETRIC ALTERNATIVES
Joshua Daniel Patrick
The Analysis of Variance (ANOVA) F test is used to test the equality of
population means for three or more samples. However, the F test has some assumptions
that are frequently ignored and often violated when used in real world applications. These
assumptions include that the data is normally distributed and the population variances are
equal. Violating these assumptions can lead to an inflated Type I error and a decrease in
power. Simulations are conducted to analyze the Type I error and power of the F test
when the assumptions are violated. Two nonparametric alternatives to the F test, the
Kruskal-Wallis test and the normal scores test, are also analyzed. The results show that
the nonparametric tests have a lower Type I error and higher power under certain cases of
nonnormality and unequal variances. These simulations serve as a guide to which test
should be used based on the type of data being analyzed.

vi
CHAPTER I
INTRODUCTION
Applied researchers sometimes need to decide whether sample differences in
central tendency reflect true differences in parent populations. In the case of two or more
groups, the most commonly used test is the Analysis of Variance (ANOVA) F test.
However, problems arise when the assumptions of the F test are violated. These
violations may cause the Type I error to be different than what is assumed and also have
an effect on the power of the test. The assumptions for the F test include that the data is
normally distributed, the sample variances are equal, and the samples are independent.
When the assumption of normality and/or equality of variance are violated, it has been
recommended that the researcher use a non-parametric procedure.
This paper will analyze the F test in terms of empirical Type I error and empirical
power when the assumptions are violated and when they are not. The Kruskal-Wallis
rank test and the expected normal scores test are two nonparametric counterparts to the F
test so the paper will also discuss the empirical Type I error and empirical power of these
tests. The F test, the Kruskal-Wallis test, and the normal scores test will all be analyzed
using the same data sets. This analysis will be conducted using a SAS program that will
generate random data from a specified distribution with given parameters. Each of the
three tests will then be conducted with the data at a significance level of 0.05. This
simulation will be repeated 1000 times for each set of parameters, and then the proportion
1
of times the null hypothesis is rejected is recorded. If the parameters are set for equal
means, the proportion of rejections is the empirical Type I error of the test. If the
parameters are set for unequal means, it is the empirical power of the test. From these
results, the Type I error and power can be analyzed for each of the three tests to see
which performs better under the various violations of assumptions.

2
CHAPTER II
PRELIMINARIES
The ANOVA F Test
Analysis of variance (ANOVA) refers to the procedure that tests
hypotheses of differences between means for three or more samples. This test is useful
for researchers who are looking to compare the effects caused by k different levels or
treatments. In order to accomplish this, a test statistic that is a ratio of variance estimates
is used.
The test statistic is a ratio of two independent estimates of the population
variance. The first estimate is sensitive to treatment effect and error and is known as the
mean squares for treatments (MSTR). It is calculated as

where
k n ( )2
SSTR =∑∑ y − y
ij
i=1 j=1
which is known as the treatment sum of squares. The second estimate is sensitive only to
error within treatments and is known as the mean square for error (MSE) and is
calculated as
3
where
SSE = ∑k ∑n (y − y)2
ij
i=1 j=1
which is known as the error sum of squares. Here, N denotes the sum of all of sample
sizes in the k treatments. If there is no treatment effect, then the two estimates (MSTR
and MSE) should be equal and their ratio would be at or close to one. For ratios that are
larger than one to a “large” enough extent, a difference in the means of the treatments is
suggested. When the null hypothesis is true, that is when the means are equal for all
groups, the ratio of the two estimates has an F distribution with k – 1 and N – k degrees of
freedom. Therefore, the test statistic is calculated as
.
The null hypothesis, , should then be rejected at the α level of

significance when .
The Kruskal-Wallis Test
One of the nonparametric alternatives to the F test is the Kruskal-Wallis test
(Kruskal & Wallis 1952). For this test, the data is ranked so that the smallest of all
observations receives a rank of one, and the largest of all observations receives rank N.
The test statistic is calculated as
(cid:3038)
12 (cid:1844)(cid:2870)
(cid:3037)
(cid:1834) (cid:3404) (cid:3533) (cid:3398)3(cid:4666)(cid:1840)(cid:3397)1(cid:4667)
(cid:1840)(cid:4666)(cid:1840)(cid:3397)1(cid:4667) (cid:1866)
(cid:3037)
(cid:3037)(cid:2880)(cid:2869)
where (cid:1866) is the sample size for the jth sample, N is the total sample size, and (cid:1844)(cid:2870) is the
(cid:3037) (cid:3037)
sum of the ranks for the jth sample. If there are ties present,
4