AUTHOR Fixed-, Random-, and Mixed-Effects ANOVA Models

24 Pages·2012·0.24 MB·English

DOCUMENT RESUME
ED 426 098
TM 029 328
AUTHOR
Frederick, Brigitte N.
TITLE
Fixed-, Random-, and Mixed-Effects ANOVA Models: A
User-Friendly Guide for Increasing the Generalizability of
ANOVA Results.
PUB DATE
1999-01-00
23p.; Paper presented at the Annual Meeting of the Southwest
NOTE
Educational Research Association (San Antonio, TX, January
21-23, 1999).
Speeches/Meeting Papers
PUB TYPE
Guides
Non-Classroom (055)
(150)
EDRS PRICE
MF01/PC01 Plus Postage.
*Analysis of Variance; *Mathematical Models; Research Design
DESCRIPTORS
IDENTIFIERS
*Fixed Effects; Generality; Random Variables; *Rules of
Thumb (Statistics)
ABSTRACT
Most researchers using analysis of variance (ANOVA) use a
fixed-effects model. However, a random- or mixed-effects model may be a more
appropriate fit for many research designs. One benefit of the random- and
mixed-effects models is that they yield more generalizable results. This
paper focuses on the similarities and differences between the various ANOVA
models and the factors that should be considered when determining which model
to use. An explanation of the "Rules of Thumb" for deriving the correct
formulas for computing "F" statistics is also given. (Contains 2 tables and
15 references.)
(SLD)
********************************************************************************
Reproductions supplied by EDRS are the best that can be made
from the original document.
********************************************************************************
Running head: FIXED-, RANDOM-, AND MIXED-EFFECTS ANOVA MODELS
U.S. DEPARTMENT OF EDUCATION
PERMISSION TO REPRODUCE AND
Office of Educational Research and Improvement
EDUCATIONAL RESOURCES INFORMATION
DISSEMINATE THIS MATERIAL HAS
CENTER (ERIC)
BEEN GRANTED BY
efriiis
document has been reproduced as
received from the person or organization
originating it.
edee i cI
El Minor changes have been made to
-t-
improve reproduction quality.
TO THE EDUCATIONAL RESOURCES
Points of view or opinions stated in this
INFORMATION CENTER (ERIC)
document do not necessarily represent
official OERI position or policy.
Fixed-, Random-, and Mixed-Effects ANOVA Models:
A User-Friendly Guide For Increasing the
Generalizability of ANOVA Results
Brigitte N. Frederick
Texas A & M University
CO
(s1
Cy)
CN1
CD
Paper presented at the annual meeting of the Southwest
Educational Research Association, San Antonio, January, 1999.
Random- and Mixed-Effects
2
Abstract
employ
fixed-
ANOVA
procedures
a
using
Most researchers
effects model. However, a random- or mixed-effects model may
for many research designs.
be a more appropriate fit
One
and mixed-effects models
they
that
benefit
random-
is
of
yield more generalizable results. This paper focuses on the
similarities and differences between the various ANOVA models
and the factors that should be considered when determining
which model to utilize. A description of the "Rules of Thumb"
for deriving the correct formulas for computing F statistics
will also be explained.
3
Random- and Mixed-Effects
3
Fixed-, Random-, and Mixed-Effects ANOVA Models:
A User-Friendly Guide For Increasing the
Generalizability of ANOVA Results
variance
analysis
using
Most
researchers
(ANOVA)
of
although they may
procedures choose a fixed-effects model,
not realize that they are making this choice or realize its
models most
Although
are
consequences. fixed-effects
the
common of the ANOVA models, they are not necessarily the most
appropriate and/or useful
experimental
procedures
for all
designs. There are two additional types of ANOVA models that
are less commonly used: random-effects (also called Model II)
models and mixed-effects (also called Model III) models.
A random-effects model is used when the researcher wants
within
generalize levels/conditions
findings
to
to
ways/factors beyond the levels that are represented in
the
Hays (1981) commented,
study (Jackson & Brashers,
"the
1994)
.
random-effects model is designed especially for experiments
in which inferences are to be drawn about an entire set of
including
not
distinct
treatments
some
factor
levels,
or
A mixed-effects model
actually observed"
is
a
376).
(p.
combination of fixed- and random-effects models, comprised of
one or more random ways and one or more fixed ways.
the present paper
The purpose of
outline
the
to
is
differences between
mixed- and random-effects
the
fixed-,
way should
on deciding whether
ANOVA designs.
be
Tips
a
classified
random
will
fixed
also provided.
be
or
as
as well as the
"rules of
Computational differences,
thumb"
4
Random- and Mixed-Effects 4
for deriving the correct formulas to compute F statistics in
the random and mixed models will also be explained.
Three Models
the null hypothesis
In a fixed-effect model,
that
is
there are no differences between the means of the levels of
In
the way that are utilized in the study
1988)
(Ostle,
.
random-effects
tested
hypothesis
in
contrast,
the
a
model is that there are no differences in the means of
in the
of a way that are possible
all of the levels
was
levels
sample
of
levels
population
the
that
of
not just in the sample of levels that is
chosen from,
utilized in the study
1988). Thus, random-effects
(Ostle,
models attempt to generalize beyond both the sampled people
and the sampled levels.
what researchers
between
There incongruence
some
is
consider a random way. Some argue that any way that does not
include all the possible levels should be treated as random
Others diSagree.
Wike
Richter & Seay,
1987)
1973;
(Clark,
.
outline three basic methods for selecting
and Church (1976)
a way to be used
the
First,
study.
levels
in
a
the
of
a way
researcher could use all of
the possible levels
of
EQUALS the number of
levels
the
in
(number of
levels,
p,
population of levels, P). Clearly, this is rarely the case in
actual research. Second, the researcher could randomly choose
a subset of all possible levels of a way to use in the study
is less than the number of levels in
(number of levels, p,
5
Random- and Mixed-Effects
5
with random selection of levels).
Third,
the population,
P,
the researcher could choose to nonrandomly select a subset of
a way to use in the study
the possible levels
all
of
of
is less than the number
(number of levels in the sample, p,
with nonrandom selection of
of levels in the population,
P,
It seems clear that the p=P case must be treated as
levels)
.
random selection
the p<P case,
fixed-effects model.
In
a
model
random-effects
1976).
Church,
implies
(Wike
a
&
and statistical
typically
treat
the
Mathematical
formulas
as fixed
(Wike &
nonrandom selection case)
third case
(p<P,
However, some researchers argue that the third
Church, 1976) .
case should also be treated as a random-effects model (Clark,
1973; Richter & Seay, 1987).
This debate is clearly illustrated by Clark
in
(1973)
his article, "Language-as-Fixed Effect Fallacy: A Critique of
Using
Language Statistics
Research."
Psychological
the
in
Clark argued that words,
example of
language research,
as
random effects
should be treated as
well as participants,
because the words that are selected for language studies do
not extinguish all possibilities within the entire population
if the researcher wishes to generalize
of words. Therefore,
results beyond the
must
ways
study
be
scope
the the
of
classified as random. Clark (1973) commented,
treat
investigator
When should
the
The
random
effect?
language
as
a
answer is, whenever the language stimuli
do not deplete the population from which
Random- and Mixed-Effects
6
they were
answer
The
not,
drawn.
is
whenever the language stimuli used were
chosen at random from this population.
(p. 348)
Richter and Seay (1987) agreed that classifying ways as
further
support
statistical
for
random
provides
study
replicated
on
They
generalization
a
results.
of
recognition memory and found that when words were considered
fixed there were three effects statistically significant at
the .05 level and four statistically significant at the
.01
level. However, when words were reclassified as random only
remained statistically
effects
seven
original
the
one
of
Although all
results.
presents
these
Table
significant.
1
studies will not have findings that differ this dramatically
the example illustrates the differences that
across models,
are possible when ways for the same data are classified as
random instead of fixed. In response to Clark's article, Wike
and Church
ways
classifying
while
argued
as
that
(1976)
errors,
Type
decrease
can
random
lead
in
a
to
I
overclassifying ways as random can lead to an increase in
Type II errors. They also contended that generalization can
not be assumed just because ways are classified as random.
They noted, "generality is not obtained simply by selecting p
levels randomly" (Wike & Church, 1976, p. 253).
best
would
be
way
determine
method
One
to a
if
classified as fixed or random is to see if the levels are
(Jackson & Brashers,
1988).
Ostle,
"interchangeable"
1994;
7
Random- and Mixed-Effects
7
That is, could alternate levels of the way be substituted in
the study without making a difference in the results? With a
random way, a set of levels could be utilized on one run of
the experiment and an entirely different set could be used on
In
a
replication
Hopkins,
subsequent
1984)
(Glass
a
&
.
way
specific
levels
random-effects
model
of
the
a
chosen to be utilized in a study are of no particular
the levels can
to the researcher.
Therefore,
interest
fundamental
changing
without
substituted
the
be
research question.
could studying
researcher the
be
For
example,
a
peer
teaching
tutoring,
methods:
effectiveness
three
of
Instead
assisted
and
computer lecturing.
of
instruction,
from every grade
kindergarten
taking samples
students
of
-
through twelve, the researcher could randomly select a subset
The researcher could
of grades to be present in the study.
then generalize the results of the study to all of the grades
and
randomly selected
not
possible, even
those
that are
It would make no difference to
present in the study.
the
researcher if the particular grades in the study were 1st,
10th, and 12th; the
5th, and 8th or 2nd,
4th, 7th,
3rd,
2nd,
particular
no
are
represented
study
of
the
grades
in
if the researcher were
interest to the researcher. However,
reading
to use different kinds of instruction
(for example,
each selected grade
text or parent
from the
tutoring)
at
the study would have an altered meaning.
Therefore,
level,
this example would be a mixed study, where the grade of the
8
Random- and Mixed-Effects
8
a random way and the mode of instruction is
students is
a
fixed way.
a way
should be
A second method
for determining
if
classified as random or fixed is based on what conclusions
the researcher would like to draw from the results of
the
If the researcher wants to
study (Jackson & Brashers, 1994) .
make conclusions based only on the levels of the population
that are used in the study it is appropriate to classify a
if the researcher would like to draw
way as fixed. However,
are
beyond
levels
populations
that
the
conclusions
about
the way must
classified
be
represented
as
the
study,
in
random. Using the teaching method example to demonstrate this
the conclusions that can be drawn if
think of
the
point,
It could be concluded,
grade way is treated as fixed.
for
example, that peer tutoring is better than computer assisted
instruction and lecturing for individuals in grades 2,
4,
7,
if these were the grade levels actually studied.
10, and 12,
In contrast, a conclusion that could be drawn when the grade
way is classified as random would be that peer tutoring is
better than computer assisted instruction and lecturing for
students in all grades.
and
Depending on
researcher
desires the
the
the
of
classified
be
can
ways
research
same
the
question,
differently in various studies. The determination of whether
a way should be classified as
random or
fixed ultimately
the research
context
depends
(Longford,
1993).
on
of
the
way
determination
will
whether
be
a
the
However,
of
9
Random- and Mixed-Effects
9
the data
considered random or fixed must be made prior to
collection and analysis. Hicks (1973) noted,
It is not reasonable to decide after the
whether
collected
been
have
data
the
fixed
considered
are
be
or
levels
to
random. This decision must be made prior
to the running of the experiment, and if
they must
random levels are to be used,
be chosen from all possible levels by a
random process.
173)
(p.
What are the repercussions of misclassifying a way? Of
there are no absolutely right answers to how a way
course,
should be
Coleman
are
classified. "there
stated,
(1979)
usually
analyze an
ways
experiment,
correct
several
to
a matter
and...the better choice
wisdom
than
more
of
is
However, there are always
mathematical correctness"
243)
(p.
.
situations where it is most appropriate to classify a way in
way
particular
situations
these
In
way. a
if
a
is
misclassified, different undesirable consequences can occur.
it will be
If a fixed way is misclassified as random,
statistical
overly conservative
subject
test
of
to
a
significance and therefore the likelihood of making a Type II
othesis) will increase
error (not rejecting a false null h
way
random
Church,
Inversely,
(Wike
1976).
is
if a
&
misclassified as fixed, there is a greater chance of making a
rejecting
Type
hypothesis)
error null
(falsely true
a
I
if a random way is classified as
1973). In addition,
(Clark,
10