ERIC ED463565: Do High Stakes Tests Drive Up Student Dropout Rates? Myths versus Reality. Knowledge Brief.

14 Pages·2001·0.3 MB·English

by  ERIC

DOCUMENT RESUME
ED 463 565
EA 031 564
AUTHOR
Rabinowitz, Stanley; Zimmerman, Joy; Sherman, Kerry
Do High Stakes Tests Drive Up Student Dropout Rates? Myths
TITLE
versus Reality. Knowledge Brief.
INSTITUTION
WestEd, San Francisco, CA.
SPONS AGENCY Office of Educational Research and Improvement (ED),
Washington, DC.
PUB DATE 2001-00-00
NOTE
13p.
CONTRACT
ED-01-00-0012
AVAILABLE FROM
WestEd, 730 Harrison Street, San Francisco, CA 94107-1242.
Tel: [phone]; Tel: [phone] (Toll Free)
For full
.
text: http://www.wested.org.
PUB TYPE Reports
Evaluative (142)
EDRS PRICE
MF01/PC01 Plus Postage.
DESCRIPTORS
Dropout Characteristics; *Dropout Rate; *Dropout Research;
*Dropouts; *Educational Assessment; High Schools; *High
Stakes Tests; Potential Dropouts; Test Use
ABSTRACT
This report proposes that not enough good data or research
has been done to settle the debate over whether testing affects high school
dropout rates. Advocates argue that the threat of missing out on a diploma or
of being retained motivates students to work harder, resulting in higher
academic achievement. Critics argue that failing a high school exam, being
retained, or anticipating such failure can push students over the academic
edge, causing them to quit school. Problems with understanding the dropout
issue include inconsistencies with how dropouts are defined and reported.
Testing produces information that is limited in both scope and analytical
methodology. While some studies suggest broad reasons for dropping out, such
as not liking school or not getting along with teachers, no causal connection
has been revealed between any single factor and the decision to quit school.
The report recommends that negative unintended consequences of high-stakes
test programs be disclosed, that keeping accurate dropout data is important,
and that states allow a fair phase-in program to allow for alignment of
curriculum, teaching practice, and assessment. Research recommendations
include doing more credible and extensive data collection, and implementing
more longitudinal studies and causal research. (Contains 38 references.)
(RT)
Reproductions supplied by EDRS are the best that can be made
from the original document.
' I
I
U.S. DEPARTMENT OF EDUCATION
Office of Educational Research and Improvement
EDUCATIONAL RESOURCES INFORMATION
1;r<CENTER (ERIC)
his document has been reproduced as
received from the person or organization
originating it.
Minor changes have been made to
1:1
improve reproduction quality.
Points of view or opinions stated in this
document do not necessarily represent
official OERI position or policy.
Tests
Do HIGH
STAKES
drive up
tudent
dropout rates?
Itiffftfiffig
'
myths versus reality
it
VI
Ilustration by James Yang
The increased use of testing to determine whether students can graduate from high
school or be promoted to the next grade has evoked much discussion about its
possible impact on students' decisions to drop out of school. Advocates argue that
Written by
this new student accountability mechanism
the threat of missing out on a
Stanley Rabinowitz
diploma or, for younger pupils, of being retained
motivates students to work
Joy Zimmerman
Kerry Sherman
harder, resulting in higher academic achievement. Critics, on the other hand,
argue that failing a high school exit exam, being retained, or, even, just
WestEd
anticipating such failure can push some students over the academic edge, causing
Improving education through
them to quit school. The result, they say, will be higher dropout rates. Each side
research, development, and service
can point to some research findings that support its contention.
2
BEST COPY AVAILABLE
Do High-Stakes Tests Drive up Student Dropout Rates?
This Knowledge Briefproposes that there has not
several reasons, not least of which is the relative
been enough good data or research to settle that
newness of the standards-and-accountability
debate. More importantly, it suggests that if we
package. While exit exams and other forms of high-
really want to understand dropout trends, the
stakes testing have been used in the past, never have
current focus on high-stakes testing
they been so ubiquitous or measured
per se is too narrow. More pertinent
such rigorous academic standards. In
is the impact of standards-based
addition, as this brief will further
reform writ large, which includes the
explain, some significant technical
adoption of statewide accountability
impediments hinder our full
systems intended to give it clout.
understanding of the dropout issue,
How is this broad reform effort,
especially as it relates to standards and
especially the adoption of "rigorous,"
accountability. Chief among them is
world-class" academic standards,
the difficulty of achieving consistency,
affecting the education experiences of
first, in how dropouts are defined and,
borderline pupils, those most at risk
then, in how dropout rates are
reported. Another constraint is the
of academic failure? Is it leading
them to higher achievement or is it
prevalence of an incomplete research
leaving them further behind?
model that looks primarily at the
single and narrow variable of high-stakes testing
Inherent in this standards-and-accountability model
absent the larger context of standards. Finally, there
of the late 20th and early 21" century is a
is researchers' dependency on limited correlational
methodology, rather than one that incorporates some
commitment to ensuring academic success for
student groups that have not been adequately
degree of experimental controls.
served in the past. No group more obviously fits
that definition than students who decide to quit
Before continuing, it should be noted that even in
the most effective schools, some degree of dropout
school before graduating. The degree to which a
is anticipated. Some problems in a student's life
school can offset adverse circumstances in a
outside of school may not be mitigated even by the
student's home or community is open to discussion.
most capable teacher or the most committed
But the standards-based reform model is premised
dropout prevention efforts. The intent of this brief
on the belief that when educators are well trained
and supported and when all members of the school
is to move the discussion away from the individual
student to the effects of the recent reform agenda
community are clear about what students are
expected to learn and what the stakes are, school
on groups of students considered at risk
can, in fact, ensure that all students learn. Thus, in
academically (e.g., children in poverty, special
theory, when standards-based reform was fully education students, English language learners).
implemented, dropout rates would go down. For
that reason, understanding how rates are being
Dropouts: An Elusive Indicator
affected, if at all, during the relatively early stages of
implementation would be an important step in
Neither the concern about dropout rates nor the
tracking the progress of this reform and guiding its
attempt to track them is new. For much of the last
continued implementation.
century, a high school diploma or its equivalent was
considered a passport to either a good job or to
Unfortunately, such understanding is more elusive
further education. Today, with fewer well-paying
than one might initially assume. This is true for
3
2
WestEd
positions available to those who have only a high
larger age range than does the completion rate
school education, the diploma alone (or its
(16-24 versus 18-24). This discrepancy is just one
indication of a longstanding and continuing
equivalent) is less effective in guaranteeing any kind
of non-dead-end job. But it remains essential for
understanding of the dropout
challenge to our
the advanced education and
situation: While most people find
training now requisite to a
it easy to define and discuss the
growing proportion of those jobs
dropout phenomenon in general
043ome
states have
that pay a living wage. Thus, in
terms, the variety of definitions
decided to distinguish
today's highly competitive
and collection and calculation
economic environment, the future
methods now in use make it
between school
success of those who leave school
difficult to pin down
without completing at least their
"dropouts" and
the specifics.
K-12 education is problematic
school "leavers,"
a fact that has ramifications for
Because there are several different
them as individuals, but also for
definitions of dropouts and various
excluding the latter
the economy and for society.
ways of calculating rates, a
from their dropout rate
particular definition or calculation
Recognizing the increasing
method may be selected to
calcu
lation.
importance of a high school
advance a political or policy
degree, the National Education
agenda. For example, many states
Goals Panel (NEGP) targeted, as
report an annual dropout rate of
Goal #2, achievement of at least a 90 percent high
around 4 percent. This means that in any given
school completion rate by the year 2000. Yet little
school year, approximately 4 students out of 100
progress appears to have been made. In its 1999
enrolled (in all secondary grades combined) drop
report, the NEGP noted that the national high
out of school. However, this means that by the time
school completion rate (i.e., the percentage of
the freshmen reach graduation, 16 percent of their
18- to 24-year-olds who complete high school or
class will have dropped out
a cumulative rate
receive an alternative credential) had increased only
that paints quite a different picture of the education
two percentage points since 1973, remaining at a
system's success. Similarly, some states have decided
relatively flat 85-86 percent during the 1990s.
to distinguish between school "dropouts" and
school "leavers," excluding the latter from their
Dropout rates offer a different perspective on the
dropout rate calculation. State-by-state comparisons
same story, serving as the complementary opposite
are made difficult by these and other variations,
of completion rates. For example, the proportion of
such as how a state counts students who leave
16- to 24-year-olds who are not in school and have
school and later re-enter, or how it counts students
not received a diploma or its equivalent constitutes
who complete high school by an alternative means,
what's known as the "status" dropout rate. That
such as obtaining a General Educational
rate, as reported by the U.S. Department of
Development (GED) credential. Such variations
Education's National Center for Education Statistics
naturally cast doubt on the precision and
(NCES), was approximately llpercent for 1999.
comparability of the state reports themselves and on
Readers will note that the status dropout rate does
any national rates based on those reports.
not precisely complement the 85.9 percent
completion rate also reported by NCES for 1999
Some improvements have been achieved over the
because the status rate definition encompasses a
past decade. More than three quarters of the states
4
3
Do High-Stakes Tests Drive up Student Dropout Rates?
have now adopted uniform definitions developed by
some evidence that the gap between the rates for
the NCES, up from 14 in 1992. However, even in
African American students and white students may
these states, the care with which dropout data are
be narrowing, but the dropout rate for Hispanic
compiled and reported differs from school site to
students has remained consistently higher.
school site, just as it does from state to state.
Part of the concern about high-stakes testing, of
course, is that rather than narrowing the
Variance in Group Dropout Rates:
achievement gap of which these disparate rates are
emblematic, the current accountability push could
Reflection of the Achievement Gap
widen it. Higher standards, without sufficient
support for teachers and students, do nothing to
Their general imprecision and inconsistency
improve student learning. As students who may
notwithstanding, U.S. dropout rates have yielded
already be struggling academically fall further
disconcertingly uniform information about one
behind and face the consequences of high-stakes
aspect of the phenomenon: Dropout rates have
assessments, dropout rates may be expected to
remained consistently higher in certain student
increase. However, given the problems with
groups, breaking down primarily according to race/
defining key indicators and data collection
ethnicity and socioeconomic status, although
procedures described earlier, even such simple
students with disabilities also have significantly
correlations are suspect and would warrant
higher dropout rates than the general student
more research.
population. Embedded in this country's dropout
rates, by anyone's calculation, is an
overrepresentation of minority students, whose
The Narrow Focus on Tests
dropout rates exceed 50 percent in some urban
schools.' Not surprisingly, differences by
The temptation to focus narrowly on the impact of
socioeconomic level are also stark: NCES numbers
high-stakes assessment is understandable.
show that in 1999, students whose family incomes
Assessment is, after all, the most
were in the lowest 20 percent
visible and quantifiable piece of
nationally were five times as likely
the standards-and-accountability
to drop out of high school as those
tre
nd has been
reform. When schools begin
whose family incomes were in the
towar
designing curricula based on new
d using
top 20 percent.
state standards, parents,
high-stak
es tests as
policymakers, and other members
In 1999, the high school
of the public may or may not take
indicator
the sole
completion rate was 94 percent for
note. But when large numbers of
Asian/Pacific Islander students and
t promotion
for studen
students start failing the new tests,
approximately 90 percent for white
people begin paying attention,
students, as compared with 83.5
duation.
or gra
especially parents and the news
percent for African American
media. Extra attention can be
students and 63.4 percent for
beneficial, as all involved become
Hispanic students. These figures
more aware of the hard work needed for successful
were not an anomaly. Over the past quarter century,
implementation of this reform. But often lost in
dropout rates for African American and Hispanic
that attention is the role that rigorous standards
students have consistently been higher than those
may play in the failure rate.
for white students. In recent years, there has been
5
4
WestEd
student and school accountability to support
High school "exit" exams are the most common
approach to student accountability, with
achievement of "world-class" standards.
approximately half the states now requiring or
planning to require a test for high school
Considering Assessments in a
graduation.2These tests are generally first
administered in either 10th or 11th grade, with
Broader Context
students given multiple opportunities to retake the
test if they do not initially pass it. As noted earlier,
Currently, all but one state (Iowa) have adopted
growing numbers of states are also requiring that
standards-based reform, setting rigorous statewide
most
students be tested at key grade levels
K-12 education standards in core subjects and
commonly, grades 4 and 8
to
expecting districts and schools to
inform decisions about whether to
align curriculum, assessment, and
promote or retain them.'
teacher professional development
')(a
ising
to those standards. While
In making decisions about student
ons about
expectati
standards-based reform has yet to
promotion or graduation, some
be fully implemented in most
what stud
ents will
states consider not just assessment
states, the standards themselves
scores, but additional measures of
have already wrought significant
bout their
learn and a
academic achievement and/or the
changes in the academic lives of
completion of prescribed
performa
nce levels
many students, showing up in
coursework. But the trend has
tougher curriculum and
hool life
make sc
been toward using high-stakes tests
assessment. For example, whereas
as the sole indicator for student
more cha
Ilenging.
algebra has traditionally been
promotion or graduation.' This
considered an elective course,
accountability strategy has resulted
available starting in the 9th grade, it
in an increasing number of states
is increasingly included in the curriculum as a 7th or
and school districts denying promotion and
8th grade requirement. In similar fashion, while
graduation to students who have not performed
many high schools have long had a two-year math
well enough on a particular exam, irrespective of
and a one-year science requirement, an increasing
how they have performed on other measures of
number of schools now require three years and two
academic achievement.'
years of math and science, respectively. In short,
expectations about what students will learn and
High-stakes testing itself is not new. For example, in
about their performance levels have risen, making
1984 Texas passed a comprehensive school reform
school life more challenging
and, in some
law that instituted high-stakes testing of basic skills.
for teachers and students
instances more stressful
That testing program, called the Texas Educational
alike.
Assessment of Minimum Skills (TEAMS), assessed
students at multiple grade levels and required that
Forty-eight states have also implemented statewide
they pass an "exit level" version of the test in order
assessments to measure the degree to which students
to receive a high school diploma. Nevada, New
are reaching their new standards.6 The standards
Jersey, and Florida each developed high school
assessed on today's tests represent significantly
graduation tests prior to 1980, but they, too,
higher levels of reasoning and problem solving than
focused on minimum basic skills. New today is the
previously assessed on minimum competency tests.
context in which high-stakes tests are being used: for
Using the Texas example, in 1990, changes in state
6
5
Do High-Stakes Tests Drive up Student Dropout Rates?
law required adoption of a more challenging testing
test results could trigger extraordinary services at
program than TEAMS, which had covered only
school (e.g., tutoring, summer school, after-school
basic skills. The Texas Assessment of Academic
homework help) and, perhaps, greater support at
Skills, which replaced TEAMS, is based on rigorous
home (e.g., discussions about how much time the
new state academic standards;
student should be spending on
these standards are continuously
schoolwork versus social life or
being reviewed and, in some
job). This added support could
cases, continue to be raised.
accelerate the student's learning
While not all statewide
to the extent that, within the
assessments yet fully align with
next two years, he or she was able
their state's academic standards,
to pass the test. Poor results on
higher expectations are evident
an end-of-grade promotion exam
in many of the assessments. For
could trigger a similar response;
lig
example, reading passages that
rather than automatically being
once ran less than a page in
retained, the student could, for
length now more commonly
example, be assigned to summer
run 2-3 pages, with the
school or be provided with some
questions that follow more
form of tutoring.
difficult as well.
In theory, this is how the
One result of all this is that a school can no longer
standards-and-accountability model should work
promise, as many once implicitly did, that if
with ongoing assessment (both statewide and local)
students just stay in school and attend all their
serving as an early warning system, identifying
classes, they will get a high school diploma. The
students who need extra help (and the areas in
advent of basic skills graduation tests did not much
which they need it), teachers who need support in
alter this implicit "contract." However, when an exit
adjusting their practice to better ensure student
exam is based on rigorous standards, it is not
learning, and systems (both local and statewide)
something teachers and students can "cram" for. The
that need adjusting. As noted earlier, to the extent
knowledge and understanding students are required
that the model is operating as conceived, dropout
to demonstrate on many of today's high-stakes
rates should decrease. To the extent that it's not,
assessments are cumulative. This means that for
they may well go up. For that reason, a closer look
every learning benchmark not achieved during a
at dropout rates and the factors underlying them is
student's earlier studies, the student is that much
warranted. In embarking on this effort, it helps to
farther away from being able to pass the graduation
understand what we already know
and don't
know
about why students drop out.
test.
In this context, it's easy to understand why an ill-
What We Know and Don't Know
prepared student who fails in a first (or second)
attempt at an exit exam might be tempted to give
About Why Students Quit School
up and quit school entirely. Yet that's not the only
possible scenario. The first failure could also serve
A decision to drop out of school can be influenced
as an early warning not only to the student, but
by an array of factors related to the student's own
also to the student's parents, teachers, and other key
makeup, his or her family, what's going on in his or
players in the student's academic life. These initial
her community, and what's going on in school.' In
7
6
West Ed
In the still relatively early days of this reform, as
the National Educational Longitudinal Study of
schools struggle to implement the curricular and
1988, students reported a wide variety of reasons
instructional changes required to teach the more
for dropping out of school, only 77 percent school-
challenging standards-based content, there may be
related. And even those school-related factors most
short-term decreases in academic achievement
commonly cited
did not like school, failing
overall. The effects of this "disruption" on dropout
school, could not get along with teachers'
reveal
rates, particularly for low-achieving students, need
nothing about the multitude of variables
to be examined in fully
underlying them: What made
disaggregated data sets.
them dislike school? Why and
when did they begin to fall
33chools can no
Finally, to understand possible
behind? In what ways did teacher
longer promise, as
and student not connect? While
relationships between high-stakes,
standards-based assessment and
studies like this suggest some
many once implicitly
student dropout rates, we need to
broad reasons for dropping out, it
explore more comprehensive causal
is almost impossible to
did, that if students
demonstrate a causal connection
models. For example, while the test
just stay in school and
may be the most immediate trigger
between any single factor and the
decision to quit school.'
of a student's behavior, the test
attend all their classes,
results may simply reflect or
they will get a high
Nonetheless, research has
aggravate a history of similar low
performance. The test may be seen
identified several specific school-
school diploma.
based circumstances that appear
as the messenger reflecting the
school's failure to adequately
to be predictive of students'
decisions to drop out of school. It incorporate into instructional
is this evidence, mainly correlational, that has
practice both the rigorous new standards
led a number of researchers to theorize that
themselves and the expectation that all students
higher dropout rates are or will be a negative could master them. Thus, while the test appears to
unintended consequence of high-stakes testing, have "caused" changes in the dropout rate, the die
especially when that testing is based on rigorous
was cast when academic standards exceeded the
academic standards.'° The first two factors are so
system's ability to implement them for all segments
of the student population.
closely related as to be virtually inseparable: being
retained and being overage in grade. The third
factor, also closely related, is having a history of
Conclusions and
poor academic achievement as reflected in grades
and test scores.
Policy Recommendations
What these findings underscore is the critical nature
Due in part to dropout reporting inconsistencies
of aligning standards, curriculum, and test content
and the fact that high-stakes tests based on high
to ensure that throughout their academic careers
standards have not been in place for a long period
students are being tested on what they have had the
of time, it is difficult to know what impact such
opportunity to learn. If high school exit exams or
tests have had or might have in the future.
grade promotion exams do not reflect what students
Numerous studies suggest that grade retention
are being taught, failure rates and student dropout
significantly increases a student's likelihood of
rates will invariably increase.
and some studies even point to
quitting school
8
7
Do High-Stakes Tests Drive up Student Dropout Rates?
retention as the single strongest predictor of a
high school diploma or its equivalent, states must
decision to drop out. For that reason, one could
develop policies and support systems that serve the
assume that if increased testing for grade promotion
complementary desires for increased student
resulted in greater retention, it would ultimately
achievement and decreased dropout rates. In
result in higher dropout rates as well. Yet more
developing or revisiting their standards-based reform
research is needed even in this area
effort and the accountability
because it might be that students
system designed to support it,
who consistently do poorly in
states could profit from considering
ntion may
Cete
school would drop out at higher
the following policy issues:
productive
be the least
rates irrespective of whether they
had been retained at some point(s)
Disclosure About Negative
option for d
ealing with
because of a high-stakes test.
Unintended Consequences: Full
disclosure about possible negative
students w
ho fail an
Most agree that it is important to
consequences of high-stakes
early-war
ning test.
have high expectations for
testing programs is a necessary
students and that students should
component of a statewide
achieve a level of academic
assessment program, as is an
performance that will prepare them for life after
ongoing evaluation of these effects. To minimize
high school. Therefore, some may argue that the
unintended consequences, states should carefully
benefits of more accountability and higher
monitor their accountability systems before a new
standards for students outweigh the risk of a few
test is operational and during its implementation.
more dropouts. However, with the proper approach
to implementation of standards-based reform,
Data Collection: Keeping accurate dropout data is
including high-stakes testing, this tradeoff may be
important. So, too, is disaggregating that data to
avoided. Tests can and should be used as a way of
obtain information about the relative achievement
identifying students early on who are at risk of
of student subgroups. Disaggregated data can also
failing so as to provide them with the extra help
help identify struggling students early on so
they need. Retention is not the only option
and
districts and schools can target help.11
for dealing
may often be the least productive
with students who fail an early-warning test. As
Fair Phase-in Period: It is important that states allow
noted earlier, an alternative is to promote these
for an adequate phase-in period, during which
students but provide remediation, through summer
curriculum, teaching practice, and assessment can be
school and/or the use of tutors or after-school
aligned to the new standards. If high-stakes
programs the following year. Schools could also
assessment is to play any role in enhancing student
meet the needs of these students by creating mixed achievement, students must have an opportunity to
grade classes (e.g., a 4"/5'). Thus, the likelihood of
learn the material, and teachers must receive support
students quitting school may depend less on the use
they need to fully understand and be able to teach to
of high-stakes tests than on state accountability
the standards on which the assessment is based.
policies, school/district organization and structure,
and how those play out in the implementation of
Opportunities for Meaningfid Remediation: High-
stakes tests can be used for early identification of
the tests.
students who need extra help and to enhance
Given the trend toward increased use of high-stakes
instructional planning. If, for example, test results
testing and given the continued importance of a
show that large numbers of students in one school
9
8
WestEd
student achievement, placing more students at risk
are performing poorly on
or, even, one class
of dropping out.
certain parts of the test, it may indicate the need to
further examine curriculum and/or practice.
Prevention and Intervention Programs for Students at
Risk: Many dropout prevention programs target
With the recent proliferation of high-stakes
middle or high school students who have already
tests, increasing numbers of students are assigned
had negative school experiences and/or are already
to remedial classes. These classes offer
considering dropping out. These programs tend to
concentrated instruction on the specific content
focus on helping students
knowledge and skills the test
overcome their longstanding
is intended to measure. Some
problems in order to get them
also include drills intended to
to complete school. In
familiarize students with the
contrast, some early
test format. Research needs to
intervention programs, in
focus on which remediation
place at the elementary level,
practices increase student
promote student motivation
performance and which may
and success early in a student's
frustrate and further alienate
academic experience. Evidence
student populations who are
on dropout prevention and
at risk academically.
intervention programs is
weak; there have been
Test Retake Policies: Currently,
relatively few evaluations and
all states with high school exit
those that exist do not provide enough information
exams provide opportunities for students to retake
for tailoring treatment to student subpopulations."
the test if they do not pass.'2 Depending on the
stakes, a test may need to be offered several times a
year; however, sufficient remediation time must
Research Agenda
also be factored in lest students simply repeat their
previous performance.
As mentioned earlier, further research is necessary
to find out if there is truly a causal relationship
Sufficient Validity for Each Intended Use: If a test is
between high-stakes testing and student dropout
to be fair, it must measure what it was designed to
rates or, more importantly, to find out how
measure.13 The National Research Council of the
standards-based reform writ large is serving students
National Academy of Sciences explains that the
most at risk academically. As a requisite first step,
results of tests designed and found valid for
more credible and extensive data collection is
influencing classroom practice or for holding
needed regarding student achievement, grade
schools accountable are not necessarily appropriate
retention, and dropout rates. These data must be
as the basis for high-stakes decisions about
available from all states for a number of years, using
individual students. Yet in their rush to promote
uniform definitions and data-collection procedures.
academic improvement, many states have
Student achievement and dropout data must be
developed accountability systems in which
tracked relative to key reform initiatives within and
important decisions about individual students
across state lines (e.g., implementation of new
are based on the
promotion or graduation
standards and testing programs, reauthorization of
results of tests not designed for that purpose. Such
important federal legislation).
tests may lead to inappropriate conclusions about
1 0
9