ERIC ED342798: Strategies for Statewide Student Assessment. Policy Briefs, Number 17.

5 Pages·1991·0.32 MB·English

by  ERIC

DOCUMENT RESUME
ED 342 798
TM 017 963
Moody, David
AUTHOR
Strategies for Statewide Student Assessment. Policy
TITLE
Briefs, Number 17.
Far West Lab. for Educational Research and
INSTITUTION
Development, San Francisco, Calik.
Office of Educational Research and Improvement (ED),
S?ONS AGENCY
Washington, DC.
PUB DATE
91
CONTRACT
400-86-0009
NOTE
5p.
Reports - Evaluative/Feasibility (142)
PUB TYPE
EDRS PRICE
MF01/PC01 Plus Postage.
DESCRIPTORS Achievement Tests; Basic Skills; *Educational
Assessment; Educational Change; Elementary Secondary
Education; Evaluation Methods; Holistic Evaluation;
Psychometrics; Scoring; *Standardized Tests; *State
Programs; *Student Evaluation; Testing Problems;
Testing Programs; *Test Use
Alternatives to Standardized Testing; *Authentic
IDENTIFIERS
Assessment; *Performance Based Evaluation
ABSTRACT
Traditional standardized tests of basic skills are no
longer considered meaningful by many leading authorities in
educational measurement. Alternative approaches are not yet fully
developed, although many efforts are being made. This paper explores
the issues surrounding student assessment in the context of existing
and evolving state practices, which frequently combine high-stakes
evaluations with traditional multiple-choice norm-referenced
examinations and negatively affect instructional quality. A new
generation cf alternative strategies for student evaluation is being
designed to measure student performance in situations that bear an
authentic relationship to real-world tasks. Authentic assessment is
criterion-referenced and performance-based. It has intrinsic validity
and a holistic approach. Because authentic assessments are typically
much more difficult to score than tradAtional tests, they are
expensive. The psychometric foundations of authentic assessments are
still not fully developed. Vermont, Michigan, and Kentucky are
leaders in the effort to use authentir: assessment for statewide
testing programs. The fact that many problems surround the changing
nature of student assessment means that caution must be exercised in
using assessment as a tool of educational reform. (SLD)
***********************0:***********************************************
Reproductions supplied by EDRS are the best that can be made
from the original document.
*****R************************x************************p***************
POLICY
SCOPE OF INTEREST NOTICE
FAR
The ERIC Facility has assigned
this document for processing
BRIEFS
171
WEST
to:
ER
In our 'Lodgment. this document
LABORATORY
is also of interest to the Clear-
Inghouses noted to the right.
Indexing should reflect (heir
special ()dints of view.
NUMBER SEVENTEEN
1991
U.t DEPARTMENT OF EDUCATION
Office of Edtkalional Remarch and Improvement
Strategies for
EDUCATIONAL RESOURCES INFORMATION
CENTER (ERIC)
Statewide Student Assessment
aXhill
document has been reproduced Ils
received from the Person or organization
originating it
0 Minor changes have been made to improve
David Moody
reproduction quality
Points of view or opinions staled in this docu
mein do not necessarily represent official
OERI position or policy
students (whole schools or school
and evolving statewide practices.
Introduction
districts) with national norms.
First, the essential nature of the
Over the course of the last
,:ontroversy assodated with ..4tudent
decade, statewide assessments of
CIO
Problems arise when the school
assessmmt is explained. Then the
elh
student achievement have assumed a
performance measures carry high-
variety of assessment strategies
position of prominence in the land-
11"
stakes rewards and sanctions. Teach-
currently visible on the statewide
Di* scape of educational reform. Prior to
ers then become acutely aware of
level is reviewed. The Brief concludes
po 1980,
nearly half of all states man-
these consequences and tend to teach
with some issues policymakers need
dated no programs of this kind.
to the test. This phenomenon is so
to consider as they mandate new
'Mo Today, statewide assessments are all
well known that practitioners have
strategies for statewide student
tin but ubiquitous, and the nature of the
coined an acronym for it WYTIWYG
assessment.
debate has shifted away from whether
what you test is what you get,
F...4 to conduct such programs to what
Teachers not only gear dassroom
The Nature of the Controversy
cal kind! of program to conduct.
activities to the evaluative assess-
me:its, but often also substitute those
The assessment of student
As this Policy Brief explains,
foi their own diagnostic measures. At
achievement is a conflicted and
tradifional, standardized, multiple-
that point the distinction between
sensitive area of educational policy
choice tests of basic skills are no
instructional and evaluative assess-
and practice. The nature of the
longer considered meaningful by
ment purposes begins to break down.
controversy can be frac& to the
many leading authorities in educa-
multiple uses of testing data through-
tional measurement. Alternative
To tailr,r instruction to a stan-
out the educational system.
approaches, on the other hand, are
dardized test may at first glance seem
not yet fully developed, although
a desirable result; testing for essential
Student testing data has in the
innovafive efforts are prolifwating.
skills, for example, should (by the
past served two very different
The field of student assessment, in
logic of WYTIWYG) produce students
purposes: to guide instructic ial
short, is undergoing a profound
with essential skills. Upon doser
practice and to evaluate the practitio-
transformation at the very time that
inspection, however, WYTIWYG
ners. Teachers use test results for
the demand for achievement data is
contains a deeper, more sinister
diagnostic purposes, to identify what
growing ever more acute.
implication. It implies that the wrong
students know and what they need to
kind of test may have far-reaching,
learn. State and local administrators
This Brief is designed to explore
negative effects on the quality of
use test results for accountability
these issues in the context of existing
c1P.ssroom instruction. In particular, a
purposes, to identify strengths and
multiple-choice, "fill in the bubble"
weaknesses among programs and
type of examination may lead to
personnel.
Far West Laoratory for Educational
Trivial Pursuit-type instruction that
Research cad Development serves the
produces students who can memorize
Ordinarily, different kinds of
four-state region of Arizona,
well but are rarely challenged to
tests are used for these two purposes.
California, Nevada, and Utah,
exercise "higher-orders thinking
Teachers commonly devise their own
working with cducators at all levels to
skills: to think critically mid deeply; to
tests, trade with other teachers, or use
plan and carry out school
apply knowledge in novel situations;
tests presented in dassroom texts.
improvements. Part of our mission is
to integrate many discrete pieces of
Administrators, by contrast, tend to
to help state department staff, district
information; and to collaborate with
prefer commerciaily-produced
superintendents, school principals,
others in the solution of complex
instruments designed to compare the
and classroom teachers keep abreast of
problems.
performance of large numbers of
the best current thinking and practice.
2 Ritur ellPY AVAILARE
tasks that correlate reasonably well
cally to standardized, norm-refer-
Combine these two factors
with the desired skill. In authentic
enced, multiple-choice exams of the
high- stakes evaluative assessments
assessment, the test is the demonstra-
kind exemplified by the Iowa Test of
that end up driving instruction and a
tion of the desired skill itself.
Basic Skills.
testing instrument that reflects a
narrow subset of legitimate learning
Another central feature of
Authentic assessment differs
and instructional quality
objectives
Ilentic assessment is that it is
from traditional testing in several
to suffer seriously. Unfortu-
is likel
ic. A traditional test of writing
respects. A first and fundamental
nately, statewide assessment strategies
L ails by necessity isolates a large
difference pertains to the form in
commonly meet both of these criteria.
number of discrete bits of knowledge.
which the results are reported. Norm-
Direct writing assessment, however,
referenced tests give only relative,
State departments of education, for
requires the student actually to
comparative data: they compare the
example, typically use student test
produce an essay, in which the
performance of a given student with
scores to regulate flows of money to
countless bits of knowledge are
the norm established by the perfor-
schools, to establish standards for
blended into a single whole.
mance of his or her peers. Such
graduation or for entrance into special
results are commonly expressed as
programs, and to identify schools and
In part because they are holistic
the percentage of
percentiles
districts in need of some form of
authentic assessments are typically
students who scored better or worse
external intervention. Most states make
far lore difficult to score than are
than the student in question.
test results public, and the sheer
traci.tional tests. Indeed, the hallmark
attention focused on the comparative
and principal virtue of the standard-
Authentic assessment, by contrast,
performance of schools and districts
ized, multiple-choice exam is ease of
expresses results with reference to
serves as a powerful pressure on school
most are scored with great
scoring
actual, concrete skills that a student has
personnel. In addition to attaching high-
rapidity, little cost, and complete
or has not mastered. Such tests are said
stakes consequences, many statewide
reliability by machine.
to be criterion-referenced. A criterion-
assessment strategies continue to rely on
referenced test will tell whether Jason
traditional, norm-referenced, multiple-
In order to score an authentic
or Jennifer can or cannot add or
choice exams.
assessment, however, a team of
subtract fractions; a norm-referenced
human observers is generally re-
test will tell only whether they are
Traditional and Authentic
quired. Just as panels of experts score
doing better or worse than other
Assessment
the performances of Olympic gym-
students of their grade level or age.
nasts, a collection of teachers is
The advent of large-scale testing
ideally available to evaluate a
In ordL: to qualify as authentic
programs coupled with rewards arid
student's ability to conduct a debate,
assessment, however, criterion-
sanctions has brought a spotlight to bear
deliver a speech, or investigate a
referenced results repiesent only a
upon the deficiencies in traditional
scientific question. Collective obser-
first step. Another characteristic of
testing practices. A new generation of
vation and evaluation of this kind
authenticity pertains to the level and
alternative strategies for student
require extensive preparation in order
complexity of the skills under exami-
assessment is being designed specifi-
to coordinate scoring procedures and
nation. A paper and pencil test of
cally to redress these deficiencies. The
criteria.
musical ability might reveal a certain,
cornerstone of these new strategies is
narrow subset of proficiencies
authenticity: assessment is based on
While the art or science of
whether the student can correctly
students' performance in situations that
authentic assessment is no longer in
name the notes, for example. The
bear an authentic relationship to "real
its infancy, neither is it a mature and
actual performance of a composition
world" tasks. Writing essays, perform-
stable discipline. Its basic weaknesses
on the piano, by contrast, calls into
ing science experiments, participating in
are its cost, and the incompleteness of
play technical, kinesthetic, and
these are now
collaborative activities
its psychometric foundations. Yet
aesthetic abilities, in addition to the
the substance ci assessment practice
authentic assessments evaluate a
ability to decipher musical notation.
itself, rather e an the distant goal for
much fuller range of student abilities
which testing serves as a convenient
than is possible with a multiple-
Authentic assessment, therefore,
substitute.
choice exam, and the testing process
is both criterion-referenced and
makes much more sense to students
performance-based. In addition, it
A comparison of traditional with
and teachers. Authentic assessment
characteristically dispiays a quality of
authentic forms of assessment reveals
-.sks are designed to be complex,
immediacy, vitality, and meaning to
their respective strengths and weak-
integrated, and challenging; in this
the test-taker: it has intrinsic validity.
nesses. Strictly speaking, traditional
way, they mirror and support p oti
In most forms of traditional testing.
testing practices embrace a variety of
instruction. Due to this combix..ion
validity is a laboriously achieved
approaches to assessment. In the
of strengths and weaknesses, tradi-
product of finding isolated, artificial
following discussion, however, the
phrase traditional testing refers specifi-
2 Brief
The challenges Vermont faced in
Criterion-referenced tests
tional and authentic forms of assess-
shifting to authentic assessment were
those whose outcomes are stated with
ment will probably continue to
both greater than and different from
reference to specific skills -- are used
exist for some time to come.
those anticipated. Scoring, for ex-
in combination with norm-referenced
ample, proved to be surprisingly
exams in 25 states and are relied upon
The State of State Assessment
although the math
straightforward
exclusively in 10 others. Such tests are
teachers preferred a quantitative
most commonly employed in states
As of fall, 1991, 44 states and the
scale, while the English teachers
that have articulated specific learning
District of Columbia had some form
insisted on verbal descriptors.
objectives for various ages or grade
of mandated statewide testing
Nevertheless, all the participants
levels. Critenon-referenced results tell
program in place; of the remaining six
continuc 'o endorse the process.
how many students have met those
states, all but Nebraska had plans in
Vermont i ads the nation in another
particular learning goals.
progress. Reading, mathematics, and
respect Ls well: its assessment
language arts were each tested in 40
strategy is deliberately designed not
Among assessments that go
states or more. More than half of all
to include high-stakes consequences.
beyond the multiple-choice format,
states prescribed testing in writing,
writing samples are the most wide-
science, and social studies.
Michigan is a piweer in finding
spread, occurring now in some 28
ways to undertake performance-
states. A dozen states are moving
The most common usage of
based assessments with the minimum
toward performance-based assess-
statewide assessments is to monitor
possible cost. The state has developed
ment of matheriatics achievement,
the progress of individual students.
a battery of assessment programs on
and eight use such approaches for
For example, results may function as
an incremental basis, adding one or
secondary-level science.
a criterion for grade promotion.
two in different subject areas each
Seventeen states require students to
year since 1986. To date, performance
pass a stateWde minimum compe-
Pioneers: Vermont, Michigan and
assessments have been developed in
tency exam in order to graduate;
Kentucky
art, music, math, science, social
California, Delaware, and Virginia
studies, and physical edgcation.
Several states, including Ver-
require school districts to select and
mont, Michigan, and Kentucky, are in
implement such exams. Arkansas and
The key to carrying out such a
the process of reconstructing their
Virginia employ tests as an entrance
pI rgram on a low-cost basis is to find
strategies for student assessment.
requirement for high school.
people who strongly desire to partici-
Vermont is noteworthy for the extent
pate, and to enlist their assistance at
of its commitment to authentic
The next most common usage of
every stage of the process: develop-
assessment; Michigan for its leader-
statewide test results is for school-site
ment, administration, and interpreta-
ship in devising low-cost, perfor-
both to state-level
accountability
tion. In Michigan, administration of
mance-based testing programs; and
authorides and to the public at large.
the assessment procedures and
Kentucky for its radical revision of
Results are often published, for
interpretation of results are under-
the educational system as a whole.
example, in comprehensive reports of
taken by graduate students in each.
educational progress among schools
re$ ion of the state, as well as by
Vermont has completed the first
and districts throughout the state.
preservice and inservice personnel.
year of a pilot program of statewide
student portfolios in writing and
Finally, in 20 states, funding
A couro.room challenge to
math. The state allowed participating
decisions hinge upon results of
Kentucky's formula for financing
teachers to use their own judgment in
student assessments. In some states,
school districts had an unexpected
selecting students' best and most
low-achieving xhools receive added
outcome: the State Supreme Court
characteristic pieces of classroom
funds, while in others extra money is
ruled that the educational system as a
work for inclusion in portfolios. This
a reward to schools whose scores are
whole, "in all its parts and parcels,"
process worked reasonably well in
high.
was constitutionally invalid. The 1990
the assessment of student writing; but
legislatare faced the daunting task of
scorers of the math portfolios discov-
The majority of states continue to
mandating a new code for education
eTed that some forty pment of the
employ nationally normed, multiple-
from the ground up. In so doing, it
submissions consisted of worksheets.
choice exams, such ts the Iowa Test of
wrote into law a dual system of
Specification of more lifelike, perfor-
Basic Skills, the Stanford Achieve-
statewide assessment strategies: a
mance-based math activities is high
ment Test, and the Comprehensive
nationally normed, traditional test of
on the list of changes for this year's
Test of Basic Ski115. Because they are
academic skills was coupled with an
full-scale implementation of portfolio
norm-referenced, these tests allow
exte.isive, to-bt:-developed, perfor-
assessments in grades 4 and 11.
states to compare their students'
mance-based system.
performance with that of students
throughout the nation.
Brief 3
4
interact with actual instruction in
What distinguishes Kentuckys
significant and often unexpected
strategy, however, is not this dual
Briefs is published by the Far West
ways. These must be anticipated
Educational
approach, but rather the extremely
for
Laboratory
Research and Development. The
and monitored with care.
high stakes attached to site-level
publication is supported by federal
outcomes. The law mar:dates precise
funds flom the U. S. Department of
State demands for accountability,
levels of improvement in student
Education, Office of Educational
or "top-down" reform, could
percentiles that trigger release of
Research and improvement, con-
directly impede a major "bottom .
supplementary funds. Schools whose
tract no. 400-86-0009. The contents
up" reform strategy: the restora-
performance declines by set amounts
of this publication do not neccessar-
tion of authority to teachers and
ily reflect the views or policies of
incur an avalanche of penalties:
the Departrr ent of Education, nor
principals at the school site. Top-
parents are notified that their children
does mends:ill of trade names, com-
down regulation cf the system
are now fiee to transfer; all certified
merrial products or organizations,
may inadvertently contribute to
staff are automatically placed on
imply endorsement by the United
passivity and burnout among
probation; and a Kentucky "distin-
States Government. Reprint rights
those charged with the actual
guished educator" visits the site to
are granted with proper aedit.
delivery of educational services.
determine which employees, from
Staff
teachers through disti.J superinten-
The costs of state-mandated testing
dents, shall continue to hold their jobs.
Mary Amsler
must be considered. What fraction
Director, Policy Support
of the total educational budget
The bifu. ated nature of the
Services
does student assessment warrant?
assessment f e.lcl today is reflected in a
In an ea of scarce resources, this
pair of developments at the national
David Moody
decision must be weighed against
level. On one hand, The National
Senior Research Associate
alternative educational needs such
Assessment of Educational Progress
as funding for restructuring,
has recently entered upon the state-by-
James N. Johnson
professional development, or
state assessment of student achieve-
Communications Director
curricular innovations.
ment, using a traditional testing
format. It is now possible to compare
Fredrilca Baer
State-level support for assessment
the math achievement of eighth-
Administrative Assistant
research should be a corollary of
graders in Colorado with that in
the state-level demand for educa-
Maryland or Hawaii on a common
tional accountability. As the
scale.
demand for assessment data
continues to grow, policymekers
On the other hand, in a parallel
Briefs welcomes your comments
must recognize that the field of
initiative from the federal level, the
and suggestions. For additional
student testing is undergoing
Office of Educational Research and
information, please contact:
fundamental change. In order to
Improvement has funded a new center
ensure an effective transition, more
(under the auspices of UCLA's Center
James N. Johnson
research into the statistical under-
Far West Laboratory
for Resterch on Evaluation, Standards,
730 Harrison Street
pinnings and the instructional
and Student Testing) designed to help
San Franciscc, CA 94107
consequences of alternahv,:
states exchange information about
[phone]
assessment strategies is needed.
authentic, performance-based assess-
ments, such as the portfolio approach
The assessment of student
under development in Vermont.
academic achievement is evidently
not as straightforward a matter as the
Issues to Consider
evaluation of athletic accomplish-
FAR WEST LABORATORY
ment, for example. Learning is not as
The diversity of mandated student
FOR EDUCATIONAL RESEARCH
susceptible to objective observation
assessment programs across the 50
AND DEVELOPMENT
and measurement as is phy Acal
730 HARRISON STREET
states reflects competing viewpoints
sAN FRANCISCO, CA 94107
prowess, and high-stakes ( ducational
about appropriate ways to test stu-
[phone]
testing may interact with the very
dents. As policymakers re-think
system it is attempting to measure.
student assessment systems, they need
OBRI
The prudent policymaker would do
to consider a variety of issues:
well to tread gently in this sensitive
OFFICE OF EDUCATIONAL
territory, and to exercise caution in
Testing programs are not an
RESEARCH AND DEVELOPMENT
seeking to use assessment as a
U. S. DEPARTMENT OF
independent element in the educa-
EDUCATION
primary tool of reform.
tional system; on the contrary, they
4 Brief