ERIC ED377251: Two State Minimum Competency Testing Programs and Their Effects on Curriculum and Instruction.

42 Pages·1991·0.85 MB·English

by  ERIC

nOCUMENT RESUME
TM 022 543
ED 377 251
Wilson, Bruce L.; Corbett, H. Dickson
AUTHOR
Two State Minimum Competency Testing Programs and
TITLE
Their Effects on Curriculum and Instruction.
Research for Better Schools, Inc., Philadelphia,
INSTITUTION
Pa.
Office of Educational Research and Improvement (ED),
SPONS AGENCY
Washington, DC.
PUB DATE
91
43p.; Some pages contain light, broken type.
NOTE
Research/Technical (143)
PUB TYPE
Reports
MF01/PCO2 Plus Postage.
EDRS.PRICE
Academic Achievement; Curriculum Development;
DESCRIPTORS
*Educational Assessment; Elementary Secondary
Education; Field Tests; High School Graduates;
Instructional Effectiveness; *Minimum Competency
Testing; School Districts; *State Programs; *Testing
Programs; *Test Use
High Stakes Tests; *Maryland; *Pennsylvania; Testing
IDENTIFIERS
Effects
ABSTRACT
Effects of assessment on curriculum and instruction
were studied in an investigation that considered local educators'
reactions to statewide minimum competency testing, the instructional
effects of implementing these tests, and variations within each
state. The two states examined were Pennsylvania, a low-stakes
situation with relatively minor consequences for the student, and
Maryland, a high-stakes situation where high school graduation
depends on passing the tests. Pennsylvania students were tested in
grades 3, 5, and 8, while Maryland students were tested beginning in
grade 9. Fieldwork was conducted in six sites in each state, and over
250 local educators and students were interviewed. A survey completed
by 277 of 501 Pennsylvania districts and 23 of 24 Maryland districts
provided additional information. The impact of the testing program
was far greater in the high-stakes situation, and this impact seemed
positive as long as the districts were not under too much pressure.
Both positive and negative consequences are discussed. Five tables
present study findings. (Contains 14 references.) (SLD)
***********************************************************************
*
Reproductions supplied by EDRS are the best that can be made
*
from the original document.
***********************************************************************
to
PROGRAMS AND
TWO STATE MINIMUM COMPETENCY TESTING
INSTRUCTION
THEIR EFFECTS ON CURRICULUM AND
"PERMISSION TO REPRODUCE THIS
MATERIAL HAS BEEN GRANTED BY
T.-DoiveNoc
E TEIe
DEPARTMENT OF EDUCATION
Improvement
Office of Educat10001 Mamma erg
INFORIAATIO14
EDU¢ATIONA L RESOURCES
CENTER (EMI
Bruce L. Wilson
reOroduted as
Tars Document MS Mon
orgarsaltson
recewect from the Mrson or
°cpiloting rt
TO THE EDUCATIONAL RESOURCES
tO MOrov
.
Minor onsnoes nave been mad*
INFORMATION CENTER (ERIC)."
rproduCtion
Clualdy
H. Dickson Corbett
fh*O0Cf
Pornis of vNwo OOMOnsitlitadm
monsoon! otitclal
meat sto not nocoStanty
OE RI posttion or 00110y
Research for Better Schools
444 North Third Street
19123
Philadelphia, PA
from the U.S. Department
The preparation of this paper was supported by funds
The
(OERI).
of Education, Office of Educational Research & Improvement
policy of OERI,
opinions expressed do not necessarily reflect the position or
Appreciation is expressed to
and no official endorsement should be inferred.
earlier
Jane Hannawav and Bob Stake for offering helpful comments on an
version of the chapter and to Arlene Ziviello for word processing.
BEST COPY AVAILABLE
TESTING PROGRAMS AND
TWO STATE MINIMUM COMPETENCY
INSTRUCTION
THEIR EFFECTS ON CURRICULUM AND
of the states in this country have
Although close to sixty percent
testing (Marshall, 1987), debate continues
mandated some form of standardized
The
of implementing such testing programs.
about the local-level impact
clear and have not been well
effects of assessment initiatives are not
Stake,
(Airasian, 1987; Rosenholtz, 1987;
informed by empirical research
curric-lur
Little is known about how the
Bettridge, Metzer, & Switzer. 1987).
less is
by statewide standardized testing; even
and instruction are affected
and school district
known about how differences in state programs
This chapter is an effort to
the effects.
characteristics magnify or minimize
address those issues.
of
chapter is based belongs in the genre
The study upon which the
effects on local educational
research projects that examine assessment
of the intended and unintended consequences
agencies (LEA) - -that is, the study
The study
implementing assessment programs.
for curriculum and instruction of
the
(1) to gather local educators' reactions to
had three purposes:
competency testing in their
initiation of statewide, mandatory winimum
instructional effects of implementing
respective states; (2) to compare the
in two states; .and (3) to
these testing programs on local school systems
effects within each state.
explain district-to-district variations in
the first purpose are presented in
In this chapter, findings related to
Educators' responses to selected, individual
the form of a "Gallup Poll."
in the two states are reported.
items from a questionnaire administered
questionnaire items were combined
To address the second purpose, individual
adjustments to facilitate
into scales measuring various local system
Implementation effects were not uniform across the
between-state comparisons.
school systems within each of the two states and the third purpose of the
The remainder of this chapter
study was to explain these differences.
describes the testing programs in two states, presents the conceptual
framework that guided data collection and analysis, details the research
methods used, and summarizes the results.
The Testing Prozrams in Two States
The two states represented "low stakes" (Pennsylvan4a) and "high stakec"
The level of the stakes associated with a test is the
(Maryland) situations.
teachers,
extent to which test performance is perceived by students,
administrators, and/or parents to be "used to make important decisions that
Relatively minor
immediately and directly affect them" (Madaus, 1957:7).
Pennsylvania's minimum competency
consequences attended student performance on
The purpose of both tests originally was
tests (MCT) in lanzuaze and math-.
t.-
identify students needing additional classroom instruction who mav not have
Maryland's "high stakes" strategy required
been identified by other means.
students to pass reading, writing, math, and citizenship minimum competency
The tests were being phased
tests in order to receive a high school diploma.
in as graduation requirements; at the time of the survey only the reading and
math tests "counted."
The two states' MCT programs had several important differences (see Table
Second,
The first difference concerned the purposes detailed above.
1).
Pennsylvania students took their tests in the third, fifth and eighth grades.
Maryland tested students beginning in ninth grade, although a' ractice test
Third, the Pennsylvania state
was administered in the eighth grade.
legislature made a special appropriation to fund remediation efforts, whereas
Maryland offered no financial assistance for this purpose.
Fourth,
2
4
for educational
legislative response to the calls
Pennsylvania's program was a
after soliciting educators' input on
reform in the early 1980's and,
bid on
commerical test publishers were invited to
appropriate test objectives,
Maryland initiated a statewide
the state's instrument.
a contract to develop
testing
several years prior to beginning the
curriculum improvement program
the instructional quality
expressed purpose of anticipating
program with the
Educators from around the state were
wall on the tests.
necessary to perform
the tests.
into the content and form of
used by the SEA to provide input
Table 1
Comoetency,
Summary of Two Mandatory, Minimum
State Testing Program
STATE
PA
Areas of Difference
Reading, Math,
Reading, Math
TEST CONTENT
Writing, Citizen-
ship
8 (Practice)
3, 5,
8
GRADES TESTED
9, 10-12 Retests
Mandatory
Mandatory
PARTICIPATION
Identification of
Use of test results
STATE FOCUS
failing students
to identify students
to aid districts
in need of additional
in curriculum
instruction
planning
Students must
Additional funds
LOCAL CONSEQUENCES
pass test to
for low scoring
graduate; LEAs
students
required to pro-
vide appropriate
assistance to
failing students
3
7
In Penniylvania, the
during the study.
The programs' stakes changed
released district rankings based on the test
_Chief State School Officer (CSSO)
1937-88 school year and touted the test as an appropriate
:scores prior to the
Study interviews conducted subsequent to
indicator of school effectiveness.
the part of local educators that
this event revealed considerable concern on
which they were not originally intended,
the tests were being used in ways for
quickly withdrawn due to the furor surrounding
even though the rankings were
increased for both educators
Regardless, the importance of the tests
them.
its districts
Maryland had no similar dramatic event; instead
and the public.
day when all four tests would
had to reconcile themselves to the inevitable
tests generating
affect whether students graduated, with the two new
The difficulty students were
considerable controversy and calls for revision.
educators even under an
having passing the two tests augmented the pressure on
already high stakes condition.
Conceptual Framework
mandatory statewide testin2
The effects of introducing and operating a
adjustments in the local instructional
program were expected to require
An underlying assumption of this study
program, organization, and culture.
had far-reaching ramifications for the
was that the mandatory testing programs
depending upon
technology, structure, and values in place in school systems
This chapter looks at instructional
iSee Figure 1.)
what was at stake.
adjustments--specifically at the strategies devised by a district to improve
and teaching intended to
test scores and at modifications to curriculum
The other two adjustment
improve-the match between course and test content.
categories in Figure 1 are examined only to the extant that they help
understand variations in the instructional adjustments.
Whether or not adjustments actually occured was partially dependent on at
Summarized in Figure
least two aspects of a system's operating environment.
selected features of school district
context, and
1, these aspects were:
(1)
state testing program.
(2) characteristics of the
Figure 1
SYSTEM
SYSTEM
SYSTEM
EFFECTIVENESS
ADJUSTMENTS
ENVIRONMENT
1. Instructional
1. Student Focus
1. District Context
test scores
strategies
internal contextual and
demographic characteristics
dropout rate
0 curriculum &
instruction attendance
district-SEA relationship
post-school plans
2. Organizational
2. Teacher Focus
I. State Testing Program
job satisfaction
information flow
high or low stakes
* benchmark,
* comtitment
o enga.aement
3. Cultural
quality of work life
quzlity of student life
With respect to school district context, years of research cn educational
change point to an inescapable conclusion:
some programs work some times in
some places, and it is mostly the time and the place that explain the fate of
a program (Berman, 1981; Corbett, Dawson, & Firestone, 1984).
Both Elmore
(1980) and Berman. (1981) argue that policy implementation can only be
understood in terms of the context of the "target's!' setting; policy makers'
intentions become diffused and redirected as they pass through the prism of
local politics, organization and culture.
Thus, changes in the test scores
over time were assumed to be the product of the complex interaction among
system demographic and internal contextual characteristics, its relationship
with the external environment--particularly the state eudcation agency (SEA),
and the kinds of adjustments the system made to implement the tests.
Features of the state testing program also would influence the type and
The essential
magnitude of local system adjustments that were made.
difference in this study was that the program in Maryland made graduation from
high school dependent upon a student's passing writing, citizenship, reading,
In Pennsylvania,,the test was intended formally to serve as a
and math tests.
tool for fine-tuning classroom instruction to meet certain students'. needs.
Thus, the study compared Maryland's high - stakes program having consequences
According to
for graduation to Pennsylvania's low-stakes MCT testing prozram.
Madaus (1987), "high-stakes" programs are used for important decisions and
thus have the power to modify local behavior; "low-stakes" programs are
generally not anticipated to be central to decision-makinz, and test
The
performance usually does not stimulate significant rewards or sanctions.
two states were selected for study to accentuate this high- stakes
/low - stakes
distinctisn.
There are several reasons why higher stakes situations can be expected to
First, mandatory tests are likely to force
have greater local impacts.
adjustments in a system by creating expectations for what the outcomes of
According to Mintzberg (1983), stipulating outcomes is
schooling should be.
Some standard--
one means used widely in organizations to affect operations.
no matter how narrowly defined--is to be met, regardless of what else staff
members may want to accomplish.
Second, one of education's fundamental tasks is to move students smoothly
Staff
through a series of grades to graduation (Schlechty, 1976).
responsibilities, the number of classrooms needed, and the availability of
sufficient materials are all predicated in most communities on the assUmpticn
that most first graders will become second graders and that most seniors will
A few exceptions cause no problems, but testing programs
graduate on time.
BEST con;
AvAII.ARI F
6
unpredictable checkpoint for
change the assumptions by inserting an
students that is based on something other than
determining progress for all
in school.
student age, credits obtained, or time spent
students must meet as a visible
Third, establishing a standard all
the ethos of many educators
indicator of effectiveness runs counter to
of
In spite of enormous standardization, a tone
(Rosenholtz, 1987).
Teachers are
(Lortie, 1975).
individualism permeates American education
what and how to teach, and they
allowed considerable autonomy in determining
Testing programs, therefore,
their own.
expect to handle their classrooms on
and instruction decisions.
challenge an ingrained ethos concerning curriculum
test administration dates
Test items highlight critical content to cover,
item formats affect how the
determine the deadline for teaching the content,
add a quality of sameness to
information will be accessed, and the standards
what students should achieve.
additional
The conceptual framework (Figure 1) also points to an
Have the instructional adjustments made the district more
important question:
examination
Narrowly conceived, this question merely suggests an
effective?
the standards set by the
of a district's success in helping students meet
definitions of
However, it is becoming more and more clear that
test.
dependent
effectiveness and the extent to which they are shared are context
Effectiveness, thus, may be defined
(Rossman, Corbett, & Firestone, 1988).
stimulates
More by how well a system prevents dropouts, improves attendance,
differences than by
student enthusiasm fol learning, or addresses student
A study of this magnitude is not an appropriate
doing better on a test.
While the study does tap perceptions of
vehicle for answering this question.
focus is on explaining system
a district's reach for improvement, its major
adjustments, not the ultimate effectiveness of the testing programs.
Study Design
simplifies a very complex situation.
The conceptual framework
involves a
mandatory statewide testing program
Introducing and operating a
Many of these can be
challenges to a district.
wide range of potential
.
However;
above.
framework such as the one
deduced from a conceptual
take advantage of
in which the research can
using an inductive approach
Fcr
& Huberman, 1980.
be equally valuable (Miles
Unexpected developments can
open-ended qualitative
designed to include both
this reason, the study was
questionnaires.
fieldwork and structured
of
First, a preliminary round
three phases.
The study was conducted in
visited each of 12
performed wherein researchers
qualitative fieldwork was
wide variety of staff
several days to interview a
school districts for
interviews were used to design a
Second, the results from the
members.
studied.
throughout districts in the states
questionnaire to be administered
round of feedback and
used to structure a final
Third, the survey results were
originally visited.
interviews in the sites
Fieldwork in 12 sites
Phase One:
Site selection was
the two states were visited.
Six sites in each of
served, primarily
size and type of community
made on the basis of district
staff
assumed to determine the kind of
because these characteristics were
Equally important was the
implementing the test would make.
resource demands
of this phase
participate because the purpose
willingness of the district to
larger population.
in depth, not to generalize to a
was to explore issues
staff
the input and assistance of key SEA
Selection was carried out with
members in each state.