ERIC ED362568: Analysis of State Education Indicators: State Profiles and NAEP Results Related to State Policies and Practices, 1993.

81 Pages·1993·3.1 MB·English

by  ERIC

DOCUMENT RESUME
TM 020 637
ED 362 568
Analysis of State Education Indicators: State
TITLE
Profiles and NAEP Results Related to State Policies
and Practices, 1993.
Council of Chief State School Officers, Washington,
INSTITUTION
DC. State Education Assessment Center.
PUB DATE
93
NOTE
81p.
Council of Chief State School Officers, State
AVAILABLE FROM
Education Assessment Center, One Massachusetts
Avenue, N.W., Suite 700, Washington, DC 20001-1431
($12.50).
Evaluative/Feasibility (142)
Reports
PUB TYPE
MF01/PC04 Plus Postage.
EDRS PRICE
*Achievement Tests; Educational Policy; *Evaluation
DESCRIPTORS
Methods; Grade 8; Junior High Schools; Junior High
School Students; *Mathematics Achievement; National
Surveys; Outcomes of Education; *Profiles; School
Statistics; *Secondary School Mathematics; State
Boards of Education; State Programs; State Surveys;
Tables (Data)
*Educational Indicators; *National Assessment of
IDENTIFIERS
Educational Progress
ABSTRACT
With this volume, the Council of Chief State School
Officers introduces a new 2-year format for ,.tate education
indicators. The first section of this report is an analysis of
state-level data related to the first-ever state-by-state achievement
results from the 1990 mathematics assessment of the National
Assessment of Educational Progress (NAEP). The analysis includes
background characteristics, program inputs and policies, and
educational outcomes. The second section of the report provides
profiles of each state on a number of available indicators. Several
states are still presenting a mathematics curriculum that heavily
emphasizes numbers, operations, and measurement over concepts in
geometry and algebra functions. The analysis aiso makes it apparent
that states differ widely in the proportion of teachers who are
certified in middle school or secondary school mathematics, and that
these teachers vary widely in their college preparation. Findings
also indicate that use of calculators does not deter students'
mathematical thinking and that grouping students for instruction does
not significantly affect mathematics proficiency. A final finding is
that the availability of instructional materials and resources is
related to mathematics proficiency. Eleven figures and 10 tables in
included.
an appendix present study findings. State profiles are
(Contains 17 references.) (SLD)
***********************************************************************
*
Reproductions supplied by EDRS are the best that can be made
*
from the original document.
***********************************************************************
'744Firgrs,,17.7.:
4
19*41rrlif
U S DEPARTMENT OF EDUCATION
Coke nI Educatonal Research end omprchement
Fouc TIONAL P.-SOURCES INFORMATION
CENT; R IERICI
his document has been reproducel as
received horn the person or organaarrOo
".1/41-44,!.91
ortrnath-y
0 Mmo changes have been made to mwrove
r eproduction Que.rly
Pchnts ot view or opiruons stated in tMsdocu
ment do not necessanly represent ort,..al
OERI pos.hoh or policy
"PERMISSION TO REPRODUCE THIS
BY
MATERIAL HAS BEEN GRANTED
RESOURCES
TO THE EDUCATIONAL
INFORMATION CENTER (ERIC)"
".r.r
'
,
),4
'
174,
z-
'
"
'.<
1,?.5;
+AN,
s ,
411111b..2
,f
14..?
.1_.!ILE
r:J:iirf '7)f
2
The Council of Chief State School Officers (CCSSO) is a nationwide
of
COUNCIL OF CHIEF SI-11'E
non-profit organization of the 57 public officials who head departments
SCHOOL OFFICERS
public education in every state, the District of Columbia. the Department
CCSSO
of Defense Dependents Schools, and five extra-state jurisdictions.
WERNER ROGERS (Georgia)
seeks its members* consensus on major education issues and expresses
Pre.sident
their views to civic and professional or2anizations, to federal agencies. to
ALAN MORGAN (New Mexico)
President-Elect
Congress, and to the public. Through its structure of standing committees
and special task forces. the Council responds to a broad range of concerns
THOMAS BOYSEN (Kentucky)
Co-Chair. Task Force on Teacher and
issues.
about education and provides leadership on major education
ctudent Assessment
Because the Council represents each state's chief education adminis-
TED SANDERS (Ohio)
establishment in
trator, it has access to the educational and governmental
Co-Chair. Task For cr on Teacher and
student Asse3sment
each state and to the national influence that accompanies this unique posi-
tion. CCSSO forms coalitions with many. other education organizations
GORDON AMBACH
Executive Director
and is able to provide leadership for a variety of policy concerns that
affect elementary and secondary education. Thus, CCSSO members are
RAMSAt SELDEN
Director, State Educanon Asse.ssnient
to the education of America's
able to act cooperatively on matt i.
Center
young people.
Counc.l of Chief State School Officers
The State Education Assessment Center leads efforts by states
One Massachusetts Avenue. NW
individually and collectively to enhance the breadth. quality, and compa-
Suite 700
Washington. D.C. 20001-1431
of that
rability of information about education and to improve the use
202/408-5505
The
information by educators, educational policy-makers, and the public.
frameworks for state-
Center conducts projects to develop the consensus
th1.1 report Mal he ordered for
Copte.8
Progress.
It
bv-state testing in the National Assessment of Educational
SI2.50 per ccpy from
help states collab-
also rum consortia in student and teacher assessment to
Council of Chief State School Officers
11 encourages
kie\Llopment of -tate-of-the-art assssment
rat,.
mate Education ssessment Center
the establishment of standards for American education. so these assess-
Onc Massachusetts Avenue. NW
Suite 700
undamental societal lodgements of
inenl 1''OI!2rains can be anchored on t
Washington, D.C. 20001-1431
conduct projects to improve
1/4 hat students should learn. The Center also
dointz in
statistics and other indicators of how well the school s stems are
prep:n.1112 students.
15/0?.
With this volume, we introduce a new, two-year format for State Education Indicators. This year we have
prepared a new analytical report. The first section is an analysis of state-level data related to the first-ever
state-by-state achievement resultsfrom the 1990 NAEP mathematk.'s assessment. The analysis includes a
comprehensive set of educational indicators: background characteristics, program inputs and policies, and educa-
tional outcomes. The second section of the report provides profiles of each state on a number of available indicators.
Next year, we will issue a compendium of key state statistics organized by indicator as we have reported state indica-
tors since 1987.
The decision to try this two-year format was guided by several factors. First, we convened meetings of advi-
the field of educational assessment,
sors who recommended producing a more analytical report. Second, because
analysis. Third,
indicators, and accountability has progressed, it is now possible to undertake a more sophisticated
conclusions beyond earlier attempts.
we believed it was now possible to provide interpretations and
The states have the major role in policy decisions to advance educational quality. They have a decade and a
local and
half of policy-based reform efforts and are moving to "systemic" strategies aimed at comprehensive state,
school actions.
We search for interpretations and causal relationships between programs, characteristics and results, yet
there are great limitations in analysis of what works based on mixed and partial sets of indicators. Good data are
jurisdictions (partici-
available in some areas but not others. We are working with fewer than forty states and other
from tightly
pating in the 1990 NAEP), limiting the conclusions that can be reached. Indicators are different
the state level
designed research in the conclusions they can support. Nevertheless, it is important to look at results at
what does not.
in relation to the conditions under which they are obtained, to try to discover what works and
for
The first part of this report attempts to do this. Working from the 1990 NAEP mathematics results
high or low performing
several states, we try to find patterns in characteristics related to the achievement results. Are
curriculum? Are teacher qualifications and
states different in the extent to which they seem to cover aspects of the
interpretations are
experiences different? How should we factor in socioeconomic differences? Some preliminary
analyze
reached. But, the limitations of these interpretations must be understood. This is a pioneering attempt to
factors related to achievement results in one subject from the states' perspective.
We very much
John Dossey drafted the analysis and worked with us in preparing this section of the report.
the report. We will appreciate
appreriate his contribution. State data-collection staff and other experts have reviewed
achieves the purpose of adding value to indi-
comments or suggestions readers can provide. We hope this new format
cators in informing policy decisions.
RAMSAY SELDEN
atad2e/ igeeMpgaZo.
CADELLE HEMPHILL
)9.4/pp 14.14
-OLP
BLANK
Curriculum and Evaluation
The national focus on
Standards for School
educational reform has often
Mathematics (1989) and the
centered on school mathe-
Professional Standards for
matics. This scrutiny is a
Teaching Mathematics (1991)
result of several factors. The
What Do the
have provided a focus for
first is undoubtedly the crucial
NAEP Math Results
reform. A third reason for the
centrality of mathematics
education in reform is the
Mean for States?
number of recent comparative
role that the mathematics
studies of international
curriculum plays as distributor
achievement in mathematics
of opportunity. This opportu-
In school mathematics the United States is an under-
(Husen, 1967; Travers &
nity to learn mathematics is
achieving nation, and our curriculum is helping to
Westbury, 1989; Robitaille &
tightly tied to a child's oppor-
what we
create a nation of underachievers. We are not
Garden, 1989; Lapointe.
tunities in life (Steen. 1989).
ought to he; we are not even close to what we can he.
It is a time for changea time to renew school mathe-
Mead, & Phillips, 1988;
..und factor is the leading
A
matics in the United States.
Lapointe, Mead. & Aske
role that the mathematics
1992). as well as recent results
education profession has
MCKNIGHT ET AL., 1987
from the National Assessment
played in the development of
of Educational Progress
standuds for curriculum, eval-
By the year 2000. U.S. students will he first in the
(NAEP) in mathematics
uation, and teaching. The
world in science and mathematics achievement.
(Dossey, Mullis, Lindquist, &
development, release, and
ALEXANDER, 1991
Chambers, 1988; Mullis,
growing acceptance of the
Dossey, Owen, & Phillips,
National Council of Teachers
1991).
of Mathematics' (NCTM)
This report examines data
John Dosser. Professor of
on the mathematical education
Mathematics at Illinois State
University, vi as the primary
of American 8th graders.
author qf this section. with assis-
working from a state-level
tance from Rolf Blank,
CCSSOISEAC.
perspective. It examines 1990
NAEP data collected in both
the assessment of the nation
5
2 State Education Indicators-1993
CURRICULUM
the pattern in countries that do
The resulting data
and the trial assessment of the
better in achievement. At the
pro..ide perhaps the strongest
From the time of the First
states (Mullis et al.. 1991),
8th grade level in the United
indicators, to date, of the
International Mathematics
supplemented with informa-
States, the breadth of the
health of U.S. school mathe-
Study (Husen. 1967), there has
tion on state programs and
curriculum eilst be expanded
matics and the factors that
been a great deal of interest in
other state characteristics
to encompass more topics than
help shape it. Special
the influence of student oppor-
drawn from other sources.
the historical emphasis on
emphasis is given in the
tunity to learn, or curriculum
The NAEP proficiency
arithmetic (numbers and oper-
analyses reported here to the
coverage, on achievement.
data cover student perfor-
ations) and measurement. The
nature of the 8th grade
A variety of measures has
mance in five content areas of
NCTM Standards call for all
curriculum, to teachers' back-
been used to assess student
mathematics: numbers and
students to see a mathematics
grounds, to the schools'
curricular expos'
since that
operations; measurement;
curriculum that also considers
instructional programs, and to
time (Robitaille & Garden,
geometry: data analysis, statis-
data analysis, geometry, and
the policy contexts in which
1989; Travers & Westbury,
tics. and probability; and
the study of algebra and func-
students mathematics educa-
1989). A major facet of the
algebra and functions. The
tions (NCTM. 1989).
tion takes place in the states.
opportunity-to-learn question
test items were developed and
These variables were selected
is not only the coverage but
reviewed by mathematics
based on patterns in the
the "intensity" given a topic
educators, measurement
national NAEP results and
during a year of study.
specialists, and representatives
questions raised by them in
Results of the Second Inter-
of the states involved in the
the mathematics education and
national Mathematics Study
trial state assessment. These
policy communities (Mullis et
(McKnight et al.. 1987)
items were administered to
al., 1991). These also are
suggest that beyond opportu-
nearly 7.000 students in public
factors that are under the
nity, in general. the ability of a
and private schools nationally
country to focus heavy
control of state policymakers.
and to approximately 2,500
emphasis on a topic of instruc-
8th grade students drawn from
tion within a year may be
about 100 schools in each of
preferable to diffusing the
40 participating states and
same amount of instruction
territories (Mullis et al.. 1991).
over a period of years. That is
6
1993State Education Indicators
3
Islands. Alabama. Guam,
emphases in
both
The NAEP data were
Teacher Emphasis on
Areas of Mathematics
Arkansas, Kentucky, and
numbers/operations and
analyzed by state to determine
Florida, in decreasing order.
In the 1990 NAEP
measurement may be offering
if there are patterns among the
Mathematics Assessment, 1 he eight states giving the
rather traditional programs of
states in teacher emphasis on
lowest proportion of their
teachers of the 8th grade
study, while those giving
areas of the math curriculum
students a heavy emphasis in
students in the study were
heavy emphasis to the geom-
and to determine if these
these areas were Colorado,
asked to indicate the degree to
etry and algebra/ functions
differences by state are related
which they had given heavy,
Oregon, Wisconsin,
areas at the 8th grade level
to differences in student math
moderate, or little or no
Minnesota, Wyoming,
may be moving toward a
proficiency.
Montana. New Hampshire.
emphasis in their mathematics
richer, more ambitious
First, several of the
and Nebraska.
instruction curriculum to:
program of study, similar to
curriculum areas were found
Figure 2 shows the
numbers and operations;
that called for by the
to be interrelated. The state-
proportion of students in a
measurement; data analysis
level analysis showed a strong
Standards.
state receiving heavy
and statistics; geometry; and
Results of the state-by-
correlation (r = .93) between
emphasis in numbers/
algebra and functions. The
state analysis of the 1990
the percentage of students
operations and measurement.
initial report of the results
NAEP data showed that the
receiving heavy emphases in
with the states ordered
showed that at the national
relative emphasis that teachers
numbers/operations and in
according to the
level there is a strong associa-
state rank on
in a state give to different
measurement. There is also a
tion between the topics
the average math proficiency
areas of the 8th grade math
high correlation (r = .81)
(see Table 1 in the
teachers emphasize and
score
curriculum is strongly related
between the percentage of
Appendix for state scores).
student proficiency in those
to the level and type of math
students receiving heavy
The state percentages for
areas (Mullis et al., 1991).
proficiency of students in the
emphases in geometry and in
emphasis on numbers/opera-
Students tend to do better in
state. Figure 1 shows an
algebra/functions. States
tions and measurement are
one of those five areas when
ordering of states according to
providing large percentages of
expressed by quintile aver-
teachers emphasize it. whether
the level of teacher emphasis
their students with heavy
ages. The bar graph shows
it is numbers and operations or
on numbers/operations and
that states with higher profi-
algebra and functions. In
measurement. Eight states Or
ciency tend to have fewer
other words, we tend to do
territories gave the largest
students receiving heavy
better where we place our
proportions of their students a
heavy emphasis in numbers/
effort.
operations and measurement:
Georgia, Texas. the Virgin
4
State Education lndirarors-1993
I '#-ZiffirgiiiilliNNIMMENEINsw
Figure
Percent
fir
students with Teachers Emphasizing
Numbers/Operations and Measurement
GA
TX
VI
AL
GU
AR
KY
FL
NJ
DC
Figure 3 shows the
emphasis on numbers/ opera-
NM
LA
results of ordering states
tions and measurement (corre-
CT
OK
according to the percentage of
lation = -.632, which is
NC
OH
significant at the .05 level of
students whose teachers
R I
IN
reported heavy emphasis on
statistical significance). The
DE
geometry and algebra/func-
states in thc top quintile of
N
IA
tions. The eight states
math proficiency (North
PA
AZ
reporting the most emphasis
Dakota. Montana ... Wy-
CA
WV
on these areas of the
oming had an average of 27
HI
ID
curriculum were New Jersey,
percent of students receiving
VA
IL
Texas. New York, Montana,
curriculum emphasis on
NY
MI
Illinois, North Dakota, New
numbers/operations and
MO
NE
Mexico, and Georgia. (Some
measurement, while the states
NH
MT
states report high emphasis in
in the bottom quintile of math
WY
MN
both "types" of curriculum.)
proficiency (Alabama.
WI
The eight states giving the
Hawaii ... the Virgin Islands)
OR
CO 0.1.17':.
lowest average emphasis to
had an average of 37 percent
50
10 15 30
5 20 25
0 40
45
Percent of Students (Nation3A%)
these areas were Hawaii.
of students receiving emphasis
Source: U.S. Deportment of Edurethon, National Center for Education Statistics 'The
State of Mathematics Achievement HAEP's 1990 Militiamen( of the Natlon and the Trial
Arkansas, West Virginia.
in these areas.'
Assessment of the States' June 6, 1991. CONICS of Chief Stais School Moen, State
Education Assessment Center. Washington. D.C.. Fail 1992.
The statistical analysis of NAEP
Figure 2
results by state showed that the
Percent of Students with Teachers Emphasizing
socioeconomic status (SES) back-
Numbers/Operations and Measurement by State Rank on
ground of students is strongly
Overall Math Proficiency
(Quintile WI Average)
related to average math profi-
North Dakota, Montana, Iowa, Minnesota,
ciency and to curriculum
Nebraska, Wisconsin, Now Hampshiro, Idaho, Wyoming
01 l=
IIII
emphasis of teachers. and the
116111111111111111
Oregon, Connecticut, New Jersey,
average SES of students in a st(lte
Colorado, Indiana, Pennsylvania, Michigan, Ohio, Virginia
can account for most of the vari-
IIIIMIIIIIMIIIIIIINMIIIIMIIIIIMIMMIII
02
ance in math proficiency. The
Oklahoma, Delaware, Naw York.
Illinois, Maryland. Rhode Island, Arizona
analysis in this section attempts to
03 111111111111111111111111111111111111151111111111111111111
show differences in curriculum
Georgia, Texas, Arkansas.
emphasis of teachers by state.
Californm. Kentucky. New Mexico, West Virginia, Florida
wah the understanding that at
04 1111111111111111111111111111111111111111111116MINE111111111111111111
least part of the state differences
Alabama, Hawaii. North Carolina.
Louisiana. Distnct of Columbia. Guam. Virgin Islands
are due to the influence and
05 11111111111M111111111MONIMIll IN1111111111111 111111111 III
expectations of parents and the
25 40
30 35
20
10 15
5
o
school community (represented by
Percent of Students (Nation-36%)
a measure of SES).
Sou:re: U.S Department of Educatton, Nalttnal Center for &fur-aeon Statisks. 'The
State of Mathematics Achievement: NAEP's 1990 Auessment of the Nation and the Trial
Assessment el the States" June 6, 1991 Council of Chief State School Officers State
BEST COPY AVAILABLE
Educatsn Assessment Canter, Washington. D.C., Fall 1992.
5
K2
Figure 3
Percent of Smdents with Teachers Emphasizing
Geometry and Algebra/Functions
NJ pip
TX
NY
MT
ND
NM
Delaware. the Virgin Islands.
GA
tions, whereas the states in the
CT
Guam. Indiana. and Rhode
bottom quintile of math profi-
NH
IA
Island.
ciency had an average of 31
LA
MO
What can be determined
percent of students receiving
OH
OK
about the relationship between
emphasis in these areas.
DC
KY
teachers curriculum emphasis
A third step in looking at
CA
CO
and student performance?
the relationship between math
VA
ID
Figure 4 shows the proportion
curriculum emphasis and
NE
MN
of students receiving heavy
student proficiency is a statis-
AL
MI
emphasis on geometry and
tical "cluster analysis-
AZ
algebra/functions, with the
(Wilkinson. 1989). We used
PA
WI
states ordered according to
such an analysis to look at the
WY
OR
average math proficiency
relationship between three
NC
FL
score. The bar graph indicates
variables: a) average state
RI
)14
that states with higher profi-
math proficiency, b) the per-
GU
ciency tend to have more
centage of students who
students receiving heavy
receive heavy emphasis on
emphasis on geometry and
numbers/ operations and
40
30 50
35
20
45
25
15
5 10
algebra/functions (correlation
measurement. and c) the
Percent of Students (Nation...39%)
Source: U.S. Department ol Education. National Center for Education Statistics. 'The
= .335, which is significant at
percentage of students who
State of Mathematics Achievement: NAEPS 1990 Assessment of the Nation and tho Trial
Council of Chief State School Officers. State
Assessment of the States.' June 6. 1991
Education AsseSsment Center. Washington, D C.. Fall 1992.
the .05 level of statistical
receive heavy emphasis on
significance). The states in
geometry and algebra/
Figure 4
Percent of Students with Teachers Emphasizing
the top quintile of math profi-
functions.
Geometry and Algebra/Functions by State Rank on Overall
Math Proficiency (Quintile [Q] Average)
ciency had an average of 37
North Dakota. Montana, Iowa, Nebraska,
percent of students receiving
Minnesota. Wisconsin. New Hampshire, Wyoming, Idaho
01 IIMMINIE
1111111111111
Lurriculurn emphasis on
Oregon. Connecticut. New Jersey,
Indiana. Colorado. Pennsylvania. Virgtnta. Ohio. Michigan
geometry and algebra/func-
0211111111111111111111112V111111111111111111111=11111111111111111111111
Oklahoma. New York. Delaware,
,
Rhode Island, Maryland. Mors. Arizona
i
111111111151111111111111N MEG 11511111
o311111111111111111111
Texas. Georgia. West Virginia.
New Mexico, Kentucky, California. Arizona. Florida
04
111111111111111111111111111111111111111111111111111i=
Alabama, Hawaii, North Carolina,
Louisiana. Guam. Distncf of Columbia, Virgin Islands
33 34
32
30
29
31
28
Percent of Students (Nation-39%)
Source: US. Department of Education. National Cantor for Education Statist=. 'The
Stale of Mathematics Achievement' NAEP's 1990 Assessment of the Nation and the Trial
Assessment of the States.' June 6. 1991. Council of Chat State School Deicers. State
Eoucation Assessment Center. Washington, D C.. Fall 1992
that 8th grade teachers have a
specifically in mathematics
(mean = 253). The states in
The results revealed that
broad range ot coUrtiessork in
tor teachers teaching
cluster three had medium
states tell into three clusters.
mathematics content.
mathematics.
emphasis on numbers/opera-
The t irst cluster had low
To measure teachers'
Preparation tor teaching
tions and measurement and
reported teacher emphasis on
preparation against this stan-
mathematics at the sth grade
bit, emphasis on geometry and
numbers/operations and
dard. the NAEP teacher ques-
level falls at a critical immure
algebra/functions and also had
measurement and medium
tionnaire asked teachers about
in the ile.ittn of teacher educa-
lots er average math prof 1-
reported emphasis On geom.
their certification status and
tion and certification. In most
ciency (mean = 254) than
etry and algebra/functions.
their collegiate coursework in
states, teachers of 8th grade
states in cluster one. Thus, the
The average math proficiency
set en areas of mathematics
mathematics :ire required to
results 01 the statedet el
ol states in cluster one st as
and computer science:
have some inatheinatics
analysis show that curriculum
significantly higher (mean =
number systems and numer-
education beyond that of the
emphasis iii the classroom
2(19) than the average for
ation, geometry. probabilit)
basic elementary education
seems to he related to dif ler-
states in the other ttto
and statistics, abstract or linear
degree I ts hich is usuidly Its 0
ences Ifl student inath profi-
slates tt ith
The I
clusters.
algebra, calculus, computer
mathematics content courses
ciency as tested in NAEP.
the highest math proficiency
science, and computer
and one course in the methods
ot erall %%ere in this cluster:
TEActiER
progranuning.
if teaching mathematics). hut
North Dakota. Montana. lost a.
PREPARATION IN
the typical state does not
Nebraska. Minnesota.
NIATHENIATICS
require a minor or mator in
Wisconsin. New Hampshire.
NAEP provides several
mathematics (Blank &
Wyoming. Idaho. and Oregon.
usetul indicators ot the extent
Dalkilic. 19921. The
States in the second
to ts hich teachers are prepared
Pri4e.+Aunial Stamhirds /fir
luster !lad noire emphasis
to teach 8th grade mathe-
Teoi.hint.! Alathematus
than those in cluster one On
matics. One indicator of
NCT M. 19911 recommend
numbers/operations. measure-
particular relet ance for states
ment. and geometry and
is the percentage 01 teachers
algebra/I-um:11,ms and /int e
M ith state certification in
at erage math proliciency
mathematics. \ irtually
teachers are cerutied to teach
in some field or grade level.
This analysis considers the
role ot teacher certification
0
ators
7
/993Stale Eall«1(10/1