ERIC ED445914: Teaching Practices and Student Achievement: Report of First-Year Findings from the 'Mosaic' Study of Systemic Initiatives in Mathematics and Science.

105 Pages·2000·1.2 MB·English

by  ERIC

DOCUMENT RESUME
ED 445 914
SE 064 086
AUTHOR
Klein, Stephen; Hamilton, Laura; McCaffrey,
Daniel; Stecher,
Brian; Robyn, Abby; Burroughs, Delia
TITLE
Teaching Practices and Student Achievement:
Report of
First-Year Findings from the 'Mosaic' Study
of Systemic
Initiatives in Mathematics and Science.
INSTITUTION
Rand Corp., Santa Monica, CA.
SPONS AGENCY
National Science Foundation, Arlington, VA.
ISBN-0-8330-2879-0
ISBN
PUB DATE
2000-00-00
NOTE
104p.
CONTRACT ESI-96-15809
AVAILABLE FROM
RAND, 1700 Main Street, P.O. Box 2138, Santa Monica, CA
90407-2138; web site: http://www.rand.org
PUB TYPE
Books (010)
Reports - Descriptive (141)
EDRS PRICE
MF01/PC05 Plus Postage.
DESCRIPTORS
*Academic Achievement; Critical Thinking; *Educational
Change; Elementary Secondary Education; Mathematics
Education; Problem Solving; Piofessional Development;
Science Education; *Teaching Methods; Urban Schools
IDENTIFIERS
National Science Foundation; *Systemic Educational Reform
ABSTRACT
This book reports on the first wave of the Mosaic Project
which investigates the effects of educational change and teaching practices
in mathematics and science on student achievement. The Mosaic project was
supported by the National Science Foundation (NSF) and featured two waves.
The states that participated in the first wave were State Systemic
Initiatives (SSIs; in Connecticut and Louisiana; Urban Systemic Initiatives
(USIs) in Columbus, OH and San Francisco, CA; and Local f3ystemic Change (LSC)
projects in Fresno, CA, and El Paso, Socorro, and Ysleta, TX. This document
contains four chapters:
"Introduction: The Systemic Initiatives Programs,
(1)
Earlier Evaluations of Systemic Initiatives, Evidence of Relationships
between Teaching Practices and Achievement, Measuring Student Achievement,
Overview of the Mosaic Project";
(2) Methods: Site Selection, School,
Subject, and Grade-Level Selection, Data Collection, Participation Rates,
Analysis";
(3) First-Year Results: Distribution of Teaching Practices,
Relationships between Teaching Practices and Achievement, Alternative
Formulations for Site Models, Differences between Test Formats, Study
Limitations"; and (4) Discussion: Summary of Year 1 Findings, Plans for
Future Data Collection and Analysis". (Contains 33 references.)
(YDS)
Reproductions supplied by EDRS are the best that can be made
.
.
Teaching Practices
and Student Achievement:
Report of First-Year Findings
from the "Mosaic" Study
of
Systemic Initiatives in
Mathematics and Science.
Stephen Klein
Laura Hamilton
Daniel McCaffrey
Brian Stecher
Abby Robyn
Delia Burroughs
U.S. DEPARTMENT OF EDUCATION
Office of Educational Research and
Improvement EDUCATIONAL RESOURCES
INFORMATION
CENTER (ERIC)
O
This document has been reproduced as
received from the person or
organization originating it.
O
Minor changes have been made to
improve reproduction quality
Points of view or opinions stated in this
document do not necessarily represent
official OERI position or policy.
2
BEST COPY AVA1MLE
eaching Practices
t
and Stude
evement
c
Findin,
First-Year
Report of
of
'Mosaic' Study
from the
Systemic Initiatives
and Science
in Mathematics
W '
11
The research described in this report was supported by the National
Science Foundation, grant number ESI-96-15809.
ISBN: 0-8330-2879-0
Building on more than 25 years of research and evaluation work,
RAND Education has as its mission the improvement of educational
policy and practice in formal and informal settings from early child-
hood on.
RAND is a nonprofit institution that helps improve policy and deci-
sionmaking through research and analysis. RAND® is
a registered
trademark. RAND's publications do not necessarily reflect the opin-
ions or policies of its research sponsors.
© Copyright 2000 RAND
All rights reserved. No part of this book
may be reproduced in any
form by any electronic or mechanical
means (including photocopy-
ing, recording, or information storage and retrieval) without permis-
sion in writing from RAND.
Published 2000 by RAND
1700 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138
1333 H St., N.W., Washington, D.C. 20005-4707
RAND URL: http:/ /www.rand.org/
To order RAND documents or to obtain additional information,
contact Distribution Services: Telephone: [phone];
Fax: [phone]; Internet: [email protected]
4
Teaching Practices
Student
and
Achievement
Findings
Report of First-Year
Study of
from the 'Mosaic'
Systemic Initiatives
and Science
in Mathematics
HAMILTON,
STEPHEN KLEIN, LAURA
BRIAN STECHER,
DANIEL MCCAFFREY,
BURROUGHS
ABBY ROBYN, DELIA
RAND
EDUCATION
PREFACE
During the 1990s, the National Science Foundation (NSF) funded a
number of large-scale initiatives designed to change the way math-
ematics and science are taught in schools. These efforts, called Sys-
temic Initiatives (SIs), shared a common emphasis on aligning all
curricu-
aspects of the educational system in support of ambitious
lum and performance standards. Particular emphasis was placed on
teacher training and professional development to promote changes
in instructional practice that would enable students to achieve the
new standards.
Funds were given to states, to urban school districts, and to consortia
of districts to implement reforms consistent with NSF's purposes.
Sites had considerable flexibility in designing their programs, and
they adopted very different strategies for promoting reform. As a re-
sult, initial research on the SIs focused on the complex process of de-
velopment and implementation. Although individual sites gathered
information, after five years of funding, NSF had no broad picture of
the effects of the reform on student achievement.
In 1996, NSF provided funds to RAND to investigate the relationships
between student achievement in mathematics and science and the
systemic re-
use of instructional practices that are consistent with
forms. The study, called the Mosaic project, was conducted in two
districts)
waves: A set of six sites (including both states and urban
that were implementing systemic reforms was studied during the
1996-97 school year, and a similar set of six sites was studied during
the 1997-98 school year. The same basic analytic design was repli-
cated at each site, and the study draws much of its power and gen-
eralizability from this replication.
iii
iv
Teaching Practices and Student Achievement
This report presents results for the first
wave of the study. The re-
sults should be of interest to educational policymakers
at all levels of
government, as well as to program developers and school adminis-
trators interested in mathematics and science education.
CONTENTS
iii
Preface
vii
Figures
ix
Tables
xi
Summary
xvii
Acknowledgments
Chapter One
1
INTRODUCTION
2
The Systemic Initiatives Programs
3.
Earlier Evaluations of Systemic Initiatives
Evidence of Relationships Between Teaching Practices
5
and Achievement
5
Measuring Student Achievement
6
Overview of the Mosaic Project
Chapter Two
9
METHODS
9
Site Selection
10
School, Subject, and Grade-Level Selection
10
Data Collection
11
Student Achievement Data
13
Teacher Questionnaires
15
Demographic Data
16
Participation Rates
16
Analysis
vi
Teaching Practices and Student Achievement
Chapter Three
FIRST-YEAR RESULTS
19
Distributions of Teaching Practices
19
Relationships Between Teaching Practices
and Student
Achievement
21
Alternative Formulations for Site Models
30
Differences Between Test Formats
30
Study Limitations
34
Chapter Four
DISCUSSION
37
Summary of Year 1 Findings
37
Plans for Future Data Collection and
Analysis
39
Appendix
A.
Participation at Year 1 Sites
41
B.
Items on Teaching-Practices Scales
47
Full Regression Models
C.
51
D. Details of Pooled Analysis of Regression
Coefficients
65
E.
Results from Analysis of Format
Differences
67
F.
Sensitivity Analyses: Use of
Contemporaneous Test
Scores
69
G. Sensitivity Analyses: Combining Reform
and Traditional
Scales in a Single Model
73
References
81
FIGURES
xv
Pooled Results from Mathematics Analyses
S.1.
xv
Pooled Results from Science Analyses
S.2.
Distribution of Teacher Scores on Reform and
3.1.
21
Traditional Scales by Site and Subject
Estimated Coefficients by Site and Pooled Across Sites
3.2.
for Mathematics: Reform Practices, Open-Ended
23
Tests
Estimated Coefficients by Site and Pooled Across Sites
3.3.
for Mathematics: Reform Practices, Multiple-Choice
23
Tests
Estimated Coefficients by Site and Pooled Across Sites
3.4.
24
for Science: Reform Practices, Open-Ended Tests
. .
.
Estimated Coefficients by Site and Pooled Across Sites
3.5.
for Science: Reform Practices, Multiple-Choice
24
Tests
Estimated Coefficients by Site and Pooled Across Sites
3.6.
for Mathematics: Traditional Practices, Open-Ended
25
Tests
Estimated Coefficients by Site and Pooled Across Sites
3.7.
for Mathematics: Traditional Practices, Multiple-
25
Choice Tests
Estimated Coefficients by Site and Pooled Across Sites
3.8.
for Science: Traditional Practices, Open-Ended
26
Tests
Estimated Coefficients by Site and Pooled Across Sites
3.9.
for Science: Traditional Practices, Multiple-Choice
26
Tests
vul