ERIC ED389805: Effective Practices and Policies: Research and Practitioner Views. Publication Series #95-8.

50 Pages·1995·1.4 MB·English

by  ERIC

DOCUMENT RESUME
UD 030 724
ED 389 805
Wang, Margaret C.; And Others
AUTHOR
Effective Practices and Policies: Research and
TITLE
Practitioner Views. Publication Series #95-8.
Temple Univ., Fiiadelphia, PA. National Education
INSTITUTION
Center on Education in the Inner Cities.
Office of Educational Research and Improvement (ED),
SPONS AGENCY
Washington, DC.
PUB DATE
95
60p.; Paper originally presented at the Annual
NOTE
Meeting of the American Educational Research
Association (San Francisco, CA, April 1995).
Research/Technical (143)
Reports
PUB TYPE
Speeches/Conference Papers (150)
MF01/PC03 Plus Postage.
EDRS PRICE
*Administrators; Educational Assessment; Educational
DESCRIPTORS
Planning; Educational Policy; *Educational Practices;
Educational Research; *Educational Researchers;
*Evaluation Methods; Outcomes of Education; Program
Development; Program Implementation; *Research
Utilization; *School Effectiveness
ABSTRACT
In a survey of educational researchers, school and
district administrators, and policymakers, 1,818 respondents rated
educational practices and policies according to their influence on
learning and assessability. Classroom practices, design and delivery
of curriculum, and schoolwide practices were rated as more
influentlal than federal, state, and district policies; however, the
latter were rated as more assessable. In general, researchers and
practitioners agreed about which practices and policies are
influential but not about their assessability. Practices and policies
rated both influential and assessable are the most feasible for use
in educational planning and evaluation; those rated influential but
less assessable call for development of new observational measures.
This survey data can be used to guide local program development,
assessment development, and the monitoring of program implementation
and evaluation of outcomss. (Contains 8 tables, 2 figures, and 47
references.) (Author)
***********************************************************************
Reproductions supplied by EDRS are the best that can be made
from the original document.
***********************************************************************
1
Effective Practices and Policies:
Research and Practitioner Views
by Margaret C. Wang, Geneva D. Haertel, and Herbert J. Walberg
The National Center on
Education in the Inner Cities
U S DEPARTMENT OF EDUCATION
ann imp,osnment
Office 0, tducal,000- Res
EDUCATIONAL RESOURCES INFORMATION
CENTER tERICI
0 This document has been reproduced as
received born the person or organization
originating it
111
p5;6.inor changes have been made to
improve reproduction quality
Points el view or opinions stated in this
document do not necessarily represent
official OEHI positron or policy
BEST COPY AVAILABLE
Temple University Center for Research
in Human Development and Education
Effective Practices and Policies:
Research and Practitioner Views
by Margaret C. Wang, Geneva D. Haertel, and Herbert J. Wa /berg
The National Center on
Education in the Inner Cities
This paper was originally presented at the annual meeting of the American Educational
Research Association, April 1995, San Francisco, California.
The research reported herein was supported in part by the Office of Educational Research and
Improvement (OERI) of the U.S. Department of Education through a grant to the National Center
University Center for Research in Human
on Education in the Inner Cities (CEIC) at the Temple
the
Development and Education (CRHDE). The opinions expressed do not necessarily reflect
position of the supporting agencies, and no official endorsement should be inferred.
# 95-8
3
Abstract
In a survey of educational researchers, school and district administrators, and policymakers, 1,818
respondents rated educational practices and policies according to their influence on learning and
assessability. Classroom practices, design and delivery of curriculum, and schoolwide practices were
rated as more influential than federal, state, and district policies; however, the latter were rated as more
In general, researchers and practitioners agreed about which practices and policies are
assessable.
influential but not about their assessability. Practices and policies rated both influential and assessable
are the most feasible for use in educational planning and evaluation; those rated influential but less
assessable call for development of new observational measures. This survey data can be used to guide
local program development, assessment development, and in the monitoring of program implementation
and evaluation of outcomes.
Effective Practices and Policies:
Research and Practitioner Views
Excellence in Education, 1983), public attention
Since A Nation at Risk (National Commission on
Identifying practices
improving students' achievement.
and efforts to reform schools have focused on
interests of all stakeholdersparents, educators, and
and policies that enhance learning serves the
1980s and 1990s has been dominated by reform
policymakers. Indeed, the education landscape of the
efforts.
effective provides a knowledge base to improve
Research on what makes learning more
capacity of schools for
Research-based practices and policies can improve the
conventional practices.
standards, and to improve
Calls to upgrade the nation's teaching corps, to set
student achievement.
what is known about learning into the reform agenda.
student learning underscore the need to bring
from a survey of educators' and researchers' judgments
Toward that end, this paper reports the findings
student learning and the degree of assessability of those
about practices and policies that influence
provided the impetus for the survey are discussed in the
learning influences. Several developments that
assessable policies and
They show the need for research-based information on
next two sections.
practices that work effectively.
Raising Standards
in the educational reform movement.
Standard setting has become the most visible activity
Mathematics' Curriculum and Evaluation Standards
Beginning with the National Council of Teachers of
education organizations developed content standards that
for School Mathematics (1989), professional
instructional delivery. States and school districts
would influence curricula, assessment practices, and
mathematics, science, English, language arts,
participated with the various standard-setting efforts in
(1994) reported that in 1989 and 1990, the National
history, and geography. Fuhrman and Elmore
Bush in "calling for, establishing, and promoting
Governors Association worked with former President
setting as a
goals" (p. 66). Federal government agencies embraced standard
a set of national education
5
For example, the National Assessment of
enterprise.
central approach to improving the educational
Both the
mathematics assessment to the NCTM standards.
Educational Progress (NAEP) aligned its
establish
the new Title I legislation require that each state
Goals 2000: Educate America Act and
(Ganda',
educational reform efforts around achieving them
standards for student achievement and center
1995).
products to parallel the standards set by
Publishers and testing companies have tailored their
Praxis III, a component of a teacher assessment system
professional education groups and Goals 2000.
high-stakes assessment used to determine whether
developed by Educational Testing Service, is a
continuing license. According to Danielson and Dwyer
provisionally licensed teachers should receive a
of Praxis III] was the establishment of teaching standards.
(1995), "A critical element [in the development
interrelated aspects of a complex performance" (p. 66).
The resulting 19 criteria in 4 domains represent
standards are being identified. "Systemic
To reform the educational system more broadly, other
delivery, and system delivery standards (Smith,
reform" coordinates content, student performance, school
students must
Content standards identify the knowledge and skills that
Fuhrman, & O'Day, 1994).
identify the degree of competency that must be demonstrated for
master; student performance standards
identify criteria indicating whether a school provides
each content standard; and school delivery standards
identified in the content standards. Examples of
students with the "opportunity to learn" the material
quality instructional materials needed to teach the content
school delivery standards might include:
standards; and training staff to teach them (Smith
standards; alignment of the school curriculum with the
standards address the quality of the district, state, or federal systems'
et al., 1994). System delivery
The identification and
standards.
capability to educate all students as specified in the content
delivery standards presumably result in
implementation of content, performance, and school and system
learning.
systemic educational reform and improvement of student
Effective Practices - 2
operationalized in terms of teacher behaviors, classroom
School and system delivery standards are
which a school, local
therefore essential to measure the degree to
practices, and schoolwide policies. It is
policy.
association has implemented a particular practice or
education association, or state education
challenges researchers to identify practices and policies
Setting school delivery standards, in particular,
learning and that can be reliably and validly assessed.
that are linked to improved student
Upgrading the Nation's Teaching Corps
imposed higher standards on teacher
During the 1980s state legislatures passed laws that
that prospective teachers typically enrolled in college
preparation and licensing. Evidence had accrued
taken by students in noneducation degree programs. The
classes that were less rigorous than those
Carnegie Task Force on Teaching as a Profession (1986)
Holmes Group Executive Board (1985) and the
They recommended that teachers engage in a 5-
called for upgrading weak teacher education programs.
of study and set more rigorous course requirements.
year, rather than 4-year, course
and systematic observation of
Paralleling these efforts, merit pay, career ladder programs,
have been of limited utility because they
inservice teachers were also employed. These approaches may
For this reason, teacher evaluation systems and
often were not linked to improved student learning.
include evidence of student learning as one of several
efforts at professional development more often
Schalock, Schalock, Myton, & Girod, 1993).
criteria of teacher effectiveness (Sanders & Horn, 1993;
Plake, & Fager, 1993; Impara, Plake, &
Surveys conducted by Impara and colleagues (Impara,
educational organizations to identify tmcher and
Merwin, 1994) have been used by professional
In one survey,
assessment-related tasks.
administrator competencies, knowledge, and attitudes in
elementary and secondary school principals)
approximately 1,700 school administrators (superintendents,
of
which they performed 24 assessment tasks and the importance
were asked to rate the frequency with
knowledge and need of 13 assonsment-related practices.
these tasks. They also were asked to rate their
administrators and 555 teachers focused on knowledge of
A second survey of approximately 300
Effective Practices - 3
P-S
35 items based on the Standards for
school administrators received
classroom assessment. Teachers and
Federation of Teachers, 1990).
Assessment of Students (American
Teacher Competence in Educational
in assessment. Both of
about their use of tests and their training
Survey respondents also were asked
formulating standards for
professional education organizations in
these surveys were conducted to assist
students.
educator competency in assessing
departments of education would benefit from
Colleges of education, school districts, and state
well as results
improved student performance. Survey data as
empirical information linking pedagogy to
design professional
develop courses of study for education majors,
of empirical studies could be used to
efforts.
inservice teachers, and aid in standard-setting
development programs for preservice and
Improving Student Educational Performance
McDill,
result in increased student achievement.
Raising academic standards alone may not
risk
academic standards may seem unattainable to students at
Natriello, and Pallas (1987) warn that high
disengaging and,
result in some of these students academically
of school failure. Raising standards could
variety
academic standards requires the implementation of a
eventually, dropping out of school. Meeting
of instructional and schoolwide practices.
that support for standard-setting efforts is
Although recent developments in Congress suggest
performance
Feb. 1, 1995), efforts to raise low levels of student
waning ("National Historj Standards,"
the public alike. Given the limited resources of many
continue to receive backing from educators and
Models of
powerful influences on learning clearly is desirable.
school systems, identifying the most
identify most, if not all, of the important influences
educational performance and school learning aspire to
syntheses conducted over
participation in school. Meta-analyses and research
on students' learning and
educational
that can guide the identification of effective
the past 15 yeats also provide a foundation
practices and policies.
Effective Practices - 4
SYNTHESES
PREVIOUS RESEARCH
Walberg (1986) argues that research
reform is research synthesis.
One basis for research-based
(p. 214).
maturity to research on teaching"
brought a new level of scientific
synthesis has ".
.
.
and accessible to consumers of
that make their results more valid
Quantitative syntheses have features
features of research synthesis as follows:
(1986) describes the advantageous
research results. Walberg
evaluation and summary of
techniques and standards to the
[It] explicitly applies scientific
also provides detailed
summarizes effects across studies but
research, it not only statistically
metrics of study effects,
of literature, selection of studies,
descriptions of replicable searches
with respect to experimental
overall and exceptional results
statistical procedures, and both
214)
conditions, context, or subjects. (p.
set
syntheses that focus on education converge on a
Increasingly, conclusions from quantitative
and Wilson (1993) assert that the
enhance educational outcomes. Lipsey
of practices and policies that
small, are mostly positive and
reported in these syntheses, although
magnitude and direction of the effects
of such quantitative syntheses
They further conclude that the results
have genuine practical significance.
reviews" (p. 1).
credible than those of conventional
. more
are ". .
Multiple Influences on Learning
practice or on
focused either on a specific instructional
Historically, research syntheses have
syntheses fall
their relative effectiveness. The following
several diverse practices in order to compare
into the latter category.
of research
published one of the first quantitative syntheses
Walberg, Schiller, and Haertel (1979)
impact or
published between 1969 and 1979 on the
The authors collected reviews
on teaching.
Among the instructional
students' cognition, affect, and behavior.
association of instructional variables on
incentives, open versus
task, mastery learning, psychological
practices synthesized were: time on
thirds of the effect sizes or
of advance organizers. Nearly two
traditional classroom practices, and use
practices enhance student
that many well-established educational
correlations were positive, indicating
achievement.
Effective Practices - 5
9
process-student outcome research, Waxman
In a systematic examination of 19 reviews of teaching
that cognitive engagement, motivationa!
and Walberg (1982) found consistent and substantial agreement
and classroom climate are
incentives, pupil involvement in learning, reinforcement, and management
positively associated with student learning outcomes.
of the influence of instruction,
During the 1980s, Walberg and colleagues conducted syntheses
achievement. The syntheses focused
psychological environments, and student aptitudes on educational
hypothesized to be consistently related to educational learning: student age
on nine theoretical constructs
(including prior achievement); motivation; amount of instruction; quality
or developmental level; ability
environments of the class, home, and
of instruction; exposure to the mass media; and the psychological
provided systematic evidence that the constructs are consistently
peer group outside school. The results
correlated with learning (Walberg, 1984).
research on factors related to
Fraser and colleagues (1987) compiled an extensive review of
bivariate studies that identified nine aptitudinal,
school learning. They summarized .-esults of over 2,000
influences on academic learning.
instructional, and environmental factors *.*.iat consistently exhibited strong
achievement as an outcome, as well
Fraser et al. (1987) also synthesized 135 meta-analyses with school
the studies spanned 50 years of research in the United States
as 92 meta-analyses of attitude outcomes;
contextual factors, including student and teacher
and elsewhere. Among the influences examined were
environment, and school climate.
characteristics, curriculum materials, facilities and equipment, home
school learning. Search
Wang et al. (1990) reported a content analysis of research literature on
chapters, commissioned papers, and
and selection procedures yielded 179 handbook and annual review
characteristics, instruction, and
other authoritative reviews. Results confirmed the primacy of student
distal variables, such as state and district
home and community influences on academic learning. More
synthesized ratings of 61 research
policy, proved less influential. More recently, Wang et al. (1993)
and narrative reviews, compiling approximately
experts, 91 meta-analyses, and 179 handbook chapters
Effective Practices - 6
0