ERIC ED518092: "WordSift": Supporting Instruction and Learning through Technology in San Francisco. The Senior Urban Education Research Fellowship Series. Volume IV

2011·2.3 MB·English

by  ERIC

WordSift: Supporting Instruction and Learning
S
e
through Technology in San Francisco
i
r
e
S

p
i Kenji hakuta
h
S
w
o
l
l
e
F

h
c
r
a
e
S
e
r

n
o
i
T
a
c
U
d
e

n
a
b
r
U

r
o
i
n
e
S

e
h
T
Volume IV
SprIng 2011
The CounCIl of The greaT CITy SChoolS
The Council of the Great City Schools thanks the Institute
of Education Sciences (IES) for supporting the Senior
Urban Education Research Fellowship Program.
The findings and conclusions presented herein are those
of the authors and do not necessarily represent the views
of the Council of the Great City Schools or IES.
ii The Council of the Great City Schools
The Senior Urban Education Research
Fellowship Series
Volume IV:
WordSift: Supporting Instruction and Learning
through Technology in San Francisco
Kenji Hakuta
Spring 2011
The Council of the Great City Schools is the only national organization exclusively representing the needs
of urban public schools. Founded in 1956 and incorporated in 1961, the Council is located in Washington,
D.C., where it works to promote urban education through legislation, research, media relations, instruction,
management, technology, and other special projects.
The Senior Urban Education Research Fellowship Series, Volume IV - Spring 2011 iii
iv The Council of the Great City Schools
Table of conTenTs
Overview: The Senior Urban Education Research Fellowship Program ....................................................2
About the Senior Urban Education Research Fellow ................................................................................................4
About the Research Partnership ................................................................................................................................................5
Executive Summary ...............................................................................................................................................................................7
Introduction: Addressing the Need for Academic Language Development ...........................................10
Part I: Development of a Teacher-Centric Language Development Tool ..................................................14
Part II: Implementation and Patterns of Usage ...............................................................................................................20
Part III: Early Efforts to Evaluate the Efficacy of WordSift...................................................................................24
Discussion .....................................................................................................................................................................................................28
References ....................................................................................................................................................................................................30
Acknowledgements ...............................................................................................................................................................................34
figures
Figure 1. Screenshot of the Result of WordSift Text Analysis .......................................................................16
Figure 2. Effect of Web-Based Vocabulary Tool
on Reading Comprehension Assessment Scores by Gender ............................................................................26
The Senior Urban Education Research Fellowship Series, Volume IV - Spring 2011 1
overvieW
ThE SEnIoR URban EdUCaTIon Though a great deal of education research is produced
RESEaRCh FEllowShIP PRoGRam
every year, there is a genuine dearth of knowledge
regarding how to address some of the fundamental
Large urban public school districts play a significant
challenges urban school districts face in educating
role in the American education system. The largest 65
children, working to close achievement gaps, and
urban school systems in the country – comprising less
striving to meet the challenges of No Child Left Behind.
than one half of one percent of the nearly seventeen
Moreover, while there is a history of process-related
thousand school districts that exist across the United
research around issues affecting urban schools, relatively
States – educate about 14 percent of the nation’s
few studies carefully identify key program components,
K-12 public school students, approximately a quarter
document implementation efforts, and carefully examine
of its economically disadvantaged students, a third of
the effects of well-designed interventions in important
its African American students, a quarter of its Hispanic
programmatic areas on key student outcomes such as
students, and a third of its English Language Learners.1
academic achievement. In sum, there is an absence of
Clearly, any attempt to improve achievement and to
methodologically sound, policy-relevant research to help
reduce racial and economic achievement gaps across
guide practice by identifying the conditions, resources,
the United States must involve these school districts as a
and necessary steps for effectively mounting initiatives
major focus of action.
to raise student achievement.
These school districts face a number of serious,
In order to address this need, the Council of the Great City
systematic challenges. To better understand the problems
Schools, through a grant from the Institute for Education
in urban education and to develop more effective and
Sciences, established the Senior Urban Education
sustainable solutions, urban districts need a program
Research Fellowship (SUERF) program.
of rigorous scientific inquiry focusing on what works
to improve academic outcomes in the urban context. The Senior Urban Education Research Fellowship was
Moreover, in order to produce such evidence and to move designed to facilitate partnerships between scholars and
public education forward generally, the standards of practitioners focused on producing research that is both
evidence in education research must be raised in such a rigorous in nature and relevant to the specific challenges
way as to bring questions regarding the effectiveness of facing large urban school districts. We believe such
educational interventions and strategies to the fore and partnerships have the potential to produce better, more
to promote careful scrutiny and rigorous analysis of the practically useful research in at least three ways. First,
causal inferences surrounding attempts to answer them. by deepening researchers’ understanding of the contexts
within which they are working, the program may help them
It has been argued that, in order to move such an effort
maximize the impact of their work in the places where it is
forward, a community of researchers, committed to a
needed the most. Second, by helping senior staff in urban
set of principles regarding evidentiary standards, must
districts become better consumers of research, we hope
be developed and nurtured. We contend further that, in
to increase the extent to which the available evidence
order to produce a base of scientific knowledge that is
is used to inform policy and practice, and the extent to
both rigorously derived and directly relevant to improving
which urban districts continue to invest in research. Third,
achievement in urban school districts, this community of
by executing well-designed studies aimed at the key
inquiry must be expanded to include both scholars and
challenges identified by the districts themselves, we hope
practitioners in urban education.
to produce reliable evidence and practical guidance that
can help improve student achievement.
1 Council of the Great City Schools (2010). Beating the Odds: An Analysis of Student Performance on State Assessment and NAEP.
Results from the 2008-2009 School Year. Washington, DC.
2 The Council of the Great City Schools
The primary goals for the Senior Urban Education The following volume of the Senior Urban Education
Research Fellowship are to: Research Fellowship Series documents the work of
Dr. Kenji Hakuta, working in collaboration with the San
• promote high quality scientific inquiry into the ques-
Francisco Unified School District under the auspices of
tions and challenges facing urban school districts;
the Strategic Education Research Partnership (SERP).
• facilitate and encourage collaboration, communi- Both the research and reporting is the sole intellectual
cation, and ongoing partnerships between senior property of Dr. Hakuta, and reflects his personal
researchers and leaders in urban school districts; experience and perspective as an education researcher
working in collaboration with SFUSD.
• demonstrate how collaboration between scholars
and urban districts can generate reliable results
Dr. Hakuta’s work developing and piloting the WordSift
and enrich both research and practice;
tool in San Francisco makes an important contribution
• produce a set of high quality studies that yield to the growing field of instructional technology. In
practical guidance for urban school districts; recognition of the role academic language plays in
student achievement, he has designed this tool as a
• contribute to an ongoing discussion regarding
resource for content area instructors, helping them to
research priorities in urban education; and
become teachers of the academic language of their
• promote the development of a “community of
discipline. The need to support and advance academic
inquiry”, including researchers and practitioners
literacy instruction across the curriculum is a fundamental
alike, committed to both a set of norms and prin-
challenge facing researchers and educators alike, and
ciples regarding standards of evidence and a set
technology and innovation will likely play a significant role
of priorities for relevant, applied research in urban
in our efforts to reform and improve. We hope you will
education.
find this report both interesting and relevant to your own
work in urban education.
The SUERF program benefitted greatly from the guidance Thank you.
and support of a Research Advisory Committee made up
michael Casserly
of experts and leaders from large urban school districts
Executive Director
and the education research community. The committee
Council of the Great City Schools
included Dr. Katherine Blasik, Dr. Carol Johnson, Dr. Kent
McGuire, Dr. Richard Murnane, Dr. Andrew Porter, and
Dr. Melissa Roderick. This extraordinary group helped to
identify and define the objectives and structure of the
fellowship program, and we thank them for lending their
considerable insight and expertise to this endeavor.
The Senior Urban Education Research Fellowship Series, Volume IV - Spring 2011 3
abouT The senior urban
educaTion research felloW
Kenji Hakuta is the Lee J. Jacks Professor of Education at Stanford
University. An experimental psycholinguist by training, he is best known
for his work in the areas of bilingualism and the acquisition of English in
immigrant students. He is the author of numerous research papers and
books, including Mirror of Language: The Debate on Bilingualism and
In Other Words: The Science and Psychology of Second Language
Acquisition. He chaired a National Academy of Sciences report Improving
Schooling for Language-Minority Children, and co-edited a book on affirmative action in higher education, Compelling
Interest: Examining the Evidence on Racial Dynamics in Higher Education. Hakuta is also active in education policy.
He has testified to Congress and other public bodies on a variety of topics, including language policy, the education of
language minority students, affirmative action in higher education, and improvement of quality in educational research.
He has served as an expert witness in education litigation involving language minority students. Hakuta received his BA
Magna Cum Laude in Psychology and Social Relations, and his Ph.D. in Experimental Psychology, both from Harvard
University. He has been on the faculty at Stanford since 1989, except for three years (2003-2006) when he helped start
the University of California at Merced as its Founding Dean of Social Sciences, Humanities and Arts. He is an elected
Member of the National Academy of Education, Fellow of the American Association for the Advancement of Science
(Linguistics and Language Sciences), and Fellow of the American Education Research Association.
4 The Council of the Great City Schools
abouT The research ParTnershiP
ThE STRaTEGIC EdUCaTIon administrators fail in the collaborative effort, their
RESEaRCh PaRTnERShIP
primary source of professional identity is largely
unaffected. A coordinating organization fails if the
This work was conducted in the context of a larger
collaboration fails, creating incentives to persist
research partnership at the San Francisco Unified
through the most difficult challenges, and search
District organized by the Strategic Education Research
for solutions that will strengthen the long run
Partnership (SERP). SERP was conceived and incubated
prospects of the collaboration.
at the National Academy of Sciences, and began operating
as an independent institute in 2005. Its ambitious • The SERP Model assumes that building
mission is to create an infrastructure for problem-solving a knowledge base will require an active
research, development, and implementation in education. effort to link sites—not just a sharing of
The SERP partnership model is unique in several information, but a reshaping and retesting
respects: of ideas. The coordinating SERP organization
takes on responsibility for ensuring that each
• The SERP model assumes that the new site has access to the research tools and
development and maintenance of effective instruments developed in other sites, and that
partnerships between researchers and knowledge and experience that emerge from
practitioners is itself a form of expertise one site, as well as the interventions and tools
that must be developed and nurtured developed, are actively carried to other sites.
over time. Knowledge and experience must
SERP and ThE San FRanCISCo
be accumulated regarding the trajectory of
UnIFIEd SChool dISTRICT
trust building, and the negotiation of agendas.
Policies and practices must be developed that The existence of the SERP-San Francisco field site within
place the district in the lead on defining problems the San Francisco Unified School District (SFUSD) was
for attention, and researchers in the lead on crucial to the development and initial testing of WordSift.
designing research. It must nurture a sense of The SERP partnership with the district began in January
shared responsibility for designing approaches 2007, and the primary focus of the SERP-SFUSD
to improvement. Expertise must be developed on work during the period of this project was defined as
the types and frequency of interaction required addressing the achievement gap in middle schools,
for sustained commitment to the partnership, the especially in math and science, and the language and
participants who must be at the table to support literacy needed to support the closing of the gap. The
progress, the sources of competing attention that problem was defined through the District’s recognition
must be taken into account, and the predicable that its gap between high- and low-achieving students is
obstacles that can derail the effort or diminish its among the highest in California urban districts, and that
productivity. Thus, a major SERP role is forming much of the low end of the achievement spectrum was
and maintaining researcher-district relations. due to the performance of African-American and Latino
students.
• The SERP model assumes that a third party
organization charged with coordination
The SERP work in science and science literacy is built
can ensure the goals of the partnership
on the premise that narrowing that gap will require a
are primary at all times. There are strong
high level of specificity regarding what students who
incentives for both researchers and education
do well in science know, and what those who do poorly
practitioners to respond to pressures and rewards
need to know. The work makes a further assumption that
in their own institutional and professional contexts.
much of what is taught is not critical to mastery of the
And if university researchers or school district
core ideas, and that progress in narrowing the gap will
The Senior Urban Education Research Fellowship Series, Volume IV - Spring 2011 5
abouT The research ParTnershiP
require the identification of a subset of “core concepts” The costs of developing and piloting this tool were
that should be given intensive focus in all classrooms. covered by grants to SERP and the present fellowship
Finally, it is assumed that teachers need a set of tools grant from the Council of Great City Schools through
and instructional strategies to teach this core content the Senior Urban Research Fellowship Program, funded
and monitor student learning effectively. by IES, from 2007 to 2009. The previously established
relationship with the SFUSD administration and with the
Over the past two years through these SERP activities,
instructional leaders in SFUSD schools made possible
the science co-developer teachers, district supervisors
a highly collaborative process for developing WordSift,
and administrators, and researchers have moved toward
which included tool design and developing uses for
a system of knowledge and trust-building that begins
teachers and students. SERP has supported the use of
to address these important issues in instruction and
this tool with the current group of teacher co-developers,
assessment. Through work with the Berkeley Evaluation
having them use the tool in 6 collaborating schools,
& Assessment Research Center (BEAR) at University
collecting and processing feedback from teachers in
of California at Berkeley, (Mark Wilson, Director), the
those schools, and incorporating the feedback into the
co-developers have created explicit progress maps in a
design of the tool.
number of domains of science instruction in the 6th and
8th grade that correspond to learning progressions for SERP played a key role in the work described here. SERP
students. has built a strong framework to facilitate communications
between the practitioners in SFUSD and the research
In this context, a group of us working on the language
team so that there is a regular feedback loop between
characteristics and demands of the curriculum have
the district and the researchers. These include period
elevated teacher awareness of these issues and taken
meetings focused on the design nature of the research
the first steps in addressing these gaps. Professor
and practice work, as well as subject-specific working
David Pearson and Post Doctoral fellow Marnie Nair
groups that oversee the direction and progress of the
from University of California at Berkeley participated
research work. In addition, a core team of district and
in the initial work in identifying language needs. Karen
research leaders meets regularly to discuss the needs of
Thompson, a graduate student from Stanford, provided
district and the progress of ongoing research efforts. The
early analysis of the linguistic complexity of the science
core team provides the general guidance and permission
textbooks being used. WordSift was created as part of
for work within the school district so that the research
this effort, to help address the recognition that teachers
work is pertinent to the needs of the teachers and
of science needed tools help to become teachers of the
principals and aligned to the goals of the district strategic
academic language of their discipline.
plan.
6 The Council of the Great City Schools