ERIC EJ1070484: How Do Openers Contribute to Student Learning?

2012·0.92 MB·English

by  ERIC

International Electronic Journal of Elementary Education, 2012, 5(1), 79-92.

How do Openers Contribute to Student
∗∗∗∗
Learning?

∗∗∗∗∗∗∗∗
Amber ZERTUCHE
University of California, Berkeley, United States of America

Libby GERARD
University of California, Berkeley, United States of America

Marcia C. LINN
University of California, Berkeley, United States of America

Received: 20 August 2012 / Revised: 24 October 2012 / Accepted: 24 October 2012
Abstract
Openers, or brief activities that initiate a class, routinely take up classroom time each day yet little is
known about how to design these activities so they contribute to student learning. This study uses
technology-enhanced learning environments to explore new opportunities to transform Openers
from potentially busy work to knowledge generating activities. This study compares the impact of
teacher-designed Openers, Opener designs based on recent research emphasizing knowledge
integration, and no Opener for an 8th grade technology-enhanced inquiry science investigation.
Results suggest that students who participate in a researcher-designed Opener are more likely to
revisit and refine their work, and to make significant learning gains, than students who do not
participate in an Opener. Students make the greatest gains when they revisit key evidence in the
technology-enhanced curriculum unit prior to revision. Engaging students in processes that promote
knowledge integration during the Opener motivate students to revise their ideas. The results suggest
design principles for Openers in technology-enhanced instruction.
Keywords: Technology, Science Education, Teaching Practices, Classroom Assessment, Formative
Assessment

∗
This article is based on the first author’s unpublished MA thesis entitled “How Teacher-led Openers
Contribute to Student Learning in a Web-based Inquiry Science Environment.” This material is based
upon work supported, in part, by the National Science Foundation under Grant DRL-0733299 [Logging
Opportunities in Online Programs for Science (LOOPS)]. Any opinions, findings, and conclusions or
recommendations expressed in this material are those of the author(s) and do not necessarily reflect
the views of the National Science Foundation.
∗∗ Amber Zertuche, Graduate School of Education, Tolman Hall, University of California, Berkeley,
U.S.A. Phone: [phone]. E-mail: [email protected]

ISSN:1307-9298
Copyright © IEJEE
www.iejee.com
International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012

Introduction
Openers, or brief activities that initiate a class, routinely take up classroom time each day yet
little is known about how to design these activities so they contribute to student learning.
Teachers often give mini-lectures to remind students of what they studied in the previous
class. Or, teachers may assign a short writing assignment. Technology-enhanced learning
environments provide teachers with new tools to transform Openers from busy work to
valuable learning activities. Openers can engage students in knowledge integration activities
such as making predictions, critiquing a claim, assessing peer essays, or reflecting on
progress—all activities that have been shown to improve student learning (Chiu and Linn,
2011; Linn and Eylon, 2011; White and Frederiksen, 1998). This study compares the impact of
teacher- and researcher-designed Openers to no Opener in an 8th grade technology-
enhanced inquiry science investigation. Questions include: How effective are Openers? How
does the design of the Opener contribute to student learning? And, what are effective post-
Opener revision processes?
Teacher-led Openers can play an essential role in the success of technology-enhanced
inquiry science instruction. Several studies show that the quality of teacher implementation
of technology-enhanced instruction predicts the impact of the technology on learning
(Tamin, Bernard, Borokhovski, Abrami, and Schmid, 2011). In a second-order meta-analysis
conducted on technology-enhanced instruction over the past 40 years, the authors conclude
that the teachers’ ability to monitor student understanding and help students sort out their
ideas is essential to successful technology implementation (Williams, Linn, M, Ammon, and
Gearhart, 2004). This meta-analysis found that as teachers asked more knowledge oriented
questions during the course of a technology-enhanced science investigation as opposed to
procedural questions, learning gains improved significantly. Further, to realize the potential
of technology-enhanced instruction, teachers need to help students critically analyze
visualizations relative to the conceptual learning goal as students often overestimate their
understanding of dynamic computer visualizations, particularly in Chemistry (Chiu and Linn,
2012).
Technology-enhanced learning environments provide teachers with unique tools to
structure effective Openers. The computer stores and organizes a record of each student’s
work including their multiple revisions, and provides ways to make examples of the student’s
work public to the whole class. This means teachers can purposefully select examples of the
student’s work for class discussion. Effective selection of examples could prompt students
(and their teachers) to monitor understanding, sort out ideas about difficult concepts, and
revisit and refine their reasoning (Izsak, 2012; Linn and Eylon, 2011).
Using Student Work to Design Openers
Student work is collected in technology enhanced learning environments and can be used to
design openers. Black and Wiliam (1998) reviewed hundreds of studies and came to the
conclusion that the use of student work for formative assessment and refinement of
instruction is one of the most powerful ways to increase students’ learning gains, particularly
among low achieving students. Similarly, Shute (2008) reviewed work on formative feedback,
defined as information communicated to the learner that is intended to modify his or her
thinking or behavior to improve learning, and found that effective feedback is non-
evaluative, supportive, timely, and specific. Gerard, Spitulnik, and Linn (2011) showed that
when teachers use student work to refine their teaching, students’ learning benefits. These
studies distinguish between activities providing verification, where students choose whether
an answer is correct or not, and elaboration, where teachers provide cues to guide learners
toward a correct answer. Gerard et al found that use of student work to design instruction is
80
How do Openers Contribute to Student Learning? / Zertuche, Gerard & Linn

most effective when the resulting instruction provides advice on what to do to improve
performance, rather than on comparisons to other students or on accuracy of responses.
One example of a successful use of student work is for self- and peer-assessment. Research
suggests that students can benefit from self and peer assessment activities when they
recognize the desired learning goal, have adequate evidence to determine where the work
stands relative to the learning goal, and have an understanding of a way to close the gap
between the two (Black and Wiliam, 1998). Under these conditions, students can distinguish
between their own ideas and the goal of instruction and strengthen their understanding.
Taken together, these results suggest that effective Openers should elicit student work and
provide hints or guidance.
What are the challenges of formative assessment?
Black and Wiliam’s review of formative assessment studies point out that, although formative
assessment has proven to be useful to student learning, there are concerns about how to
enact successful formative assessment (1998). One issue is that in order to implement
formative assessment in a useful way, teachers must have easy access to evidence of
students’ ideas and efficient routines to elicit students’ ideas during class so they can help
students to distinguish among these ideas. This calls for assessment or discussion questions
that prompt students to make their reasoning explicit and provide multiple entry points for
students with various levels of understanding. Students must also be actively involved in the
feedback they are receiving in order for it to have an effect on their learning. This requires
significant changes in a traditional secondary science classroom. Traditional science
classroom routines often center on teacher-directed lectures and demonstrations, leaving
little space for students to reflect on their understanding and sort out the variety of ideas
they hold about the topic gathered from everyday experiences, peers, and school curricula.
Ruiz-Primo and Furtak (2007), for instance found that teachers’ formative assessment
routines focused on procedural elements of inquiry learning, learning such as checking the
students’ knowledge of the correct procedure and asking them to apply a procedure to a
new situation, rather than knowledge generation.
How to Design an Effective Opener in Chemistry?
This study investigates the design of knowledge integration Openers. Typical classroom
Openers usually reflect the absorption model of instruction and use the question, response,
evaluation (QRE) approach. In this model, the teacher asks a question with one answer in
mind and prompts until a student gives the answer or fills in the response if none is elicited.
This theory of learning assumes that the task of the learner is to acquire the body of
connections that an expert analysis of the subject matter reveals (Greeno, Collins, and
Resnick, 1996).
In contrast, the knowledge integration perspective on learning resonates with the Black and
Wiliam’s findings (2011) and guides the design of the Openers, curriculum and assessments
in this study. The knowledge integration perspective draws on findings from learning
sciences research. Specifically, learners hold multiple conflicting ideas about scientific
phenomenon as has been documented in numerous studies of student intuitions about
science topics (diSessa, 2000). In addition, learners, often in collaboration with others, can
deliberately sort out, link, and critique their ideas when making sense of new scientific
phenomena and benefit from encouragement to engage in this process (Linn, Lee, Tinker,
Husic, and Chiu, 2006; Linn and Hsi, 2000; Novak and Gowin, 1984; Slotta, Chi, and Joram,
1995). This means that providing opportunities for students to compare alternative ideas to
their own, develop criteria for sorting-out and distinguishing among ideas, and reflect on
their ideas can help them form coherent hypotheses or explanations.
81
International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012

The knowledge integration Openers in this study were designed to help students make
connections among their ideas about chemical reactions. In chemistry, one of the most
difficult things for students to learn is how chemicals react. Students often have difficulties
translating between symbolic representations, molecular representations, and observable
phenomena (Ardac and Akaygun, 2004). Particularly, students struggle to make sense of
chemical phenomena at the molecular level (Johnstone, 1993; Krajcik, 1991). For example,
many students think of chemical reactions as an instantaneous process without bond
breaking and formation, while others think all the molecules break into atoms. In addition,
prior studies demonstrate that students often isolate molecular visualizations rather than
linking them to existing knowledge or everyday experiences and have difficulty interpreting
stand-alone dynamic visualizations (Tversky, Morrison, and Betrancourt, 2002; Zhang and
Linn, 2011). This study addresses this gap in learning chemical reactions with Openers that
ask students to reflect upon and critique peers’ visual molecular representations of hydrogen
and oxygen combustion.
Methods
Research Design
This study investigates how a researcher-designed Opener, teacher-designed Opener, and a
control condition (no Opener) contribute to students’ revision of work and understanding of
chemical reactions. Three central questions guide this research:
1. Do Openers contribute to student understanding of chemical reactions?
2. How does the Opener design influence students’ learning outcomes?
3. What are effective post-Opener revision processes?
Curriculum and Assessments
The Web-based Inquiry Science Environment (WISE) is an open-source on-line learning
environment that includes multiple standards-aligned science inquiry curriculum units. To
engage students in knowledge integration processes, WISE projects guide students in
collaborative activities with visualizations of scientific phenomena that are difficult to
observe, such as molecular views of chemical reactions (Figure 1). Students investigate
hypotheses, design solutions to problems, critique scientific claims, and build scientific
models, scaffolded by guidance based on knowledge integration principles.
Students in this study worked on the WISE Hydrogen Fuel Cell Car unit. This is a one week
unit designed to teach students about chemical reactions, alternative fuels, and energy
(http://wise.berkeley.edu/webapp/vle/preview.html?projectId=911). The project begins by
asking students if they would rather buy a hydrogen or gasoline powered car. It also elicits
their ideas about energy and adds ideas about conservation of energy. Then, gasoline
combustion in cars is explored including the relationship between carbon dioxide, a product
of gasoline combustion, and temperature changes over the last 200 years. Students then
create their own energy story about cars. This story includes where the energy came from to
power the car and any chemical reactions that are involved in their story. Hydrogen
combustion is then explored using a dynamic visualization of hydrogen combustion (Figure
1).
Then students are taught about the difference between exothermic and endothermic
reactions and finally, students investigate a visualization of a hydrogen fuel cell to learn how
this technology works. Students are then asked which kind of technology they would prefer
when buying a car.
82
How do Openers Contribute to Student Learning? / Zertuche, Gerard & Linn

Assessments are embedded throughout the WISE projects to help students and teachers
monitor student understanding and progress as students interact with visualizations (Figure
2). The embedded assessments ask students to make predictions about the visualizations,
sort out evidence, and link ideas together to explain their thinking. Students can also get
hints to help them complete the tasks.

Figure 1. One of the visualizations in WISE’s Hydrogen Fuel Cell Cars project models the
hydrogen combustion reaction. Students can run, stop, reset, and ‘spark’ to start the reaction
and explore the nature of chemical reactions.
In the Hydrogen Fuel Cell Cars unit, an embedded assessment immediately follows the
visualization of hydrogen combustion asking students to draw four frames of hydrogen
combustion (Figure 2). This is meant to help students make sense of the dynamic
visualization, recognizing features such as conservation of mass, bonds breaking, and the
progression of the reaction. The Openers in this study focused on student work from this
embedded assessment since in previous years this particular task was particularly
challenging to students and yet still, understanding how to draw basic hydrogen
combustion is critical to student understanding of chemical reactions.

Figure 2. The WISE project screen has an inquiry map on the left, a navigation bar on top, and
an embedded assessment in the center of the screen.
83
International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012

Teacher Assessment Tools
WISE provides teachers convenient access to a record of student work as students progress
through a WISE unit (Figure 3). The WISE grading tools allow teachers to view embedded
assessment data by step, and the flag tool allows teachers to select key student work
examples for display. Students can view flagged work at any time by clicking on a tab at the
top of their WISE screen. This allows students to actively see their peers' work and be a part
of the feedback process to improve their understanding. Students can revise their work
based on teachers’ and peers’ comments. All revisions are logged in the grading tool so the
teacher can measure the impact of their Opener, or comments, by viewing the change in
students’ ideas from their original to revised work.

Revision Time Stamp Flag Work

Number Checkbox

Figure 3. WISE assessment tool with revision number on the left, including time stamp, and
checkbox to flag work on the right.
Study Participants
Two 8th grade teachers and their 236 students from 1 public school participated in this
study. The school has medium diversity (22% receiving free lunch, 5% ELL, and 27% non-
white). Both teachers had over five years experience teaching with WISE and frequently used
Openers in their regular and WISE instruction.
Students were randomly assigned by class period to one of the three conditions. There were
78 students in the teacher-designed Opener, 128 students in the researcher-designed
Opener, and 30 students in the control group condition. The uneven sample sizes were due
to the uneven number of class periods that each teacher taught Physical Science that year.
Opener Design
The researcher-designed Opener engaged students in Knowledge Integration processes, as
shown in Table 1. Activities included a small group discussion, voting, a whole group
discussion, and then a closing summary by the teacher. The examples were selected to
illustrate the range of conceptual errors in student representations of the chemical reaction.
These related to (a) conservation of mass, (b) breaking of bonds, and (c) progression of the
reaction.
84
How do Openers Contribute to Student Learning? / Zertuche, Gerard & Linn

The teacher-designed Opener alternatively focused on picking out what is good and bad
about the student work and a lecture to try to re-teach chemical reactions from a different
angle. Examples of student work were selected to illustrate responses that could be
described as, “the good, the bad and the ugly”.
Table 1. Description of Openers used in Teacher 1 and Teacher 2’s classroom using the
Knowledge Integration Framework.
Teacher-Designed Researcher-Designed
Teacher 1 Elicit Ideas: Question on the board about Students open WISE project to view
how a balanced equation obeys the law of four examples of student work in
conservation of mass. WISE project by clicking on the
Students fill out a chart on how much mass “Flagged Work” button. Each
(amu) exists before and after reaction. example has one unique link
Add Ideas: Students use physical model of missing (Figure 4)
molecules to break bonds and put back Elicit Ideas: Teacher asks students
together. to write down which drawing best
Teacher shows multiple types of reactions. represents the visualization of
Distinguish Ideas: Teacher shows four hydrogen combustion and use
examples of student work and asks evidence to explain why.
students if the example is good or bad. Students vote and teacher tallies
Integrate Ideas: ____ votes.
(~12 min) Add Ideas: Students discuss their
Teacher 2 Elicit Ideas: ____ choices in groups of 4 and revisit
Add Ideas: Used embedded assessment evidence in the visualization.
problem and a tree to house analogy to Distinguish Ideas: Students
take students through the chemical reconsider their initial choice in
reaction steps. light of their discussion with peers
Tree must break into parts, then and revisit the evidence. Make a
recombine parts and build a house. new vote.
Uses physical models of hydrogen and Integrate Ideas: Teacher tallies
oxygen molecules to show students new votes and asks students to
progression necessary to make water. justify their choice. Teacher
Distinguish Ideas: ____ synthesizes criteria used to evaluate
Integrate Ideas: ____ drawings, and instructs students to
(~8 min) revise their own drawing.
(~20 minutes)

Link missing: conservation of mass Link missing: bonds breaking

Figure 4. Examples of student work shown during the researcher-designed Opener.
85
International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012

Data Collection and Analysis
Data sources include: the original student work on the embedded assessment, the revised
student work on the embedded assessment after the Opener, WISE log files illustrating
student navigation through WISE immediately after the Opener, pre and post tests
administered immediately before and a 1-14 days after the WISE project, classroom video of
teacher implementation of the researcher-designed and teacher-designed Openers, and
teacher interviews.
Knowledge integration rubrics were used to score the embedded and pre/post assessments.
The embedded assessment rubric is shown in Table 2.
Table 2. KI rubric for embedded assessment where students make step-by-step drawings of
hydrogen combusting with oxygen
KI Level Score Characteristics Example
Invalid 1 Blank or I don’t know
No links 2 No normative ideas conveyed
but work has been done
Simple, 1 3 Represents conservation of mass
link OR bonds breaking OR
progression*
Advanced, 4 Conservation of mass AND
2 links progression*
OR
Conservation of mass AND bonds
breaking
OR
Progression* AND bonds
breaking

Complex, 5 Conservation of mass AND bonds
3 links breaking AND progression* of
reaction

* Progression must be apparent on all 3 transitions
Pre and post questions were also scored using a KI rubric. The main ideas that students
should understand in the pre- and post- test are that there is a reaction process that occurs
and that H and Cl should start breaking bonds before forming new bonds.
2 2
Results
We examine the effects of Openers on student learning outcomes, and then explicate the
contributing factors including Opener design and students’ learning practices. We focus on
embedded assessments and pretest-posttest performance. We consider the actual
implementation of the conditions and student performance as reflected in the log files.

86
How do Openers Contribute to Student Learning? / Zertuche, Gerard & Linn

Embedded assessments
To investigate the impact of the Opener versus no Opener on student understanding we
compared students’ original (before Opener) and revised (after Opener) responses to the
embedded assessments. Responses were scored using the knowledge integration rubric.
Both teacher and researcher-designed Openers (n=105 pairs) were compared to the no
Opener condition (n=30 pairs). Pairs who did not complete the embedded assessment
before the Opener were excluded from the analysis (n=26 pairs). Although the aim was to
facilitate the Opener after 75% of students completed the embedded assessment, this
number was based on an automated progress screen that only monitored whether or not
students submitted work at least once for this assessment. Once the researchers looked at
the student’s work, it was obvious that, although work was submitted, many students did
not finish their drawings and therefore these were not included in the data analysis.
Responses were scored using the KI rubric. Time stamps from the WISE log files were used to
identify students’ final work immediately before the Opener and their revised work on the
day of the Opener.
The analysis suggests that students who had an Opener, either teacher- or researcher-
designed, made substantially greater learning gains than students who did not have an
Opener. As shown in Table 4, students who had an Opener (M = .29, SD = .78) doubled the
mean gain score of those students who did not have an Opener (M = .13, SD = .51) on the
embedded assessment. There was no significant difference between conditions in students’
pre-Opener scores.
Students who had an Opener (M = .68, SD = .48) were significantly more likely to revisit and
revise their work than students who did not have an Opener (M = .33, SD = .48), t(131) = -3.5,
p < 0.001. In the Opener condition, revision was twice as likely as in the no Opener condition.
Table 3. Pre to Post Gain Scores with and without Opener.
N Pairs Who Revised Mean Gain (SD) Rate of Revision (N Pairs who
Their Work Revised/Total Pairs)
Opener 105 .29(.78) 66%
No 30 .13(.51) 33%
Opener
The Openers were particularly effective for students who demonstrated at least a basic
understanding on the embedded assessment prior to the Opener (Table 7). Having an
Opener had a significant effect on students who began with at least a partial understanding,
or level 3 on the knowledge integration scoring rubric (M = .71, SD = .69), t(26) = -2.07, p =
.05. Students who began with partial understanding developed their ideas into a basic
understanding (level 4) after the Opener. In contrast, students who began with partial
understanding and had no Opener continued to demonstrate only partial understanding
after revising their work (M = .18, SD = .60). The large effect of the Opener on level 3 students
is partially due to the examples of student work selected for critique in the researcher and
teacher-designed Openers. The selected examples that were shown during the Opener
illustrated characteristics of level 3 understanding, making them most accessible to this
population of students. Since student work was chosen this way, this Opener was
unwittingly designed to increase the learning gains of students with an already basic
understanding of chemical reactions.
Students with non-normative ideas, or level 2 understanding, made modest gains with or
without an Opener, as shown in Figure 5. Level 2 students may need examples aligned with
their own ideas to improve to partial or high level understanding. For instance, showing
87
International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92, 2012

student work with no conservation of mass or bond breaking would lead to a discussion
where students point out that both are missing from the drawing. Alternatively, showing a
common level 2 student work may show this population of students that their attempt is
acknowledged and that they have direct feedback for improving their drawing.

Pre Opener: No Links Present Post Opener: No Links Present

Figure 5. Examples of students’ work with a level 2 understanding before and after an
Opener.
Table 6. Pre to Post Opener Gain Scores Distributed by Pre-Opener Scores
Pre- N Pairs Who Mean Gain N Pairs who Revised Mean Gain Without
Opener Revised Their With Opener (SD)
Score Work Opener(SD)
2 10 .4(.70) 5 .4(.89)
3 17 .71(.69) 11 .18(.60)*
4 42 .21(.42) 10 0(0)
*p=.05
Pre-test to Post-test effects. The pre- and post- tests were administered before students began
the week long project and after they finished the project. In spite of the gains found for the
specific item addressed by the Opener, there was no difference on the pre/post tests
between students who had an Opener (M = 1.74, SD = 2.50) and those who did not (M = 1.55,
SD = 2.16), t(255) = -.511, p = .61. This is not surprising since the Opener treatment was only
10-20 minutes out of the 5-7 hour long project. Perhaps several Openers over the project
length would have been able to affect the pre test to post test results. Also, one of the
teachers in the study did not give the post test until 2 weeks after the project was
completed. Although the schedule was controlled, the teacher ultimately has the authority
on when to give the post test. This delay may have masked any immediate effects of the
Opener on the post test. Because of these time related anomalies, student performance on
the embedded assessment is a better representation of the immediate effects of the
Openers.
How does the Opener design influence students’ learning outcomes?
The Opener designs, described in Table 2, differed primarily in their support for
distinguishing and integrating ideas. In both Openers, students were presented with
evidence regarding a chemical reaction and prompted to think about conceptual features of
a chemical reaction. The researcher-designed Opener had additional components to support
integration of ideas. After considering the new evidence, students were guided to reconsider
88