ERIC EJ728478: Using Vignettes To Build and Assess Teacher Understanding of Instructional Strategies

2005·0.2 MB·English

by  ERIC

Using Vignettes To Build and Assess Teacher
Understanding of Instructional Strategies
Carolyn Jeffries Dale W. Maeder
California State University, Northridge University of California, Los Angeles
Abstract
In the last fifty years, the use of stories in education has included vignettes as an effective stimulus for discussion
of real-life contexts and problems. However, vignettes have rarely been used as an assessment tool and there is no
reported consensus on their definition and design. This article documents the use of vignettes as an effective method
of assessing pedagogical understanding in our teacher development courses from 1995-2003, suggesting that vignettes
are significantly correlated with more traditional forms of assessment, are highly predictive of course-ending project
performances, and represent an episode of learning in their own right. Finally, we propose a more concise definition
and a more rigorous course of study for vignette development and implementation.
A critical theme of institutions of higher contexts and problems.
education in the preparation and continuing Vignettes have rarely been reported as a
education of teachers is that educational research means of assessing mastery of course content.
informs and is informed by practice. Identifying Furthermore, there are few definitions or con-
and assessing what and how teachers learn during struction criteria for vignettes in these contexts,
their professional development is an important and certainly no reported consensus on their
component in assisting them to better teach their definition and design. Given the benefits of using
students. The use of stories in various formats has vignettes in research, modeling, discussion, and
long been a powerful and successful method for reflection, and the need to determine teacher
modeling, teaching, and researching behavior and qualification, it is important to determine whether
understanding in general education, health sci- vignettes can also be an effective assessment tool
ences, social sciences, and behavioral sciences. in teacher education.
One type of story, the vignette, has been used An important step in this process is the collec-
in the social sciences and more recently in the tion of evidence that vignettes are a reliable as-
health and education fields as a reflective, re- sessment tool that complements more traditional
search, and modeling tool (Callicott, 2003; Camp- assessment styles. In addition, it should be deter-
bell, n.d.; Chambers, 1999; Galguera, 1998; Hughes mined whether vignettes can augment the infor-
& Huby, 2002; King, Murray, Salomon, & Tandon, mation gathered in more traditional assessments.
2002a; Kruse, 1997; New Hampshire Equity Hand- Besides content and skill mastery, do vignettes
book Writing Team, n.d.; Pransky & Bailey, 2002; help to measure affective goals, such as the moti-
Schwartz & Riedesel, 1994; TESOL, 2001; Vaughn vation to learn? Do vignette assessments provide
& Klingner, 1999; Volkmann, 1998). Researchers evidence of problem solving and critical thinking?
and educators have found vignettes to be very Furthermore, it is critical to determine if
effective in these contexts for several reasons: vignettes can assist in state-mandated assessment
vignettes are relatively easy to construct, they of teacher understanding of pedagogy. The chal-
provide a useful focus and stimulus for discus- lenge of assessing portfolios may be partially
sion, they are valuable in addressing difficult-to- addressed through vignette scoring guides that
explore and sensitive topics, they can be used with quantify responses that are unique and authentic.
individuals and groups, and they reflect real-life Finally, vignettes should be studied for evidence
Volume XXVII • Number 1 & 2 • Fall 2004 & Spring 2005 17
Carolyn Jeffries & Dale W. Maeder
of an assessment as an episode of learning. Does opment Oriented Research in Agriculture, n.d.;
an assessment task that encourages modeling, Pesonen, n.d.), and vignettes.
rehearsal, and transfer provide more opportuni-
ties for learning than more traditional forms of Vignettes in Education and Research
assessment, (e.g., multiple-choice, fill-in items)? A recent review of the literature on vignettes
Since 1995, we have been using vignettes as a revealed a wide range of definitions, usage, con-
method of assessing pedagogical understanding struction, testing, and examples. Definitions were
in teacher development courses. We believe the rarely provided and varied greatly, which caused
vignette to be a highly effective assessment tool confusion of vignettes with other types of stories.
and suspect that vignettes have a strong predic- In fact, many studies and articles used the terms
tive component as well as provide a rich learning case study, case story, scenario, and vignette
experience for eliciting and promoting teacher interchangeably, incorrectly, or uniquely, (e.g.,
understanding. We would like to share our experi- Jackson, 1998; Kruse, 1997; Walen & Hirstein,
ences and data with teacher educators and re- 1995; Weigle & Scotti, 2000; Weiner, Rand, Paga-
searchers in order to propose a course of study on no, Obi, Hall, & Bloom, 2001).
the use of vignettes in assessing pedagogical Broadly defined, vignettes are “short stories
understanding in teacher education programs. about hypothetical characters in specific circum-
Based on the literature and our experiences, we stances to whose situation the interviewee is
would also like to propose a more concise defini- invited to respond…moving from the abstract to
tion and establish more rigorous criteria for context-specific” (Finch, 1987, p. 106). Vignettes
vignette development and implementation in this consist of text, images, or other forms of stimuli,
assessment process. ranging from short written prompts to live events,
to which research participants are asked to re-
Review of Literature spond (Hughes & Huby, 2002). Herman (1998)
Stories as Educational Tools uses the term case vignette to describe a written
A story is defined by Cambridge Advanced description, photograph, or videotaped scene as a
Learner’s Dictionary (Cambridge University Press, brief glimpse of an educational situation. Camp-
2003) as “a description, either true or imagined, of bell (n.d.) comes closest to an operational defini-
a connected series of events and, often, the charac- tion:
ters involved in them.” Stories have been used in A vignette is a short story without an ending.
many different forms and formats as a powerful It is short, but not too short to present an
and successful method for modeling, teaching, issue. It is detailed, but not so detailed that the
and researching behavior and understanding in underlying issue gets lost. A vignette presents
general education, health sciences, social sciences, an issue, such as the under-representation of
and behavioral sciences. Story formats include case girls in advanced math courses, in a context
studies, long narrative descriptions of real or with which individuals can identify. A good
hypothetical situations in which learners are vignette has fewer complexities and personali-
asked to identify or solve a problem (Gideonse, ties than real life, sets up a situation in which
1999; Hrabe, Kinzie, & Julian, 2001; Jackson, 1998; there is no one “right” answer, and is flexible
Loughner, Harvey, & Mil-heim, 2001; Marsick, enough that individuals from different groups
1998); case stories, stories that simulate the real (teacher/administrator, female/male, liberal/
world but are written by individuals within the conservative) can identify with the story and
classroom and told from their perspectives (Hunt- bring their perspective forward in discussions
er & Hatton, 1998; Maslin-Ostrowski & Ackerman, of solutions. (p. 2)
1998); scenarios, narrative descriptions that contain
a set of realistic assumptions and facts about the Vignette Construction and Design
future used to provide a unified context for In addition to a lack of consensus on vignette
decision-making (International Centre for Devel- definition, there was a noticeable lack of informa-
18 The Professional Educator
Using Vignettes to Assess Teacher Understanding
tion on how vignettes were constructed or should the vignette character’s intellectual, cultural, and
be constructed. When using vignettes to promote religious background as well as physical attributes
discussion and problem solving concerning equity and personal information. They recommended
in math and science education, Campbell (n.d.) that vignettes be written so that people from
proposed a vignette construction process and different backgrounds could understand them as
shared samples of vignettes for different target similarly as possible, that details be scrutinized to
groups, (e.g., administrators, teachers, students, avoid introducing unwarranted assumptions, and
parents, policy makers, researchers). She listed that vignette characters be as similar as possible to
three major steps in creating vignettes: determine the audience.
issues and areas of concern, develop situations When authors did provide samples of their
that are realistic and relevant, and test the vi- vignettes, a wide range of formats were found that
gnettes on groups similar to those who will be included short and long dialogues, formulas,
using them (Campbell, n.d.). pictures, and short and long narratives with and
In their review of the use of vignettes as a without follow-up questions. Vignettes ranged
research tool in the behavioral sciences and health from 25 to 1000 words in length. The accompany-
care, Richman and Mercer (2002) proposed an ing questions and tasks varied in terms of num-
alignment along eight axes in the construction and ber, length, type, and amount of detail.
application of vignettes. These axes addressed the
function of the vignette, method of implementa- Vignette Usage
tion, collection, type of respondents, content The lack of consensus of vignette definition
covered, time available, type of response elicited, and format may be due to its diverse usage across
and ethical issues. Richman and Mercer put forth fields that seldom overlap or confer with one
the design issue of whether to use already existing another. Vignettes have been used in the social
data in the construction of the vignette or to sciences since the 1950’s (Hughes & Huby, 2002)
construct material suited to the specific purpose. and more recently in the health sciences to model
They also posed the question of vignette delivery best practices (Callicott, 2003; Orlander, Gupta,
media—written documents, audiovisual, or oral— Fincke, Manning, & Hershman, 2000), to describe
while stating that the literature strongly suggested common situations and conditions (Pickett,
that vignettes are usually given in written docu- Streight, Simpson, & Brison, 2003; Stelmachers &
ments of varying lengths. Sherman, 1990), to study technique effectiveness
In their review of the application of vignettes (Goldie, Schwartz, McConnachie, & Morrison,
in social and nursing research, Hughes and Huby 2001), and to identify attitudes and beliefs (Sleed,
(2002) addressed the differences between vig- Durrheim, Kriel, Solomon, & Baxter, 2002; Thur-
nettes and real life processes and explored some ber, Heller, & Hinshaw, 2002).
practical advantages and pitfalls of using vi- Vignettes have also been used in the field of
gnettes. They found that vignettes provided a education as models of effective teaching (TESOL,
useful focus and stimulus for discussion, may be 2001; Vaughn & Klingner, 1999), to identify and
constructed from unrealistic events and real life study attitudes and beliefs (Galguera, 1998;
events, were valuable in detecting subtleties and Schwartz & Riedesel, 1994), and as a tool to sup-
nuances, and were useful in addressing difficult- port teachers in their development, reflection, and
to-explore and sensitive topics. The issues they problem-solving abilities (Campbell, n.d.; Kruse,
presented in constructing effective vignettes 1997; New Hampshire Equity Handbook Writing
included making the topics and contexts relevant Team, n.d.; Pransky & Bailey, 2002; Volkmann,
to the audience and addressing the readers abili- 1998). Only a few instances of vignettes used to
ties and styles. assess knowledge were found, (e.g., Cohen, Shete,
King, Murray, Salomon, and Tandon (2002b) Seal, Daum, & Lauderdale, 2003), and those were
addressed similar issues when constructing in the health field.
vignettes for surveys: how and when to address
Volume XXVII • Number 1 & 2 • Fall 2004 & Spring 2005 19
Carolyn Jeffries & Dale W. Maeder
Vignettes as a Written Assessment Tool in Teacher Method
Development Vignette Definition and Construction Criteria
Richman and Mercer’s (2002) view of vignettes We define vignettes as incomplete short stories
as a feasible alternative to observation and a that are written to reflect, in a less complex way, real-
flexible and fertile component of qualitative life situations in order to encourage discussions and
research suggests that a vignette analysis assign- potential solutions to problems where multiple solu-
ment might be an assessment tool to complement tions are possible.
more traditional forms of content or skill mastery. The five criteria for this type of vignette are:
Research conducted in the fields of health and
behavioral science (Hughes & Huby, 2002) sug- (1) It is a story. It is a narrative but not a dia-
gests that vignettes could be used as an assess- logue, case study, case story, or scenario.
ment tool to collect evidence of mastery of skills, (2) It is short. Its length is 50-200 words.
terms, and concepts for the following reasons: (3) It is relevant. It simplifies a real-life situa-
tion that is relevant to participants but one
1. Vignettes provide a useful focus and stimulus in which no participant is likely to have
for discussion. expertise.
2. Vignettes may be constructed from unrealistic (4) It allows for multiple solutions/answers and is
events and real life events. intended to encourage independent thinking
3. Vignettes are valuable in detecting subtleties and unique responses. It includes a prompt
and nuances. with instructions and a set of tasks, i.e.,
4. Vignettes are useful in addressing difficult-to- specific issues to be addressed in the partic-
explore and sensitive topics. ipant’s response directly connected to a
5. Vignettes can quickly generate considerable scoring guide.
amounts of data from a large participant The use of vignettes creates an assessment
group. situation where authenticity of authorship is less
6. Vignettes can be defined and standardized to likely to be compromised. Unlike forced-choice
enable all participants to respond to the same assessment, (e.g., true/false, multiple-choice, fill-
stimulus. in), vignette responses are open-ended and allow
7. Vignettes do not necessarily require partici- for more unique demonstrations of content mas-
pants to have in-depth knowledge of the tery. Each participant chooses and defends a
topics under study. position during the analysis of the vignette. The
evaluator uses a scoring guide to verify content
Given the positive aspects of vignette use, we mastery and determine if the work has been
performed an analysis of data collected between plagiarized from a classmate or another source.
1998–2003 in two online teacher education courses
that used vignettes as an assessment task to elicit (5) It is purposely incomplete. It can be truncated
teacher pedagogical understanding. Based on the — plot line stops at a critical juncture and
research literature and our experiences and re- participants complete the vignette — or
sults, we believe that vignettes are a strong candi- abridged — story’s details are omitted so
date for assessing teacher understanding and that multiple interpretations can be de-
predicting appropriate teacher implementation of fended.
instructional strategies. In a truncated vignette participants are asked
to complete a storyline according to a set of crite-
ria defined by the course curriculum. Truncated
vignettes are typically used to evaluate process,
rather than product, i.e., examining a student’s
20 The Professional Educator
Using Vignettes to Assess Teacher Understanding
completion of the storyline to assess if a particular ior. Be sure to identify clues in the vig-
problem-solving skill has been mastered. In an nette, information from the readings, or
abridged vignette participants are asked to dem- assumptions you made to defend your
onstrate mastery of course content by answering response.
specific questions concerning the vignette and (2) Choose from Mr. Bunsen’s options, the
justifying their positions. Abridged vignettes are readings’ list of classroom management
typically used to evaluate product, rather than strategies, and/or your own ideas to rec-
process, i.e., examining a student’s analysis of a ommend a set of options for Mr. Bunsen.
situation to assess if specific knowledge has been Be sure to defend your selections.
mastered.
The following truncated vignette concerning Note the range of possible responses to these
classroom management is drawn from a teacher tasks. Although the classroom management
development course taught by one of the research- problem is clearly identified in the vignette, the
ers: cause is not and there are numerous ways for Mr.
Bunsen to solve the problem, e.g., clarify the
A colleague of yours, Bernie Bunsen, is a assignment, remind students of consequences, be
middle school science teacher who designates consistent and caring with the students, and help
the last 30 minutes of each class meeting for the misbehaving student change his behavior.
one week to complete lab activities on the The truncated vignette task is more open-
various states of water. To the delight of his ended than that of the abridged vignette, which
classmates, one student has been fooling provides scaffolding for multiple interpretations
around during the lab time, tossing ice cubes linked to the course content. The following abridg-
around the room, sabotaging the fog machine, ed vignette concerning scaffolding is drawn from
disrupting other group work, etc. Some of his the same teacher development course:
classmates are starting to imitate him to get
the same reactions from others. Mr. Bunsen is Random Guess is a high school math teacher
obviously concerned about the lack of produc- who provides a free, two-hour problem-solv-
tive work getting done as well as the safety ing workshop for juniors and seniors prepar-
hazards caused by the misbehavior. Mr. Bun- ing for their college admissions exams. Once a
sen has confided in you that he is deciding week after school during the fall semester,
among several options: send the lead misbe- students meet with Mr. Guess to work on
having student out of the classroom for deten- sample math problems. This year, he has
tion during lab time, assign him more science decided to try out some new techniques in his
work, fail him on the lab assignment, punish workshop. In addition to simply answering
the entire class for the misbehavior, speak questions posed by the students concerning
privately with the student, speak with the problems in a handout that he has given them
entire class about the misbehavior and the the prior week, he has decided to (a) review a
importance of safety measures, give the stu- math topic; (b) solve one or two related prob-
dent the option of re–submitting his lab work lems on the board while thinking out loud; (c)
for a partial score increase. work with the class to solve several more
problems on the board; (d) administer a short
Complete the following tasks: individual quiz of selected problems; and (e)
review the quiz problems before handing out
(1) Using the information provided in this next week’s handout of problems.
week’s readings, identify the type of mis-
behavior or mistaken behavior the disrup- Complete the following tasks:
tive student in the vignette is exhibiting
and discuss possible causes for his behav- (1) Analyze what Mr. Guess is doing to help these
Volume XXVII • Number 1 & 2 • Fall 2004 & Spring 2005 21
Carolyn Jeffries & Dale W. Maeder
students learn in terms of Vygotsky’s theories Vignette Assignment Descriptions
and their instructional implications in relation In the first and second vignette assignments
to scaffolding and social learning. Be sure to (abridged), the participants were given descrip-
defend your answer with examples, clues in tions of teaching situations and asked to classify
the vignette, or assumptions that you have and evaluate one of them in terms of various
made. issues of teaching and learning theory. Each vi-
(2) Describe two ways Mr. Guess could modify his gnette’s description was purposely vague so that
instructional plan to promote input from various interpretations were possible, allowing the
others. participant to find clues in the vignette description
that indicated the presence of a learning strategy.
Note again the range of possible responses to In the third and fourth vignette assignments
these tasks. There are several clues within the (truncated), participants were asked to adapt a
vignette to indicate evidence of scaffolding in Mr. hypothetical online lesson to fit a specific learning
Guess’ teaching and there are numerous ways to theory and then create an assessment plan for
promote student input, e.g., cooperative learning measuring participant performance. The vignettes
groups, student demonstrations, teach-back were scored by the instructor using a scoring
activities. guide, (see Table A1 in the Appendix), which
Vignettes differ from case studies which are weighed the defense of a choice more heavily than
longer, more detailed reports to be analyzed and a particular choice. These vignette assignments
discussed. Nor is the vignette a case story which were therefore used to measure mastery of course
is written by the student-participant and focuses content and transfer of those concepts to new
on his/her perspective. Finally, vignettes are contexts.
distinguished from scenarios which are future-
based stories. Results
To collect evidence of reliability of the vignette
Setting and Context as an assessment tool, means and correlation
Evidence supporting the use of vignettes as an coefficients were calculated using data for each of
effective learning tool and means of assessing the participants’ four vignette scores, (a cumula-
participants’ pedagogical understanding was tive vignette score with 100 possible points), a
collected in over 30 sections of two online teacher midterm score (100 possible points), and a final
education courses between 1998 and 2003. The project score (100 possible points). An alpha level
participant population consisted of employed of .05 was used for all statistical tests.
school teachers, college instructors, and corporate Mean vignette, midterm, and final project
trainers. The lectures and the use of vignettes as a scores were not significantly different across the
teaching and assessment tool in the two courses two courses. Moreover, when comparing earlier
were identical. Course topics included models of sections of each course (1998–2000) to more recent
learning theory, models of teaching theory, the sections (2000–2003), mean scores were not signifi-
application of learning theory to online education, cantly different. These findings together suggested
and distance learning assessment theory. that the scoring guides for the various assign-
Course assessment included two vignette ments were consistently administered from sec-
assignments, a midterm, two additional vignette tion to section and from course to course over the
assignments, and a final project. The midterm years and that any differences in the participant
included four vignettes to be analyzed as well as populations did not significantly affect vignette,
five essay questions. The final project was the midterm, or project performances. The data for all
creation of a detailed proposal for an online 610 participants were therefore considered collec-
course or training program. tively.
Cumulative vignette scores (M = 91.77, SD =
6.24) were not significantly different from mid-
22 The Professional Educator
Using Vignettes to Assess Teacher Understanding
term scores (M = 92.00, SD = 4.32) or final project Discussion
scores (M = 92.71, SD = 5.22). Intercorrelations Preliminary results in two online teacher
between cumulative vignette and midterm scores education courses indicated a significant link
were r = .27, p < .0001, between cumulative vi- between vignette and midterm scores and sug-
gnette and final project scores r = .21, p < .0001, gested that vignettes may be a reliable assessment
and between midterm and final project scores r = tool in measuring teacher pedagogical under-
.09, p < .03. standing and an area of research worth further
Since there was a significant overlap of course study. Since 50% of the midterm points were
content between the vignettes and the midterm, it vignette items, this correlation was a measure of
was expected that their pairing would produce the reliability, i.e., the scoring guides were consis-
greatest correlation coefficient. There was less tently used and participants tended to perform
content overlap between the final project and the similarly on the two assessments. Since the final
earlier assessments. It was nevertheless notewor- project was not vignette-related but measured the
thy that the correlation coefficient between the same course content, the correlation between
vignette scores and the final project scores was vignette and final project scores was a measure of
nearly the same as the vignette-midterm correla- construct validity, i.e., evidence that supported
tion, while the midterm and final project scores decisions concerning final projects and overall
produced a much smaller correlation coefficient. course grades.
These findings suggested that the vignette score
was better than the midterm as a predictor of final Vignette Assessment as a Learning Event
project performance. The re-submission data provided additional
Vignette scores were then sorted into three support to the notion that the vignette assessment
groups (lower, average, higher) to see if vignette is an episode of learning in and of itself. The
scores could successfully predict midterm and participants quickly learned how to complete
project scores. Mean midterm and final project vignettes and remarked that they understood the
scores were significantly different among the three course content much more clearly following the
sorted vignette groups. When midterm scores vignette assignment. Wolf (1993) refers to assess-
were sorted into three groups using the same ment as a “heads-on encounter with a culture’s
cutoff values, mean final project scores were not models of prowess. Assessments publish what we
significantly different among the three midterm regard as skill and what we will accept or reject as
groups. These two findings again suggested that a demonstration of accomplishment” (pp. 213-
the vignette score was better than the midterm as 214). An assessment therefore allows students to
a predictor of final project performance.1 see their work as someone else sees it.
In each section of the two courses, approxi- The familiar assessments, e.g., standardized
mately 10% of the participants posted incomplete multiple-choice tests, are often the scapegoat
responses to the first vignette assignment that example of assessment as a missed learning
needed to be resubmitted. These respondents opportunity. Students are not always given sam-
initially failed to address all of the tasks in the ples of different levels of performance or the
instructions (e.g., not defending their choices with criteria that define those levels. They rarely see the
vignette clues). In all cases, resubmitted vignette test booklet again but instead are handed a brief
scores were substantially higher. For these partici- summary report or, worse, just a score. Few are
pants, the first vignette assignment evidently the opportunities to discuss problem-solving
incorporated scaffolding as part of the learning strategies or attempt a second try. Yet, the re-
technique. The 10% resubmission figure dropped search is clear that worthwhile work requires
to nearly zero on subsequent vignette assign- incubation, revision, and collaboration.
ments, which suggested that participants quickly Vignette assignments are particularly well-
learned how to complete vignette assignments. suited to serving two learning purposes: as an
opportunity to learn, they encourage reflection,
Volume XXVII • Number 1 & 2 • Fall 2004 & Spring 2005 23
Carolyn Jeffries & Dale W. Maeder
rehearsal, motivation, and collaboration; and as an • delivery mode (face-to-face, online, blend-
assessment tool, the detailed analysis behind the ed, text-based, video)
score allows for re-submitted work and further • instructional method (lecture, instructor-
reflection. Wolf (1993) refers to a zig-zag path moderated discussion, learner-generated
between assessment and instruction, considering summaries)
which assessment practices will protect, nudge, • assessment type (learner-completed vi-
and inform the students during the long course of gnettes, learner-generated vignettes)
work. As learners plot a course between humility • vignette type (truncated vs. abridged)
and excellence, the assessment becomes an epi- • time mode (synchronous vs. asynchron-
sode in which students learn how to write, con- ous)
duct research, experiment, or solve problems • collaboration (cooperative learning vs.
along this zig-zag path. collaborative assessment)
One benefit of the vignette assignment is that • content area (other than instructional
the target of the assessment shifts so that instruc- development)
tors and learners rate performances. Assessments
can promote higher-order critical thinking skills Furthermore, the following research issues need to
just as well as lectures or activities. The assess- be addressed:
ment episode is not necessarily a terminal one; the
learning goes on. They need not be individualistic; • refining the definition of vignette
collaboration keeps assessment dynamic. And the • refining the vignette scoring guide, includ-
need for test security and authentic authorship is ing equating two or more vignettes used
lessened; instructors can capture a much broader in the same assessment
and more thorough set of snapshots of the learn- • collecting evidence of the reliability of
ers’ performances. As a result, the assessment’s vignettes as an instructional and assess-
reliability is strengthened, making its inferences ment tool
more valid. • collecting evidence of the validity of infer-
ences made following vignette assessment,
Suggestions for Further Study including course grades and teacher quali-
Based on these initial results we believe the fication decisions
vignette to be a highly effective assessment tool • comparing vignette assessments with
which has a strong predictive component and essays, summaries, and forced-choice test
provides a rich learning experience for eliciting items
and promoting teacher understanding. Herman’s
(1998) case vignette research, involving a related Conclusion
instructional tool in a similar setting, suggests that The beauty of the vignette activity is that, by
vignettes help teachers apply theoretical con- its very nature, learners must transfer their learn-
structs and research findings to classroom situa- ing to other situations and in doing so integrate
tions. We encourage teacher educators and re- their knowledge and skills well enough to make
searchers to engage in further study on the use of predictions about new situations. It is important to
vignettes in assessing pedagogical reasoning in provide a scoring guide to the participants before-
teacher education programs. Trends worthy of hand as well as some sample vignette responses to
further investigation include vignette effectiveness give them a clear indication of what comprises an
in formative and summative assessment, in assess- effective vignette analysis. Although this entails
ing problem-solving skills, and in assessing trans- time spent away from teaching course content, the
fer of knowledge and skills from one educational potential for making the assessment activity an
setting to another. episode of learning and for encouraging transfer
Further research is suggested to isolate the of learning to new situations is well worth the
study of vignettes from the following variables: extra time.
24 The Professional Educator
Using Vignettes to Assess Teacher Understanding
References Cliffs, NJ: Educational Technology Publica-
Callicott, K. J. (2003). Culturally sensitive collabo- tions.
ration within person-centered planning. Focus Hughes, R., & Huby, M. (2002). The application of
on Autism and Other Developmental Disabilities, vignettes in social and nursing research.
18(1), 60–69. Journal of Advanced Nursing, 37(4), 382–386.
Cambridge University Press. (2003). Cambridge Hunter, J., & Hatton, N. (1998). Approaches to the
Advanced Learner’s Dictionary. Cambridge, writing of cases: Experiences with preservice
UK: Cambridge University Press. Master of Teaching students. Asia-Pacific
Campbell, P. B. (n.d.). Collaboration for equity: How Journal of Teacher Education, 26(3), 235–246.
would I handle that? Using vignettes to promote International Centre for Development Oriented
good math and science education. American Research in Agriculture. (n.d.). Scenario and
Association for the Advancement of Science. Strategy Development. Retrieved March 11,
Retrieved June 27, 2003, from http://ehrweb. 2004, from http://www.icra.edu.org/page.
aaas.org/ehr/3_2_2.html cfm?pageid=anglolearn scenarios
Chambers, N. (1999). Close encounters: The use of Jackson, J. (1998). Cross-cultural teaching cases:
critical reflective analysis as an evaluation Vehicles for teacher development. Paper pre-
tool in teaching and learning. Journal of Ad- sented at the 1998 Korea Teachers of English
vanced Nursing, 29(4), 950–957. to Speakers of Other Languages (KOTESOL)
Cohen, N. J., Shete, P. B., Seal, J. B., Daum, R. S., & Conference. Retrieved March 11, 2004, from
Lauderdale, D. S. (2003). Physician knowl- http://www.kotesol.org/pubs/proceedings/
edge of catch-up regimens and contraindica- 1998/jackson_j.shtml
tions for childhood immunizations. Pediatrics, King, G., Murray, C., Salomon, J. A., & Tandon, A.
111(5), 925–932. (2002a). Enhancing the validity and cross-cul-
Finch, J. (1987). The vignette technique in survey tural comparability of survey research. Anchor-
research. Sociology, 21, 105–114. ing Vignettes. Retrieved June 10, 2003, from
Galguera, T. (1998). Students’ attitudes toward http://gking.harvard.edu/vign
teachers’ ethnicity, bilinguality, and gender. King, G., Murray, C., Salomon, J. A., & Tandon, A.
Hispanic Journal of Behavioral Sciences, 20(4), (2002b). Anchoring vignettes: Frequently asked
411–429. questions. Anchoring Vignettes. Retrieved
Gideonse, H. D. (1999). What is a case? What June 10, 2003, from http://gking.harvard.
distinguishes case instruction? In M. R. Sudzi- edu/vign/faq/
na (Ed.), Case study applications for teacher Kruse, S. D. (1997). Reflective activity in practice:
education: Cases of teaching and learning in the Vignettes of teachers’ deliberative work.
content areas (pp. 1–7). Needham Heights, Journal of Research and Development in Educa-
MA: Allyn & Bacon. tion, 31(1), 46–60.
Goldie, J., Schwartz, L, McConnachie, A., & Morri- Loughner, P. D., Harvey, D. M., & Milheim, W. D.
son, J. (2001). Impact of a new course on stu- (2001). Web-based instructional methods for
dents’ potential behaviour on encountering corporate training curricula. In B. H. Khan
ethical dilemmas. Medical Education, 35, 295- (Ed.), Web-based training (pp. 185–190). Engle-
302. wood Cliffs, NJ: Educational Technology
Herman, W. E. (1998). Promoting pedagogical Publications.
reasoning as preservice teachers analyze case Marsick, V. J. (1998). Case study. In M. W.
vignettes. Journal of Teacher Education, 49(5), Galbraith (Ed.), Adult learning methods: A guide
391–397. for effective instruction (2nd ed., pp. 197–218).
Hrabe, M. E., Kinzie, M. E., & Julian, M. F. (2001). Malabar, FL: Krieger.
Web-based case studies: A multipurpose tool Maslin-Ostrowski, P., & Ackerman, R. H. (1998).
for the training toolkit. In B. H. Khan (Ed.), Case story. In M. W. Galbraith (Ed.), Adult
Web-based training (pp. 451–458). Englewood learning methods: A guide for effective instruction
Volume XXVII • Number 1 & 2 • Fall 2004 & Spring 2005 25
Carolyn Jeffries & Dale W. Maeder
(2nd ed., pp. 303–316). Malabar, FL: Krieger. k-12 students Retrieved June 10, 2003, from
New Hampshire Equity Handbook Writing Team. http://www.tesol.org/assoc/k12standards/
(n.d.). Equity is more than coping with change. it/05.html
Eisenhower Regional Consortium, Regional Thurber, J. R., Heller, T. L., & Hinshaw, S. P.
Alliance for Mathematics and Science Educa- (2002). The social behaviors and peer expecta-
tion in the Northeast and Islands. Retrieved tion of girls with attention deficit hyperactiv-
June 10, 2003, from http://ra.terc.edu/ ity disorder and comparison girls. Journal of
initiatives/state_connections/NH/nh-eq- Clinical Child and Adolescent Psychology, 31(4),
uity/toc.html 443–452.
Orlander, J. D., Gupta, M., Fincke, B. G., Manning, Vaughn, S., & Klingner, J. K. (1999). Teaching
M. E., & Hershman, W. (2000). Co-teaching: A reading comprehension through collaborative
faculty development strategy. Medical Educa- strategies. Intervention in School & Clinic, 34(5),
tion, 34, 257–265. 284–292.
Pesonen, H. (n.d.). Framework for Scenario Develop- Volkmann, M. J. (1998). Integrating field experience
ment in LCA. SETAC-Europe LCA Working and classroom discussions: Vignettes as vehicles
Group. Retrieved March 11, 2004, from for reflection. Retrieved June 10, 2003, from
http://www.oikos-stiftung.unisg.ch/ http://www.ed.psu.edu/CI/Journals/1998
campus2000/article.pdf AETS/f2_3_volkman.rtf
Pickett, W., Streight, S., Simpson, K., & Brison, R.J. Walen, S. B., & Hirstein, J. (1995). Classroom
(2003). Injuries experienced by infant chil- vignette: An alternative-assessment tool.
dren: A population-based epidemiological Teaching Children Mathematics 1(6), 362–365.
analysis. Pediatrics, 111(4), 365–370. Weigle, K. L., & Scotti, J. R. (2000). Effects of func-
Pransky, K., & Bailey, F. (2002). To meet your tional analysis information on ratings of
students where they are, first you have to intervention effectiveness and acceptability.
find them: Working with culturally and Journal of the Association for Persons with Severe
linguistically diverse at-risk students. Reading Handicaps, 25(4), 217–228.
Teacher, 56(4), 370–383. Weiner, L., Rand, M., Pagano, A., Obi, R., Hall, A.,
Richman, J., & Mercer, D. (2002). The vignette & Bloom, A. (2001). Illuminating the impact
revisited: Evil and the forensic nurse. Nurse of state educational policy promoting school
Researcher, 9(4), 70–82. reform on curriculum and instruction in
Schwartz, J. E., & Riedesel, C. A. (1994, February). programs of urban teacher preparation.
The relationship between teachers’ knowledge and Educational Policy, 15(5), 644–673.
beliefs and the teaching of elementary mathemat- Wolf, D. P. (1993). Assessment as an episode of
ics. Paper presented at the annual meeting of learning. In R. E. Bennett & W. C. Ward
the American Association of Colleges of (Eds.), Construction versus choice in cognitive
Teacher Education, Chicago, IL. measurement: Issues in constructed response,
Sleed, M., Durrheim, K., Kriel, A., Solomon, V., & performance testing, and portfolio assessment (pp.
Baxter, V. (2002). The effectiveness of the 213–230). Hillsdale, NJ: Erlbaum.
vignette methodology: A comparison of
written and video vignettes in eliciting re-
sponses about date rape. South African Journal Author Note
of Psychology, 32(3), 21–29. Carolyn Jeffries, Assistant Professor, Michael D.
Stelmachers, Z. T., & Sherman, R. E. (1990). Use of Eisner College of Education, California State
case vignettes in suicide risk assessment. University, Northridge. Dr. Jeffries specializes in
Suicide and Life Threatening Behavior, 20(1), 65– instructional design, educational psychology, and
84. science instruction.
TESOL (Teachers of English to Students of Other
Languages). (2001). The ESL standards for pre- Dale W. Maeder, Instructor, Education Extension,
26 The Professional Educator