ERIC ED427524: The Multiple Challenges of Multimedia: Development, Implementation, and Evaluation.

24 Pages·1998·0.36 MB·English

by  ERIC

DOCUMENT RESUME
FL 025 696
ED 427 524
Kassen, Margaret Ann
AUTHOR
The Multiple Challenges of Multimedia: Development,
TITLE
Implementation, and Evaluation.
ISSN-0898-8471
ISSN
1998-00-00
PUB DATE
23p.; For the complete volume of working papers, see FL 025
NOTE
687.
Journal Articles (080) -- Reports - Descriptive (141)
PUB TYPE
Texas Papers in Foreign Language Education; v3 n3 151-71
JOURNAL CIT
Fall 1998
EDRS PRICE MF01/PC01 Plus Postage.
Authoring Aids (Programming); Classroom Techniques;
DESCRIPTORS
*Computer Software Development; *Computer Software
Evaluation; Educational Technology; Higher Education;
Language Research; *Listening Comprehension; *Multimedia
Instruction; Program Descriptions; Second Language
Instruction; Second Language Learning; *Second Languages;
*Spanish
Catholic University of America DC
IDENTIFIERS
ABSTRACT
This article presents an overview of an ongoing initiative
to develop, integrate, and evaluate multimedia lessons to enhance listening
comprehension in Spanish language classes at the Catholic University of
America (District of Columbia). Focusing on the process involved, the two
phases of the project that have been completed are examined, including the
rationale, selection of the Libra authoring system, the building of the
authoring team, design and revision of the lessons, and their integration
into the curriculum. Findings from the evaluation components are explored,
with particular emphasis on evidence pointing to the need to understand
better how learners actually interact with multimedia. Based on the lessons
learned from this project, guidelines are suggested for those who may be
interested in a similar undertaking. A student survey form is appended.
Contains 21 references.
(MSE)
********************************************************************************
Reproductions supplied by EDRS are the best that can be made
from the original document.
********************************************************************************
The Multiple Challenges of Multimedia: Development, Implemen-
tation, and Evaluation
MARGARET ANN KASSEN, The Catholic University
of America
OF EDUCATION
AND
U.S. DEPARTMENT
PERMISSION TO REPRODUCE
Research and Improvement
MATERIAL HAS
Office of Educahonal
INFORMATION
DISSEMINATE THIS
EDUCATIONAL RESOURCES
BEEN GRANTED BY
CENTER (ERIC)
ell'his document has been reproduced as
Organization
received from the person or
COrp.0-1/
originating it.
have been made to
0 Minor changes
quality.
improve reproduction
RESOURCES
TO THE EDUCATIONAL
opinions stated in this
(ERIC)
Points of view or
INFORMATION CENTER
represent
document do not necessarily
policy.
1
official OERI position or
.BEST COTT
AVAILABLE
The Multiple Challenges of Multimedia: Development, Implemen-
tation, and Evaluation
MARGARET ANN KASSEN, The Catholic University of America
This article presents an overview of an ongoing initiative to develop, in-
to enhance listening compre-
tegrate, and evaluate multimedia lessons
hension in Spanish language classes at The Catholic University of Amer-
the two phases that
ica. In an effort
to focus on the process involved,
have been completed are examined, including the rationale for the proj-
of the
of the Libra authoring
ect, the selection
building
system,
the
authoring team, the design and revision of the lessons, and their integra-
tion into the curriculum. Findings from the evaluation components are
explored with particular emphasis on the evidence pointing to the need
actually interact with multimedia.
to understand
better how learners
Based on the lessons learned from this project, guidelines are suggested
for those who may be interested in pursuing a similar undertaking.
INTRODUCTION
In the second language teaching field today, there is great interest in
technology and its potential to enhance second language learning. Multime-
dia has a particular appeal, due in part to what Pusack and Otto (1997) refer to
as its "capacity to access and control via computer a full range of familiar me-
dia: text, motion video, photo images, sound and graphics" (p. 2). Many of the
materials teachers have previously used effectively to promote language
learning can be placed at the learners' fingertips in the digital environment
with the promise of increased learning outcomes. The allure of multimedia
must be tempered, however, with the realities that confront any instructional
innovation: the challenges of development, implementation, and evalua-
tion. How does one go about developing multimedia lessons for use in mul-
tiple-section courses? How can these lessons be integrated in the existing cur-
riculum? How can multimedia be evaluated? These fundamental questions
will be examined in the context of a multimedia project for beginning level
Spanish undertaken at The Catholic University of America (CUA).
BACKGROUND OF THE PROJECT
The Department of Modern Languages at CUA was interested in explor-
ing language learning technology, particularly as a way to enhance students'
out-of-class language learning. This option had become feasible due to our re-
cently installed Multimedia Language Center, a small, independent study
room equipped with computers, VCRs, laserdisc, etc. As a first step, the media
director and language coordinator attended a workshop on Libra, a multime-
dia authoring system designed to facilitate listening comprehension. Libra
was developed by Robert Fischer and Mike Farris at Southwest Texas State
University at San Marcos [1] with the support of the Fund for the Improve-
ment of Post-Secondary Education (FIPSE). Because the listening comprehen-
Texas Papers in Foreign Language Education
152
sion goals of CUA's Spanish pro-
not
interaction
of
the
facilitate
learners with the text other than
gram Destinos were compatible with
through their cumbersome replay
those of Libra, the team applied for
option. Multimedia, however, be-
and received a grant to develop, im-
plement, and evaluate Libra lessons
cause of its digital nature, allows di-
as a part of Fischer's
rect access to and manipulation of
for Destinos
particular segments of text and, by
FIPSE project.
means of various help features, of-
fers the user added options for man-
LISTENING COMPREHENSION
As Joiner
the interaction.
aging
AND TECHNOLOGY
A focus on listening compre-
(1997) notes,
hension at the novice and interme-
diate levels is well supported by re-
multimedia
Computer-assisted
search and theoretical models that
comes closer than the other audio
and visual media to meeting the
point to comprehensible input and
standard for listening embodied
to second
as fundamental
intake
in
face-to-face
communication.
Rubin
acquisition.
language
As
Presence, interactivity,
control,
(1994) noted in her review of the
multisensory input, and multiple
relevant literature, the actual process
sources of assistance can be incor-
porated into this
sophisticated
of listening has received consider-
technology. (p. 90)
able research attention. Comprehen-
sion has come to be seen as a dy-
LIBRA AUTHORING SYSTEM
namic, interactive, cognitive process
Libra is a hypercard-type program for
in which listeners construct mental
Macintosh [2] that allows the teacher
representations of their understand-
to create a stack of question cards
ing of texts, using and coordinating
written to target selected segments of
both concept-driven (top-down) and
a videotext. These question cards ap-
sentence and word-level (bottom-
pear on the computer screen and are
up) processes. Furthermore, in real-
linked to video clips that have either
world, face-to-face communication,
been digitized and are accessible on
listeners have immediate access to
the same screen or that are shown
about the context of
information
on a laserdisc player, as was the case
their interactions (participants and
in this study. The Libra documenta-
setting) and paralinguistic cues (such
tion recommends beginning each
as eye movement and gestures), and
lesson with a textmap to acquaint
they generally have some control of
learners graphically with the struc-
the interaction by means of negotia-
ture of the text. These advance or-
tion and use of strategies (Joiner,
ganizers may reappear before each
1997).
lead users from one
segment to
Given this
understanding of
segment to another
logically
(see
the listening process, how can tech-
Figure 1). The five question tem-
nology best be brought to bear on lis-
plates available in the program are
skill development? While
tening
flexible and may be used to focus
the
traditional
provide
cassettes
attention on various
learners'
as-
audio component and video the
pects of listening comprehension,
visual information, these media do
Multiple Challenges of Multimedia
153
from main ideas to details and spe-
they may access a number of help
words. For multiple
features such as a dictionary, a list of
cific
choice,
checklist, binary checklist, and icon
characters, a story summary, script,
sorting questions (see Figures 2, 3, 4,
and special notes.
and 5), learners select the correct re-
sponses by clicking or dragging and
THE PROJECT
get immediate feedback. The fifth
The multimedia project at CUA
question type is open-ended and al-
has gone through two phases, each
lows users to create their own an-
of which involved developing, im-
swers by typing into
a notebook.
plementing, and evaluating lessons.
While the teacher sets parameters by
deciding which portion of the video
Phase 1: Development and Imple-
is available on a given card, learners
mentation
are given considerable control
In the initial pilot study in the
of
their learning: they may rewind, re-
spring of 1995, the language coordi-
nator/media director team and a
play, and fast-forward as needed, and
Mapa de Escenas, Ill
131110
,NicTp
Tie
rGramatica
Arturo en Mexico
Cuba y EEULI
LA EXCAVACION
Esta escena nos relate lo que Arturo este haciendo en el Distrito
Federal de Mexico mientras espera una Ilamada de Raquel. Cliquea
el icono en la caia
para continuar..
Figure 1. Advance Organizer
Texas Papers in Foreign Language Education
154
La excavación, pregunta 1
10110
control del video
Si quieres, utiliza el
para repasar la escena.
4En qué pensaba Angela?
o Pensaba en que si Roberto tenia hambre.
o Pensaba en que si Roberto tenia suficiente aire.
0 Pensaba en que si Roberto estaba enfermo.
Click here for feedback.
Move mouse over buttons for an explanation.
Figure 2. Multiple Choice Questions
Esperando: pregunta 3
el control de video para
pregunta sobre
liza
repasar la escena y contesta Ia
Raquel.
Cliquea el bot6n a la derecha para escuchar la pregunta.
Escoge todas las respuestas correctas.
pregunta
médica
abogada
0 veterinaria
o
actriz
0 ama de casa
0
profesora
Click here lor feedback.
Move mouse over buttons for an explanation.
Figure 3. Checklist Questions
6
Multiple Challenges of Multimedia
155
La civilización maya: pregunta 4
Para contestar la pregu.nta, repasa la escena usando el control del video.
Marca las respuestas como cierta o. false. Los mavas
cierta
false
0
gobemaban un vasto imperio del este al oeste.
O
0
vivian en la peninsula del Yucatan.
O
0
0
formaban una serie de estados autónomos.
0
llegaron a su maximo apogeo en el tiempo de Cristo.
O
hicieron grandes avances en el campo de la filosofia.
O
0
sabian todos los ciclos de los eclipses solares y lunares
O
Click here for feedback.
Move mouse over buttons for an explanation.
In
Figure 4. Binary Checklist Questions
Arturo en Mexico, pregunta 3
Mira la siguiente escena:
Arturo
visita...
4Ackinde va Arturo?
Cfie)
bar
banco
almacén
00
cine
gasolinera
superrnercado
Bc
farmacia
19
Click here for feedback.
Move mouse over buttons for an explanation.
Figure 5. Icon Sorting Questions
7
Texas Papers in Foreign Language Education
156
integrate smoothly into the
graduate teaching assistant (TA) de-
class-
work just as the other videos viewed
approxi-
signed
lessons
of
Libra
in the lab did. To reinforce the no-
mately 20 cards each to accompany
tion that these assignments were in-
eight Destinos episodes used in sec-
deed required, the syllabus
ond- and third-semester
listed
Spanish.
their due dates, and the course de-
The 25-minute episodes were seg-
mented into scenes, and text maps
scription stated that they counted as
led the learners to view each scene
10% of the final course grade.
in its entirety without stopping. Sub-
sequent cards asked questions on
Phase 1: Evaluation
of the
The evaluation
both main ideas and details in the
pilot
study was based on a time-series ex-
scene and provided learners the op-
portunity to review relevant por-
perimental design in which the stu-
dents alternated between the
tions of the scene as needed. Due to
ex-
time limitations, no glossary was
perimental Libra condition and the
provided, and scripts and character
control group. A modified recall pro-
lists were not available for all les-
tocol pretest served to establish pre-
existing differences among the stu-
sons.
in
At the second-semester level,
recall protocols
dents. Similar
English, collected in class the day
two instructors of four class sections
students were to have completed
with a total of 59 students partici-
viewing the assigned episode, served
pated in Phase 1. The third-semester
as the posttest. Due primarily to an
group involved three instructors of
six sections with 106 students. For
uncooperative
the
instructor,
sec-
ond-semester data were quite in-
each lesson, students had one week
to go to the Multimedia Language
complete and yielded no statistically
third-
Center individually or with a part-
results.
significant
The
ner to complete
the assignment.
semester data were reduced by stu-
dent absences, but complete data sets
They either completed the Libra les-
were available on 17 students in the
son linked with the Destinos vide-
experimental group and 20 in the
odisc in the experimental group, or
control group. An analysis of covari-
they viewed the video and filled in a
written worksheet in the control
ance revealed that on two of the four
group
experimental
group. The questions in both condi-
the
posttests,
tions were substantially the same,
significantly outperformed the con-
group. A third
though some of the written ques-
posttest
ap-
trol
tions had to be adapted from elec-
proached
significance
FlPSE,
(see
1994, for complete data and explana-
tronic to print medium. The instruc-
tors were requested not to discuss
tion). Thus, despite the limited data,
the pilot study provided evidence of
the episode until the day it was to be
the positive impact that the Libra-
completed
.
Because the episodes assigned
based viewing of the Destinos videos
for out-of-class work were an inte-
had on comprehension.
gral part of the Destinos program,
Student questionnaires admin-
istered at the end of the semester
the project team felt that they would
Multiple Challenges of Multimedia
157
complemented
the
quantitative
Phase 2: Development and Imple-
analysis. These questionnaires
mentation
re-
vealed a generally positive attitude
The second phase of the project
toward the multimedia lessons (see
focused on addressing the concerns
Table 1). The areas of concern identi-
raised in the pilot and on improving
fied by the student users included
participation. The Phase 1 authoring
the length of time required to com-
team and two additional TAs recon-
plete the lessons and the lack of un-
figured four of the lessons for sec-
derstanding of the program on the
ond-semester Spanish (see sample
part of some of the instructors, stu-
cards in Figures 2 to 5). With the
dents, and lab attendants. In its own
continued support of the FIPSE proj-
evaluation of the project, the devel-
ect, one of the TAs had received a
opment team posited two additional
grant to attend a Libra workshop,
and he
factors that may have contributed to
the
then
helped
teach
authoring system to the other TA
the lack of regular student participa-
team member as they rewrote the
insufficient
into
integration
tion:
classwork and the redundancy built
lessons.
into the Destinos program, which,
Two major changes were incor-
appropriately for a soap opera, al-
porated into the lesson revisions.
lows viewers to follow the story line
First, given the redundant nature of
the videos, the team reduced the
even if they miss an episode.
Table 1
Attitude Questionnaire Item from Phase 1 (n=80)
My attitude toward using computers to learn a foreign language:
Before This Semester
After This Semester
Hostile
2
1
Hesitant
14
13
Indifferent
30
22
Curious
31
21
Enthusiastic
21
5
9
158
Texas Papers in Foreign Language Education
amount of video to be viewed in
helped revise the Libra lessons par-
each lesson to three to four seg-
ticipated in Phase 2 of the project. As
ments: one or two that presented the
requested by the students in the pilot
key story line events, one that fo-
study and recommended by Masters-
cused on culture, and one that tar-
Wicks et al. (1996), an orientation to
geted functional language use. Sec-
the program was held in class before
the first lesson was assigned. Using
ond, several adjustments were made
to the lessons to allow the learner
transparencies of sample computer
greater flexibility in interacting with
screens, the instructors explained the
the text. A video control panel was
question types and the general navi-
added to every card so students
gation features. As before, the les-
sons represented 10% of the final
could fast-forward, rewind, and re-
play as needed at any time.
More
grade, and due dates were listed o n
help features, in the form of scripts,
the syllabus.
character lists, and a glossary, were
which
evaluation,
The
final
provided in three of the four les-
was qualitative in nature, was done
sons. A third change involved the
via a user questionnaire (see the ap-
addition of video-related grammar
pendix). The questionnaire included
exercises, but because this compo-
Likert-scale items to assess student
nent was not included in the pilot
reactions and open-ended questions
to gain more insight into the stu-
study, it will not be examined in this
dents' own perceptions of the les-
article.
sons. Instead of waiting until the
Significant changes were made
in the implementation of the re-
end of the semester to assess stu-
dents' views, the students completed
two
second-
vised lessons.
The
in the
semester Spanish instructors were
the Phase 2 questionnaire
Multimedia Language Center
closely involved
develop-
the
in
fol-
ment of the lessons, and, as a result
lowing completion of the last lesson.
of their investment in the process,
It was hoped that, by answering the
they understood Libra clearly and
questions immediately after the les-
son, learners would provide more
were able to explain it adequately to
their students. The instructor team
accurate information because their
made a conscious effort to integrate
experiences would be fresh in their
tightly with
viewings more
minds.
the
coursework. They agreed to hold de-
discussions of the
Phase 2: Evaluation Results and
targeted
tailed
scenes in class, to give brief dicta-
Discussion
tions and quizzes on them, and to
A total of 30 students out of the
incorporate them directly into exam
55 enrolled completed the question-
questions. Care was also taken to
naire. The instructors were surprised
train the lab assistants more thor-
at this low response rate because
oughly on the program.
more than 30 students reported do-
During the spring semester of
ing the Libra lessons. Some students
1997, the 55 students enrolled in the
may have done all the lessons except
the final one when the question-
second-semester
four
sections
of
Spanish taught by the two TAs who
naire was distributed. Perhaps the
o