DTIC ADA480695: Understanding Dynamic and Static Displays: Using Images to Reason Dynamically

0.41 MB·English

by  Defense Technical Information Center

CognitiveSystemsResearch6(2005)312–319
www.elsevier.com/locate/cogsys
Understanding dynamic and static displays: using images
to reason dynamically
Action editor: Wayne Gray
Sally Bogacz a,*, J. Gregory Trafton b,1
aClarifyConcepts,1330NewHampshireAvenue,NW,#915,WashingtonDC20036,USA
bNavalResearchLaboratory,WashingtonDC,USA
Received5September2004;accepted30November2004
Availableonline8January2005
Abstract
Weexaminedexpertmeteorologistsastheycreatedaweatherforecastwhileworkinginanaturalisticenvironment.
Weexaminedthetypeofexternalrepresentationtheychosetoexamine(astaticimage,asequenceofstaticimages,ora
dynamicdisplay)andthekindofinformationtheyextractedfromthoserepresentations(staticordynamic).Wefound
thateventhoughweatherisanextremelydynamicdomain,expertmeteorologistsexaminedveryfewanimations,exam-
ining primarily static images. However, meteorologists did extract large amounts of dynamic information from these
staticimages,suggestingthattheyreasonedabouttheweatherbymentallyanimatingthestaticimagesratherthanlet-
tingthe softwaredo itfor them.
!2004Elsevier B.V.Allrights reserved.
Keywords:Representation;Images;Static;Dynamic;Animations;Meteorology;Weatherforecasting;Reasoning
1. Introduction (an animation). This question is particularly rele-
vant for domains that have a strong spatial com-
Whatkindofexternaldisplaysdoexpertsuseto ponent to them like scientific reasoning (Schunn
extract dynamic information? External displays & Anderson, 1999; Trafton, Trickett, & Mintz, in
are usually either static (a diagram), or dynamic press;Trickett&Trafton,underreview),meteorol-
ogy (Lowe, 1994, 1999; Trafton et al., 2000), and
medical diagnosis (Lesgold et al., 1988).
* Correspondingauthor.Tel.:[phone].
Research examining the use of static and dy-
E-mail addresses: [email protected] (S. Bogacz),
namic displays shows mixed results across many
[email protected](J.G.Trafton).
1 Tel.:[phone]. different domains. The problem with static images
1389-0417/$-seefrontmatter !2004ElsevierB.V.Allrightsreserved.
doi:10.1016/j.cogsys.2004.11.007
Report Documentation Page Form Approved
OMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington
VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it

does not display a currently valid OMB control number.
1. REPORT DATE 3. DATES COVERED
SEP 2004 2. REPORT TYPE 00-00-2004 to 00-00-2004
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
Understanding dynamic and static displays: using images to reason
5b. GRANT NUMBER
dynamically
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
Naval Research Laboratory,Navy Center for Applied Research in REPORT NUMBER
Artificial Intelligence (NCARAI),4555 Overlook Ave.,
SW,Washington,DC,20375
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT
NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT
Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES
14. ABSTRACT
We examined expert meteorologists as they created a weather forecast while working in a naturalistic
environment. We examined the type of external representation they chose to examine (a static image, a
sequence of static images, or a dynamic display) and the kind of information they extracted from those
representations (static or dynamic). We found that even though weather is an extremely dynamic domain,
expert meteorologists examined very few animations, examining primarily static images. However,
meteorologists did extract large amounts of dynamic information from these static images, suggesting that
they reasoned about the weather by mentally animating the static images rather than letting the software
do it for them.
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF
ABSTRACT OF PAGES RESPONSIBLE PERSON
a. REPORT b. ABSTRACT c. THIS PAGE Same as 8
unclassified unclassified unclassified Report (SAR)
Standard Form 298 (Rev. 8-98)
Prescribed by ANSI Std Z39-18
S.Bogacz,J.G.Trafton/CognitiveSystemsResearch6(2005)312–319 313
is that they can impose a high working memory mation about change, just as graphics are a natu-
load when the task is to reason about a machine ral way for conveying information about space
in motion (Hegarty & Sims, 1994; Narayanan, (Tversky et al., 2002).
Suwa, & Motoda, 1994; Sims & Hegarty, 1997). One domain that seems tailor made for the use
However, many studies have found that anima- of animations is meteorological forecasting. The
tions by themselves do not improve performance forecaster has to determine the dynamics of past
(Byrne, Catrambone, & Stasko, 1999; Mayer & andcurrentweatherandpredictwhat(ifanything)
Anderson, 1991; Palmiter & Elkerton, 1993; Pal- willchangeinthefuture.Soanimationsshouldbe
miter, Elkerton, & Baggett, 1991; Rieber, Boyce, usefultoforecastersbecausethedomaintheywork
& Assad, 1990) unless they provide more infor- in forces them to explicitly consider the relation
mation than static images (Pane, Corbett, & between directional movements over time and
John, 1996; Tversky, Morrison, & Betrancourt, space (Rieber, 1990; Rieber & Kini, 1991), as well
2002). as real-time changes (Tversky et al., 2002).
The finding that animations by themselves do Previous studies that have examined the fore-
notimproveperformancehasledmanyresearchers casting process have shown that meteorologists
to question their usefulness (Palmiter & Elkerton, use a wide range of information to do their job:
1993;Paneetal.,1996),suggestingthatanimations static images, observations of the data, satellite
should be used only in very limited situations, i.e. pictures, computational weather models, display
only when necessary and when the animation is loops, textual information and dopplar radar
not too difficult to use (Betrancourt & Tversky, (Hoffman,1991;Traftonetal.,2000).Mostweath-
2000). But most of these studies have been per- er web sites present information in the form of
formedinlaboratorysettings(e.g.,Kaiser,Proffitt, both static and animated displays.
Whelan, & Hecht, 1992; Palmiter et al., 1991) or In the study discussed below, participants were
usetasksspeciallycraftedtoshowanimatedorsta- skilledNavalmeteorologists.Oneoftheirprimary
tic pictures (e.g., Pane et al., 1996; Rieber, 1991). sourcesforweatherdisplaysandotherinformation
Therehavebeenveryfewstudiesthathavelooked was the Fleet Numerical Meteorology and Ocean-
athow(orwhy)domainexpertsuseanimationsin ography Center (FNMOC) website (www.fnmoc.
complex, dynamic domains. What happens, for navy.mil). This website had a portal with links to
example, when experts in a dynamic domain have different displays, as shown in Fig. 1.
a choice of whether to use static or animated Using this portal, the forecaster could view a
images? What type of images do they look at, static image by clicking on the button marked
andwhattypeofinformationdotheyextractfrom ‘‘000’’ in the row marked ‘‘TAU’’. He would see
theseimages?Howdotheyusethisinformationto something similar to Fig. 2, which shows relative
help them solve problems? vorticity (or atmospheric swirl) for the present
Most researchers in the technology field believe time.
that animations are an important tool that can Fig. 2 shows a NOGAPS weather model out-
help us to understand complex domains. For this put for 4/29/2002. s is TAU which refers to the
reason, animations have been used in recent years time in the future for which this weather model
to teach procedures in HCI (Palmiter & Elkerton, will be valid. The zero in this case means that it
1993), to teach computer science algorithms (By- is a ‘‘prediction’’ of the current weather. The dis-
rne et al., 1999) to teach how something works play shows pressure at 500 mb and the amount of
(Mayer & Anderson, 1991; Pane et al., 1996), relative vorticity or wind spin. The original is in
and to understand other complex dynamic sys- color.
tems, like the weather (Lowe, 1999). Indeed, the Alternatively, he could view a series of images
prevalent feeling in this body of literature is that showing vorticity at different times (present time,
animations should be better than static images be- 12hintothefuture,etc.),byclickingonthebutton
cause, by a principle of congruence, animations marked‘‘all’’totheleftunder‘‘TAU’’,andwould
should be a natural medium for conveying infor- see something similar to Fig. 3.
314 S.Bogacz,J.G.Trafton/CognitiveSystemsResearch6(2005)312–319
Fig.1. AnexampleofaFNMOCportalwithchoicesofdifferenttypesofdisplay.
Fig.2. Anexampleofapicture:FNMOCNOGAPS2002042900t=0hforecastof500hPaHeights(m)andRel.Vort(10!5s!1).
Forecasters could scroll through these figures
aftertheyappearedinabrowserorotherwindow.
Sequenceswerealwaysaseriesofstaticpicturesof
theweatherthatstartedataspecifictimeandpro-
jected forward in equal time increments (typically
6 or 12 h) into the future.
Finally, forecasters could view an animation of
vorticity changing over time by clicking on the
buttonmarked‘‘Loop’’,wherehewouldseesome-
thing similar to Fig. 4.
So in their normal work environment, fore-
casters had a choice about what type of display
to look at. In this study, we examine what
happens when forecasters can choose what to
look at, by focusing on expert meteorologists
Fig. 3. An example of a sequence of images: FNMOC
as they create a weather report in naturalistic
NOGAPS 2002042900 500 hPa Heights (m) and Rel. Vort
(10!5s!1)mapsforalltimes. surroundings.
S.Bogacz,J.G.Trafton/CognitiveSystemsResearch6(2005)312–319 315
2.2. Apparatus and setting
The experimental sessions took place in a large
room.Ononesideoftheroom,twoWindows-NT
workstations were arranged side by side for the
forecaster and technician. Video recorders were
positioned to capture the forecasters! and techni-
cians! computer screens, with a third one posi-
tioned to capture the interactions between the
two forecasters.
Two subject-matter experts (SME) stood be-
hind the forecasters and took notes about what
the forecasters were doing. On the opposite side
of the room were three workstations that were
Fig. 4. An example of an animation: FNMOC NOGAPS
2002042900500hPaHeights(m)andRel.Vort(10!5s!1)maps used as the regional center under the supervision
forJavaMovie.Thegreen(grayinprint)arrowindicatesthat of a senior METOC officer. All communications
thissequenceofimageswasputintoarepeatedlycyclingloop. between the simulated shipboard and the simu-
lated Regional Center were carried out via a chat
tool called IRC Chat that all participants were
2. Method
familiar with.
Theexperimentexaminedhowexpertmeteorol-
2.3. Forecasting tools
ogists made forecasts in a complex work environ-
ment that simulated the situation on board ship.
The forecasters frequently used two programs
tovisualizetheweatherdata:JointMETOCViewer
2.1. Participants
(JMV) and a World Wide Web browser. JMV
allowsweathermodeldatatobevisualizedinvari-
All forecasters were Naval or Marine
ous ways: as a still, single image, as a sequence of
Corps forecasters and forecasters-in-training. The
images showing the same meteorological informa-
participants were representative of the range of
tion separated by time (e.g., 0,6,12,18,24 h into
expertise and training typically found for Naval
thefuture)andasananimationwhichconsistedof
METOC (METeorological and OCeanographic)
forecasters. 2 thesequenceofimagesputintoaloopthatcycled,
repetitively,asshowninFigs.2–4.TheWorldWide
Two individuals participated in each session,
Web browser allows forecasters to view satellite
playing the role of local forecasters on the ship.
imagesaswellasMETAR(aviationroutineweath-
They worked in pairs, with the more experienced
er report) and terminal aerodrome forecast (TAF)
individual acting as lead forecaster and the less
text from various websites. Satellite images could
experiencedastechnicianorforecaster-in-training.
alsobestaticordynamic.
Each forecaster provided talk-aloud protocols
It should be noted that information that is dis-
(Ericsson & Simon, 1993). The data for this study
played by all major meteorological web sites (e.g.,
consistsoftheutterancesmadebytheexpertfore-
FNMOC, NOAA) shows identical information in
caster of each of these pairs (N=7), who had an
different ways: static pictures, sequences, or ani-
averageof11yearsofforecastingexperience.Each
matedsequencesofpictures.Soallanimationsthat
forecasterhadpreparedover100weatherforecast-
a forecaster could look at had the same informa-
ing briefs the previous year.
tion as their static or sequence counterparts. What
most systems do in order to animate meteorologi-
2 All forecasters were men, so male pronouns will be used cal information is to generate the individual con-
throughoutthispapertorefertothem. secutive displays and animate them by presenting
316 S.Bogacz,J.G.Trafton/CognitiveSystemsResearch6(2005)312–319
them quickly, as though the user were flipping Table1
throughaflip-chart.Thismethodisusedtodisplay Samplecodingschemeofexternalrepresentations
almost all time-sequence meteorological products, Category Example
including global weather, local weather and satel- Picture COAMPS/SanDiego/18Z(6p.m.Zulutimeis
lite imagery. 6p.m.GreenwichmeantimeorGMT)
Sequence NOGAPS/WhidbeyIs./0Z(midnightGMT),6Z
(6a.m.GMT),12Z,18Z,24Z
2.4. Procedure
Animation Intellicast/Satelliteloop
Text PortAngelesMETARtextpage
The task was to prepare a written brief for an
airplaneflownfromanaircraftcarriertoadestina-
tion 12 h in the future (the destination was Whid- displays. Each time the forecaster examined a dis-
bey Island, Washington State). The brief was to play, it was coded (see Table 1).
covertheentireroundtripandtheforecasterswere Theexternalrepresentationswerecategorizedas
asked to provide specific weather information for a picture (a still weather map or satellite image), a
departure, en-route, destination and alternate sequence (a sequence of weather maps or satellite
airfields. In order to do this, forecasters had to images separated by equal time intervals), an ani-
determine detailed qualitative and quantitative mation (the sequence of weather maps or satellite
information about the weather conditions. This imagesputintoacontinuouslycyclingloop)ortext
taskwasaveryfamiliaronefortheforecasters. (METAR and TAF text reports) (see Figs. 2–4).
Each session began with a description of the
task by an experienced Navy forecaster, giving 2.5.2. Utterance coding
destination, times and other information. Fore- Utteranceswerecategorizedforweather-related
casters created briefs using PowerPoint or wrote extractions. Remarks made by the forecaster
the information down on paper. The forecaster about goals, military matters or astronomical
served as leader during the session, requesting observations were excluded unless they contained
information from the technician as needed. information about the weather.In addition, utter-
Forecasters were given 2 h to complete their ancesthatwerenotrelatedtoinformationgleaned
forecast and brief. Everyone completed the task from displays shown on the forecaster!s screen,
within the time allotted. If time allowed, the fore- such as questions that tutored the technician
caster presented hisbrief attheend ofthesession. (‘‘Did you interpret it correctly off the TAF?’’) or
All sessions were concluded with a debriefing dur- extraneous information (e.g., how to find a NO-
ing which the experimenter had an opportunity to GAPS product on the Internet) were also
askquestionsandtheparticipantshadanopportu- excluded. Each of the remaining utterances was
nity to give feedback to the experimenters. coded for its dynamic content.
2.5. Coding scheme 2.5.3. Dynamic and static utterances
Utteranceswerecodedfortheirdynamicorsta-
Theforecasters!utterancesweretranscribedand ticcontent.Staticutterancesdescribedtheweather
coded, and the external representations they used at one location at a specific time. Dynamic utter-
werenotedbyexaminingthesessionsonvideotape ancescapturedchangeacrosstimeorspace.These
with an audio soundtrack. included utterances where change was implied
rather than specifically stated: For example, ‘‘It!s
2.5.1. Coding of external representations going to be valid after 13:45Z’’ or ‘‘So everything
Forecasters used many data sources. Satellite from like 12Z on put a broken on at about 500
images and JMV displays were very commonly feet’’.Bothofthesestatementsaredescribingcon-
used,butforecastersalsoexaminedSkew-Ts(tem- ditions for a particular period of time, implying
perature profiles at a particular location), ME- that outside that time-frame, conditions change.
TAR and TAF text as well as several other Examples can be found in Table 2.
S.Bogacz,J.G.Trafton/CognitiveSystemsResearch6(2005)312–319 317
Table2
Samplecodingschemeofstaticanddynamicweatherutterances
Category Exampleutterances
Static Whidbeyhasaceilingof4500broken
Mostlysunnyafternoon,70,75Seattle
Dynamic 24hfromnow,definitelyalotofprecipitation
movingintothearea
There!safrontalsystemapproachingthearea
producingprecipitation
To ensure reliability of coding, one author
coded the entire dataset and the other author
Fig.5. Thepercentageofdynamicutterancesandpercentageof
coded 25% of it. The two coders agreed 87% of
differenttypesofdisplayexamined.
thetime:j=.76,p<.001.Disagreementswerere-
solved by discussion. Whatispuzzlingaboutthisresultisthatanima-
tions were just as available for use as sequences,
pictures or text. Does this lack of animation use
3. Results and discussion mean that meteorologists were not thinking
dynamically when forecasting? We examined this
This section focuses on forecasters! use of ani- questionbylookingatthetypesofutterances(sta-
mations and the implications for reasoning in a ticordynamic)thatforecastersmadewhilelooking
complexdynamicdomain.Givenachoice,dofore- at different kinds of display.
casters use animations? Do dynamic images help
forecasters reason about the weather? 3.3. Utterances
3.1. Overview We found that forecasters were thinking
dynamically about the weather for a substantial
The forecasters looked at an average of 21.4 proportion of the time: 35% of the forecasters!
externalrepresentationseach.Theyextractedinfor- utterances were dynamic. Thus forecasters made
mationfromtheseexternalrepresentationsanaver- little use of animated images, but talked dynami-
ageof5.3timesperrepresentation.Thesenumbers cally about the weather for over a third of the
are probably an underestimate of the amount of time. It could be that most of the dynamic utter-
informationextractedbecauseutteranceswerenot ances were made while examining the (relatively
codedwhentheforecasterwastutoringthetechni- rare) animated images or the (slightly more com-
cian,whichhappenedfairlyfrequently. mon) sequences of images. However, Fig. 5 dem-
onstrates (see dark bars) that forecasters made
3.2. External representations dynamic utterances constantly, with the rate of
production of dynamic utterances being approxi-
Forecastershadachoiceofwhatimagetolook mately proportional to display use. So because
at. There were four categories: picture, sequence, forecastersdidnotuseanimations,theyalsodirec-
animationandtext.Fig.5showshowoften(inper- ted very few dynamic utterances towards them.
centages) the forecasters looked at each type of
image (see light bars).
Notice that forecasters made almost no use of 4. General discussion
animations. Instead, forecasters chose to look at
pictures(whicharestaticimages)mostofthetime, When expert meteorological forecasters made
v2(3)=70.32,p<.001,Bonferroniadjustedv2sig- weather predictions, they examined static rather
nificant at p<.001. than animated displays of data. Forecasters
318 S.Bogacz,J.G.Trafton/CognitiveSystemsResearch6(2005)312–319
preferred to use static displays even though there images. If they mentally animated the static dis-
were many dynamic displays available and they plays, this would have given them control over
knew how to use them. They used the static dis- the mental images they created, enabling them to
plays even though they were thinking dynamically ‘‘set in motion’’ any relevant details. It is clear,
about the weather. And they produced dynamic however, that not only is it possible for experts to
utterances at the same rate as they looked at each extract dynamic information from static displays,
type of display. but that it is their preferredmethod of doing so.
These results are not surprising if you read the
largebodyofliteratureshowingthat,inlaboratory
settings,there isverylittleadvantage tousing ani- Acknowledgements
mations when they contain the same amount of
information as static displays (Byrne et al., 1999; This work was supported in part by Grants
Rieber et al., 1990). But technology designers, N00014-00-WX-20844 and N00014-00-WX-40002
whobelievethatanimationsareanimportanttool to Greg Trafton from the Office of Naval Re-
inhelpingustounderstandcomplexdomains,will search. We thank the forecasters who participated
besurprised.Animationsshouldbebetterthansta- in this study. We also thank Susan Trickett and
tic images at describing the weather, because ani- Nick Cassimatis for comments on an earlier draft
mations show change over time (e.g., the and four anonymous reviews for comments on a
‘‘Principle of Congruence’’; Tversky et al., 2002). later draft.
Isn!t that what forecasters want to know?
Ourresearchindicates,however,thatadynamic
animationisnotthepreferredmethodofretrieving
References
dynamic information about the weather. This
agrees with other research showing that people
Betrancourt, M., & Tversky, B. (2000). Effect of computer
do not perform better or learn more when using
animationonusers!performance:areview.TravailHumain,
adynamicdisplay.Explanationsforthislackofef-
63(4),311–329.
fectincludetheideathatanimationsimposeahigh Byrne,M.D.,Catrambone,R.,&Stasko,J.(1999).Evaluating
workload(Lowe,2000),alackofknowledgeabout animationsasstudentaidsinlearningcomputeralgorithms.
howweintegrateinternalandexternal representa- ComputersandEducation,33(4),253–278.
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis:
tions (Scaife & Rogers, 1996), comprehension dif-
Verbalreportsasdata.CambridgeMA:MITPress.
ficulty (Tversky et al., 2002), difficulty in
Hegarty, M., & Sims, V. K. (1994). Individual differences in
perceiving (Kaiser et al., 1992) and difficulty in mental animation during mechanical reasoning. Memory
use (Betrancourt & Tversky, 2000). andCognition,22(4),411–430.
How does an expert forecaster extract that dy- Hoffman, R. R. (1991). Human factors psychology in the
support of forecasting: the design of advanced meteoro-
namicinformationfromstaticdisplays?Consistent
logical workstations. Weather and Forecasting, 6(1),
withotherresearchonstaticdisplays(e.g.,Hegarty
98–110.
& Sims, 1994), we believe that forecasters animate Kaiser, M. K., Proffitt, D. R., Whelan, S. M., & Hecht, H.
static displays (or parts of the displays) mentally, (1992). Influence of animation on dynamical judgments.
andusethatinformationtocreateadynamicmen- JournalofExperimentalPsychology:HumanPerceptionand
Performance,18,669–690.
tal representation of atmospheric dynamics, e.g.,
Lesgold,A.,Rubinson,H.,Feltovich,P.,Glaser,R.,Klopfer,
how a front would move. Or they blend different
D., & Wang, Y. (1988). Expertise in a complex skill:
weather models to create a dynamic mental repre- DiagnosingX-raypictures.InM.Chi,R.Glaser,&M.Farr
sentation of the most probable path for a weather (Eds.),Thenatureofexpertise(pp.311–342).Hillsdale,NJ:
system.Ourbestguessastowhyourexpertpartic- Erlbaum.
Lowe,R.K.(1994).Selectivityindiagrams:readingbeyondthe
ipants did not use animations is that these meteo-
lines.EducationalPsychology,14,467–491.
rologists had enough expertise to be able to
Lowe,R.K.(1999).Extractinginformationfromananimation
memorize critical global features, but needed to during complex visual learning. European Journal of Psy-
see the details which are visible only in static chologyandEducation,XIV(2),225–244.
S.Bogacz,J.G.Trafton/CognitiveSystemsResearch6(2005)312–319 319
Lowe, R. K. (2000). Animation of diagrams: An aid to Rieber,L.P.,&Kini,A.S.(1991).Theoreticalfoundationsof
learning? In M. Anderson, P. Cheng, & V. Haarslev. instructional applications of computer-generated animated
Theoryandapplicationofdiagrams.Heidelberg:Springer. visuals.JournalofComputer-BasedInstruction,18,83–88.
Mayer, R. E., & Anderson, R. B. (1991). Animations need Scaife, M., & Rogers, Y. (1996). External cognition: how do
narrations:An experimental test of a dualcoding hypoth- graphical representations work. International Journal of
esis.JournalofEducationalPsychology,83(4),484–490. Human–ComputerStudies,45(2),185–213.
Narayanan,N.H.,Suwa,M.,&Motoda,H.(1994).Astudyof Schunn, C. D., & Anderson, J. R. (1999). The generality/
diagrammatic reasoning from verbal and gestural data. In specificity of expertise in scientific reasoning. Cognitive
16thAnnualConferenceoftheCognitiveScienceSociety. Science,23(3),337–370.
Palmiter, S., & Elkerton, J. (1993). Animated demonstrations Sims, V. K., & Hegarty, M. (1997). Mental animation in the
for learning procedural computer-based instruction. visuospatial sketchpad: evidence from dual-task studies.
Human–ComputerInteraction,8,193–216. MemoryandCognition,25(3),321–332.
Palmiter, S., Elkerton, J., & Baggett, P. (1991). Animated Trafton,J.G.,Kirschenbaum,S.S.,Tsui,T.,Miyamoto,R.T.,
demonstrations versus written instructions for learning Ballas, J. A., & Raymond, P. D. (2000). Turning pictures
procedural tasks: a preliminary investigation. International into numbers: extracting and generating information from
JournalofMan–MachineStudies,34,687–701. complex visualizations. International Journal of Human–
Pane, J. F., Corbett, A. T., & John, B. E. (1996). Assessing ComputerStudies,53,827–850.
dynamics in computer-based instruction. In CHI !96, Trafton, J. G., Trickett, S. B., & Mintz, F. (in press).
Vancouver,BC,Canada. Connecting internal and external representations: spatial
Rieber,L.P.(1990).Animationincomputer-basedinstruction. transformations of scientific visualizations. Foundations of
EducationalTechnology,ResearchandDevelopment,38,77–86. Science.
Rieber, L. P. (1991). Animation, incidental learning and Trickett,S.B.,&Trafton,J.G.(underreview).Moviesinthe
continuing motivation. Journal of Educational Psychology, mind:theinstantiationanduseofconceptualsimulationsin
83,318–328. scientificreasoning.
Rieber,L.P.,Boyce,M.J.,&Assad,C.(1990).Theeffectsof Tversky, B., Morrison, J. B., & Betrancourt, M. (2002).
computer animation on adult learning and retrieval tasks. Animation: can it facilitate? International Journal of
JournalofComputer-BasedInstruction,17(2),46–52. Human–ComputerStudies,57(4),247–262.