Table Of Content

A New Model of Graph and Visualization Usage J.GregoryTrafton NavalResearchLaboratory [email protected] SusanB.Trickett GeorgeMasonUniversity [email protected] Abstract shows that people switch between looking at the graph andtheaxesinordertocomprehendthevisualization. Weproposethatcurrentmodelsofgraphcomprehension Thisschemeseemstoworkverywellwhenthegraph do not adequately capture how people use graphs and containsalltheinformationtheuserneeds(i.e.,whenthe complexvisualizations.Toinvestigatethishypothesis,we information is explicitly represented in one form or an- examined3sessionsofscientistsusinganinvivomethod- other). Thus,whenanundergraduateisaskedtoextract ology. Wefoundthatinordertoobtaininformationfrom their graphs, scientists not only read off information di- specificinformationfromabar-graph,theaboveprocess rectly from their visualizations (as current theories pre- seems to hold. However, graph usage outside the labo- dict), but they also used a great deal of mental imagery ratoryisprobablynotsimplyaseriesofinformationex- (which we call spatial transformations). We propose an tractions. For example, when looking at a stock market extension to the current model of visualization compre- graph,thegoalmaynotbejusttodeterminethecurrent hensionandusagetoaccountforthisdata. orpastpriceofthestock,butperhapstodeterminewhat thepriceofthestockwillbesometimeinthefuture. A Introduction weatherforecasterlookingatameteorologicalvisualiza- tionisfrequentlytryingtopredictwhattheweatherwill If a person looks at a standard stock market graph or beinthefuture,aswellaswhatthecurrentvisualization a meteorologist is examining a complex meteorological shows(Trafton,Kirschenbaum,Tsui,Miyamoto,Ballas, visualization, how is information extracted from these & Raymond, 2000). A scientist examining results from graphs? Themostinfluentialresearchongraphandvisu- arecentexperimentcannotalwaysdisplaytheavailable alizationcomprehensionisBertin’s(1983)taskanalysis informationinawaythatperfectlyshowstheanswerto that suggests three main processes in graph and visual- herhypotheses. izationcomprehension: Howdocurrenttheoriesofgraphcomprehensionhold 1. Encode visual elements of the display: For exam- up when a graph or visualization does not contain the ple, identify lines and axes. This stage is influenced by exact information needed? Unfortunately, the theories pre-attentive processes and is affected by the discrim- donotsayanythingaboutthissituation. Infact,thereare inabilityofshapes. no specifications in any theory of graph comprehension 2. Translate the elements into patterns: For example, abouthowinformationcouldorwouldbeextractedfrom notice that one bar is taller than another or the slope of avisualizationwherethatinformationisnotrepresented aline. Thisstageisaffectedbydistortionsofperception insomeform.Ifagraphdoesnotcontaintheinformation andlimitationsofworkingmemory. needed by the user, the graph is often labeled “bad” or 3. Map the patterns to the labels to interpret the spe- “useless”(Kosslyn,1989;Pinker,1990). cific relationships communicated by the graph. For ex- Current graph comprehension theories do not have a ample,determinethevalueofabargraph. great deal to say about what to do when a graph does Most of the work done on graph comprehension has notexplicitlyshowtheneededinformationforavariety examined the encoding, perception, and representation ofreasons. Themainreasonisprobablythatmostgraph of graphs. Cleveland and McGill, for example, have comprehensionstudieshaveusedfairlysimplegraphsfor examined the psychophysical aspects of graphical per- whichnoparticulardomainknowledgeisrequired(e.g., ception (Cleveland & McGill, 1984, 1986). Similarly, Carter, 1947; Lohse, 1993; Pinker, 1990). However, in Pinker’s theory of graph comprehension, while quite real-worldsituations,peopleusecomplexvisualizations broad, focuses on the encoding and understanding of that require a great deal of domain knowledge, and all graphs (Pinker, 1990). Kosslyn’s work emphasizes the the needed information would probably not be explic- cognitiveprocessesthatmakeagraphmoreorlessdiffi- itlyrepresentedinthegraph. Thisstudywillthustryto cult toread. Kosslyn’s syntactic and semantic (and to a answer two questions about graph comprehension. Do lesserdegreepragmatic)levelofanalysisfocusesonen- expertusersofvisualizationseverneedinformationthat coding, perception, and representation of graphs (Koss- is not on a specific graph they are using? If so, how do lyn, 1989). Recent work by Carpenter and Shah (1998) theyextractthatinformationfromthegraph? Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 3. DATES COVERED 2001 2. REPORT TYPE 00-00-2001 to 00-00-2001 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER A New Model of Graph and Visualization Usage 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION Naval Research Laboratory,Navy Center for Applied Research in REPORT NUMBER Artificial Intelligence (NCARAI),4555 Overlook Avenue SW,Washington,DC,20375 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES The proceedings of the twenty-third annual conference of the cognitive science society, 1-4 Aug 2001, Edinburgh, Scotland 14. ABSTRACT We propose that current models of graph comprehension do not adequately capture how people use graphs and complex visualizations. To investigate this hypothesis, we examined 3 sessions of scientists using an in vivo methodology. We found that in order to obtain information from their graphs, scientists not only read off information directly from their visualizations (as current theories predict), but they also used a great deal of mental imagery (which we call spatial transformations). We propose an extension to the current model of visualization comprehension and usage to account for this data. 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF ABSTRACT OF PAGES RESPONSIBLE PERSON a. REPORT b. ABSTRACT c. THIS PAGE Same as 6 unclassified unclassified unclassified Report (SAR) Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 There are several possible things that users could do Method whentryingtoextractinformationfromagraph. Inthe In order to investigate the issues discussed above, we simplestcase,theinformationisexplicitlyavailable,and have adapted Dunbar’s in vivo methodology (Dunbar, they can simply read off the information from the visu- 1995, 1996; Trickett, Trafton, & Schunn, 2000b). This alization. Whatdotheydowheninformationtheyneed approach offers several advantages. First, it allows the is not available on the visualization? They could create observationofexperts,whoarethusabletousetheirdo- acompletelynewvisualizationthatdoesshowtheinfor- main knowledge to guide their strategy selection. Sec- mation. Theycouldalsocollectmoredataorconsultan- ond, it allows the collection of ”on-line” measures of othersource. Theycouldcreateanexplicitplantolook thinking,whichallowtheinvestigationofthescientists’ formoredataorrunanotherexperiment. reasoning as it occurs (Ericsson & Simon, 1993). Fi- Whatdotheydowhenthevisualizationisalltheyhave nally, the tasks (experiment design, data analysis, etc.) to work with? What kind of mental operations could conductedbythescientists,aswellasthetoolstheyuse, users perform on graphs and visualizations in order to arefullyauthentic. extract information that is not explicit? One possibility Twosetsofscientistswerevideotapedwhileconduct- isthatpeopleusesomesortofvisualimagerytoextract ing their own research. All the scientists were experts, informationthatisnotexplicitlyrepresentedonagraph havingearnedtheirPh.D.smorethan6yearspreviously. orvisualization. Forexample,aweatherforecastermay In the first set, two astronomers, one a tenured profes- mentallyimagineafrontmovingeastoverthenextsev- sor at a university, the other a fellow at a research in- eral days (Trafton et al., 2000), or a stock analyst may stitute, worked collaboratively to investigate computer- mentallyextendalineonagraphandthinkthatastock generatedvisualrepresentationsofanewsetofobserva- will continue to rise. We have developed a framework tionaldata. Atthetimeofthisstudy,oneastronomerhad for coding and working with these kinds of graphs and approximately 20 publications in this general area, and visualizationscalledSpatialTransformationsthatwillbe theotherapproximately10. Theastronomershavebeen usedtoinvestigatetheseissues. Wewillarguethatspa- collaborating for some years, although they do not fre- tialtransformationsareafundamentalaspectofcomplex quentlyworkatthesamecomputerscreenandthesame visualizationusage. timetoexaminedata. In the second dataset, a physicist with expertise in Spatial Transformations are cognitive operations that computational fluid dynamics worked alone to inspect a scientist performs on a visualization. Sample spatial the results of a computational model he had built and transformations are mental rotation (e.g., Shepard & run. Two related sessions were recorded with this sci- Metzler, 1971), creating a mental image, modifying entist over consecutive days. He works as a research that mental image by adding or deleting features to or scientistsatamajorU.S.scientificresearchfacility,and fromit,animatinganaspectofavisualization(Hegarty, had earned his Ph.D. over 20 years previously. He had 1992) time series progression prediction, mentally inspectedthedatapreviouslybuthadmadesomeadjust- moving an object, mentally transforming a 2D view ments to the physics parameters underlying the model into a 3D view (or vice versa), comparisons between andwasthereforerevisitingthedata. different views (Kosslyn, Sukel, & Bly, 1999; Trafton, Bothsetsofscientistswereinstructedtocarryouttheir Trickett, & Mintz, 2001), and anything else a scientist workasthoughnocamerawerepresentandwithoutex- mentally does to a visualization in order to understand planationtotheexperimenter(Ericsson&Simon,1993). it or facilitate problem solving. Also note that a spatial The relevant part of the astronomy session lasted about transformation can be done on either an internal (i.e., 53minutes,andthetwophysicssessionseachlastedap- mental) image or an external image (i.e., a scientific proximately 15 minutes. All utterances were later tran- visualization on a computer-generated image). What scribed and segmented according to complete thought. all spatial transformations have in common is that they Allsegmentswerecodedby2codersason-task(pertain- involve the use of mental imagery. A more complete ing to data analysis) or off-task (e.g., jokes, phone call description of spatial transformations can be found at interruptions, etc.). Inter-raterreliabilityforthiscoding http://iota.gmu.edu/users/trafton/405st.html. was more than 95%. Off-task segments were excluded We will examine the number of times that users from further analysis. On-task segments (N = 649 for needed information from a visualization. If all or most the astronomy dataset and N = 189 for the first physics oftheinformationisavailableexplicitlyonthevisualiza- datasetandN=176forthesecondphysicsdataset)were tion, we should see primarily read-offs (Kosslyn, 1989; furthercodedasdescribedbelow. Pinker,1990).If,however,aparticularvisualizationdoes TheTasksandtheData notexplicitlydisplayparticularinformationthatascien- tistwants,wewillexaminehowthescientistgoesabout Astronomy Theastronomicaldataunderanalysiswere obtaining that information. We expect that in complex opticalandradiodataofaringgalaxy. Theastronomers’ visualizations,thereisagreatdealofinformationthatis high-levelgoalwastounderstanditsevolutionandstruc- needed in addition to what is displayed, and we expect turebyunderstandingtheflowofgasinthegalaxy.Inor- scientists to use spatial transformations to retrieve that dertounderstandtheflowofgas, theastronomersmust information. makeinferencesaboutthevelocityfield,representedby Figure 2: An example of the kind of visualizations ex- aminedbythephysicist. Figure 1: An example of the kind of visualizations ex- understanding of the underlying theory. Although the aminedbytheastronomers. physicist had been in conversation with the experimen- talist, he had not viewed the empirical data, and in this sessionhewasinvestigatingonlytheresultsofhiscom- putationalmodel. However,hebelievedthemodeltobe contour lines on the 2-dimensional display. The as- correct (i.e., he had strong expectations about what he tronomers’ task was made difficult by two characteris- would see), and in this sense, this session may be con- tics of their data. First, the data were one- or at best 2- sideredconfirmatory. dimensional,whereasthestructuretheywereattempting Thedataconsistedoftwodifferentkindsofrepresen- to understand is 3-dimensional. Second, the data were tationofthedifferentmodes,shownovertime(nanosec- noisy,andtherewasnoeasywaytodistinguishbetween onds). The physicist was able to view either a Fourier noiseandrealphenomena.Figure1showsascreensnap- decomposition of the modes or a representation of the shotofthetypeofdatatheastronomerswereexamining. “raw”data. In order to make their inferences, the astronomers used differenttypesofimage,representingdifferentphenom- Figure2showsanexampleofthephysicist’sdata. He ena (e.g., different forms of gas), which represent dif- couldchoosefromblack-and-whiteoravarietyofcolor ferent information about the structure and dynamics of representations, and could adjust the scales of the dis- the galaxy. Some of these images could be overlaid on played image, as well as some other features. He was each other. In addition, the astronomers could choose abletoopennumerousviewssimultaneously. fromimagescreatedbydifferentprocessingalgorithms, each with advantages and disadvantages (e.g., more or CodingScheme lessresolution). Finally, theycouldadjustdifferentfea- Ourgoalsinthisresearcharefirst,todetermineifcom- tures of the display, such as contrast or false color. A plexvisualizationscontainalltheinformationneededby more complete description of this dataset can be found the scientists, and, if not, to investigate what happens in Trickett, Fu, Schunn, and Trafton (2000a) and Trick- whentheydonothavealltheinformationtheyneed. We ett,Trafton,andSchunn(2000b). propose that spatial transformations are a major portion Physics The physicist was working to evaluate how of extracting information from a visualization when the deep into a pellet a laser light will go before being re- dataisnotexplicitlyrepresented.Consequently,weiden- flected. Hishigh-levelgoalwastounderstandthefunda- tifiedeverysituationwhereascientistwantedtoextract mentalphysicsunderlyingthereaction,anunderstanding information from a visualization. Next, we coded what that hinged on an understanding of the relative impor- the scientist did to extract information, including read- tanceandgrowthratesofdifferentmodes. Thephysicist ing off the information directly from the graph, spatial hadbuiltamodelofthereaction;otherscientistshadin- transformations,changingthevisualization,plansordis- dependentlyconductedexperimentsinwhichlaserswere cussions about getting more data, and abandoning their firedatpelletsandthereactionsrecorded. Aclosematch attempttogettheinformation.Wenowdescribeandpro- betweenmodelandempiricaldatawouldindicateagood videexamplesofthiscodingschemeindetail. Example Explanation Afterall,itistentothe Scientistislookingataline minussix... andextractingthey-axisvalue Imean,thefactyouseesuchastrong Scientistisreadingoffthe concentrationofgasinthering,um... amountofgasinthering That’sabout220km/sec,whichisthe Scientistisreadingoff velocityspreadofanormalgalaxy. thevelocityspread Table1: Examplesofinformationthatisreadoffthevisualizations. SpatialTransformation Example Explanation CreateMentalImage Imean,inaperfect,inaperfect Scientistiscreatingamental world,inaperfectsortof imageofaspiderdiagram;there spiderdiagram... isnospiderdiagramdisplayed. ModifyImage Sothat[line]wouldbebelow Scientistisaddinganew theblackline (hypothesizied)linetoa currentvisualization ModifyImage Iftherewasnostreaming Scientisthasimagedaprevious motionorsortofpilingof mentalimageandisnowremoving gas thestreamingmotionsfromhismentalimage Comparison: Maybeit’saprojectioneffect, Scientistiscomparinga althoughifthat’strue,thereshould currentimagetoapreviously beaverylargevelocitydispersion. createdmentalimage. Table2: Examplesofspatialtransformations. Desire to extract information A scientist would fre- to a Fourier mode display). Alternatively, the scientists quentlywanttoextractsomeamountofinformationfrom could“tweak”thecurrentrepresentation(fromblackand avisualization. Commentsvariedfromtheverygeneral white to color, for example). We coded the visualiza- (“Whatdowesee?”)totheveryspecific(“Let’ssee,how tionchangeswherethescientistswerelookingforaddi- doesoh-threeversusthree-oh[look]?”). tional information. If they simply made a mistake and tweaked the visualization, we did not count that visual- Read-Off A scientist would be able to read-off infor- izationchange. Forexample,whilelookingataparticu- mation directly from the graph. Information that was larlycompressedvisualization,oneofthescientistssaid read off a visualization was explicitly on the graph and “Where’sthree-ohat? Don’tseethree[oh]. That’swhat thescientistsimplyhadtoread-offaparticularvalue.For Ifigured,Iwasgonnagetspaghetti. Let’sdoare-plot.” every utterance, we evaluated whether a value was read andthenreplottedthedatawithareduceddataset. offthevisualization. Table1showsseveralexamplesof informationthatwasreadoffofthevisualization. Planstogathermoredata Occasionally,thescientists SpatialTransformations Asdiscussedearlier,spatial wanted or needed to gather more data. We coded every transformations are cognitive operations that a scientist timetheymadeaplantogathermoredata. Forexample, performsonavisualization. Foreveryutteranceineach one scientist said “So that means that this guy is in fact protocolweevaluatedwhethertherewasaspatialtrans- between him and him, which is exactly what the exper- formation. Spatial transformations were further coded imentalist believes he saw. Now, somewhere along the as Create Image, Modify Image, or Comparison. Ta- lineIhavetogettheirresults.” ble 1 shows examples of each category of spatial transformation(notethattheseutterancesareindependentof one another and do not represent a sequence). Table 2 Abandoningtheirattempttogetinformation Some- shows several examples of spatial transformations that times the scientists either could not decide what data to wereusedbythescientists. getorsimplyabandonedtheirquestforaspecificinfor- mation. We coded every time the scientists abandoned ChangingtheVisualization Thescientistswereusing theirattempttogetinformation. Forexample,onescien- theirowntoolsandwereabletochangethevisualization tist,unabletoexplainaparticularfeatureafterextensive toacompletelydifferentrepresentation. Forexample,a investigationoftheimage,said“Yeahwell,[let’s]gloss scientistcouldchangethedatadisplayfromtherawdata overit.” Results GeneralDiscussion Ourtwogoalsinthispaperaretoexplorewhetherscien- tists are able to directly extract the amount of informa- E:Bertin, 1983; Kosslyn, 1989; Pinker, 1990 E:Carpenter & Shaw, 1998 tiontheyneedfromthevisualizationstheyexamineand ifnot,toexplorehowtheydogettheinformationthatis needed. EEnclDeomdiseep nVlatsiys oufal eleTpmraaetntnestrlsna itsneto Matop lpaabtetelsrns Howoftenisneededinformationdirectly available? Ofthe1014totalutterancesinthethreesessions,almost half(481)involvedsomeformofinformationgathering. ESp:Barerrotiwn,, 11998893; Lohse, 1993; Read-Off Yes AvaInilfaoble? AsFigure3shows,approximatelyhalfofthoseinfor- mationgatheringinstanceswereread-off,suggestingthat thescientistdidusethevisualizationagreatdealtoex- No tractinformation.However,thereweremanytimeswhen thescientistsneededinformationfromavisualizationbut Spatial Transformations itwasnotavailabledirectlyfromthevisualization.Thus, the visualizations seem to be good, but far from perfect fromaninformationgatheringpointofview. E:Current Paper Usage of Information E:Trafton et al., 2000 25 Figure4:Ourcurrenttheoreticalmodelofcomplexvisu- alizationusage. The“E:”showsevidenceforeachstage Utterances 20 ofthemodel. Task 15 On− We have conducted a detailed analysis of expert sci- Percentage of 150 etthhnaatittsttthhseeasytewhscaoivreekntciinsotlstlhedecoitreedoxwttrhnaecmltaabseoglrrvaeetaost.rdieeOsa,ularonfraeilnsyfuzolitrnsmgsahdtioaowtna fromthevisualizations.However,thesevisualizationsdo 0 not provide the scientists with all the information they Spatial Vis Read−Off Transformations Changes Plans Abandon needtoanswertheirquestions. Wefoundthatwhenthey Information Explicitly Available Information not Explicitly Available needed information that was not explicitly provided by on Visualization on Visualization thevisualization,theytendedtoperformspatialtransfor- mationstoanswertheirquestions. Figure 3: The number of read-offs, spatial transforma- It is interesting that the scientists did not simply tions,visualizationchanges,planstocollectfuturedata, change the visualization more frequently to get the anddecisionstoabandontheattempttogetmoredatafor needed information. There was some evidence in the alldatasets. protocols that it was not easy to create new visualizations. Forexample,someofthevisualizationshadtobe How was needed information extracted if it was not re-donebecauseofanerrorthatwasmadeinthedisplay simplyreadoff? AsFigure3shows,thevastmajorityof (i.e.,neededdatawasnotincludedintheplotortheplot informationthatwasnotreadoffwasgatheredbyusing was not presented logarithmically when it should have spatial transformations. In fact, there was no statistical been). However,thisproblemdidnotseemtohavepre- differencebetweenthenumberoftimesthatthescientists vented the scientists from trying to make the changes: readoffinformationdirectlyfromthegraphandthenum- there were no instances of a scientist saying the visual- berofspatialtransformations,c 2(1)=1:21;p>:20. izationtoolwastoocomplicatedordifficulttoworkwith Additionally, scientists chose to use a spatial (thoughthesetoolscouldnodoubtbeimproved). Thus, transformation to get needed information from thescientists’useofspatialtransformationsdonotseem a visualization rather than changing the visual- tobeasubstitutefor“bad”graphs, butratherastrategy ization, c 2(1)=122:25;p<:001, making plans tounderstandthedatamorethoroughly. to gather more data c 2(1)=184:19;p<:001, or As suggested earlier, current theories of graph comprehension can not account for this pattern of results. abandoning their attempt to answer their question, c 2(1)=204:02;p<:001.1 Current theories (e.g., Bertin, 1983; Kosslyn, 1989; Pinker, 1990) deal primarily with how users extract in- 1Allc 2’susedtheBonferroniadjustment. formation that is explicitly available on a graph or visualization. In this study, we have shown that users do of insight, (pp. 365–395). Cambridge, MA: MIT notsimplyextractinformationthatisexplicitlyshownon Press. a visualization; rather, they extract information and use Dunbar, K. (1996). How scientists think: Online cre- mental imagery to create similar visualizations, modify ativityandconceptualchangeinscience. InT.B. those mental images, and compare their mental results Ward, S. M. Smith, & S. Vaid (Eds.), Creative to on-screen results. These spatial transformation seem thought: An Investigation of Conceptual Struc- tobeusedforavarietyofreasons,includinghypothesis tures and Processes, (pp. 461–493). Washington, testing and understanding their own mental representa- DC:APAPress. tionthroughaprocessofaligningvariousmentalimages (Traftonetal.,2001). Ericsson,K.A.,&Simon,H.A.(1993). Protocolanal- How can we integrate these new results into current ysis: Verbal reports as data (Revised edition). theories? We believe that the current theoretical model Cambridge,MA:MITPress. shouldbeexpandedtoincludespatialtransformationsas Hegarty,M.(1992). Mentalanimation: Inferringmotion part of the cognitive processes that users go through to fromstaticdisplaysofmechanicalsystems. Jour- interpretandusevisualizations. Figure4showsourcur- nalofExperimentalPsychology: Learning,Mem- rentmodelofgraphcomprehension,alongwithevidence oryandCognition,18(5),1084–1102. thatsupportseachstageofthismodel. We believe, as Figure 4 shows, that when people use Kosslyn,S.M.(1989).Understandingchartsandgraphs. graphsorvisualizations,theyinitiallygothroughapro- AppliedCognitivePsychology,3,185–226. cesstounderstandthegraphitself.Then,whentheyneed toextractinformation,theycaneitherreadoffthatinfor- Kosslyn, S. M., Sukel, K. E., & Bly, B. M. (1999). mationdirectlyfromthevisualizationor,ifthatinforma- Squinting with the mind’s eye: Effects of stimu- tionisnotavailable, performaspatialtransformationto lusresolutiononimaginalandperceptualcompar- gettheneededinformation.2 Finally,thatinformationis isons. MemoryandCognition,27(2),276–287. actuallyusedbytheuser. Lohse,G.L.(1993). Acognitivemodelforunderstand- Acknowledgments: This research was supported in inggraphicalperception. HumanComputerInter- partbygrant55-7850-00tothefirstauthorfromtheOf- action,8,353–388. ficeofNavalResearch.WewouldalsoliketothankWai- Tat Fu, Anthony Harrison, and William Liles for com- Pinker, S. (1990). A theory of graph comprehension. mentsonapreviousdraftofthispaper. In R. Freedle (Ed.), Artificial intelligence and the future of testing, (pp. 73–126). Hillsdale, NJ: References LawrenceErlbaumAssociates,Inc. Bertin, J. (1983). Semiology of graphs. Madison, WI: Shepard,R.N.,&Metzler,J.(1971). Mentalrotationof UniversityofWisconsinPress. three-dimensionalobjects.Science,171,701–703. Carpenter,P.A.,&Shah,P.(1998). Amodeloftheper- Trafton, J. G., Kirschenbaum, S. S., Tsui, T. L., ceptualandconceptualprocessesingraphcompre- Miyamoto,R.T.,Ballas,J.A.,&Raymond,P.D. hension. JouralofExperimentalPsychology: Ap- (2000). Turningpicturesintonumbers: Extracting plied,4(2),75–100. and generating information from complex visualizations. International Journal of Human Com- Carter, L. F. (1947). An experiment on the design of puterStudies,53(5),827–850. tables and graphs used for presenting numerical data.JournalofAppliedPsychology,31,640–650. Trafton, J. G., Trickett, S. B., & Mintz, F. E. (2001). Overlaying images: Spatial transformations of Cleveland, W. S., & McGill, R. (1984). Graphical per- complex visualizations. Paper to be presented ception: theory, experimentation, and application at Model-Based Rasoning: Scientific Discovery, to the development of graphical method. Journal TechnologicalInnovation,ValuesinPavia,Italy. of the American Statistical Association, 79, 531– Trickett, S. B., Fu, W., Schunn, C. D., & Trafton, J. G. 553. (2000a). From dipsy-doodles to streaming mo- Cleveland, W. S., & McGill, R. (1986). An experiment tions: Changesinrepresentationintheanalysisof in graphical perception. International Journal of visualscientificdata.InProceedingsoftheTwenty Man-MachineStudies,25,491–500. Second Annual Conference of the Cognitive Sci- enceSociety. Dunbar,K.(1995). Howscientistsreallyreason: Scien- Trickett, S. B., Trafton, J. G., & Schunn, C. D. tific reasoning in real-world laboratories. In R. J. (2000b). Blobs, dipsy-doodles and other funky Sternberg, & J. E. Davidson (Eds.), The nature things: Framework anomalies in exploratory data 2Thismodeldoesnotshowtheotherwaystogatherinfor- analysis. InProceedingsoftheTwentySecondAn- mationbecauseitdidnotshowupasamajorcomponentinour nualConferenceoftheCognitiveScienceSociety. currentdatasets.