Eye Tracking and Visual Analytics

382 Pages·2022·44.123 MB·English

by  Michael Burch PhD

Eye Tracking and Visual Analytics
RIVER PUBLISHERS SERIES IN INFORMATION
SCIENCE AND TECHNOLOGY
SeriesEditors:
K.C.Chen SandeepShukla
NationalTaiwanUniversity,Taipei,Taiwan VirginiaTech,USA
and and
UniversityofSouthFlorida,USA IndianInstituteofTechnologyKanpur,India
The “River Publishers Series in Information Science and Technology” covers research
which ushers the 21st Century into an Internet and multimedia era. Multimedia means the
theoryandapplicationoffiltering,coding,estimating,analyzing,detectingandrecognizing,
synthesizing,classifying,recording,andreproducingsignalsbydigitaland/oranalogdevices
or techniques, while the scope of “signal” includes audio, video, speech, image, musical,
multimedia, data/content, geophysical, sonar/radar, bio/medical, sensation, etc. Networking
suggeststransportationofsuchmultimediacontentsamongnodesincommunicationand/or
computernetworks,tofacilitatetheultimateInternet.
Theory, technologies, protocols and standards, applications/services, practice and
implementation of wired/wireless networking are all within the scope of this series. Based
on network and communication science, we further extend the scope for 21st Century life
throughtheknowledgeinrobotics,machinelearning,embeddedsystems,cognitivescience,
patternrecognition,quantum/biological/molecularcomputationandinformationprocessing,
biology,ecology,socialscienceandeconomics,userbehaviorsandinterface,andapplications
tohealthandsocietyadvance.
Bookspublishedintheseriesincluderesearchmonographs,editedvolumes,handbooks
andtextbooks.Thebooksprovideprofessionals,researchers,educators,andadvancedstudents
inthefieldwithaninvaluableinsightintothelatestresearchanddevelopments.
Topicscoveredintheseriesinclude,butarebynomeansrestrictedtothefollowing:
• Communication/ComputerNetworkingTechnologiesandApplications
• QueuingTheory
• Optimization
• OperationResearch
• StochasticProcesses
• InformationTheory
• Multimedia/Speech/VideoProcessing
• ComputationandInformationProcessing
• MachineIntelligence
• CognitiveScienceandBrianScience
• EmbeddedSystems
• ComputerArchitectures
• ReconfigurableComputing
• CyberSecurity
Foralistofotherbooksinthisseries,visitwww.riverpublishers.com
Eye Tracking and Visual Analytics
Michael Burch
UniversityofAppliedSciences,Chur,Switzerland
River Publishers
Published,soldanddistributedby:
RiverPublishers
Alsbjergvej10
9260Gistrup
Denmark
www.riverpublishers.com
ISBN:978-87-7022-433-8(Hardback)
978-87-7022-432-1(Ebook)
©2021RiverPublishers
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedin
aretrievalsystem,ortransmittedinanyformorbyanymeans,mechanical,
photocopying,recordingorotherwise,withoutpriorwrittenpermissionof
thepublishers.
Contents
Preface xi
ListofFigures xiii
ListofTables xxxi
ListofAbbreviations xxxiii
1 Introduction 1
1.1 Tasks,Hypotheses,andHumanObservers . . . . . . . . . . 3
1.2 SynergyEffects . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 DynamicVisualAnalytics . . . . . . . . . . . . . . . . . . 11
2 Visualization 17
2.1 MotivatingExamples . . . . . . . . . . . . . . . . . . . . . 19
2.2 HistoricalBackground . . . . . . . . . . . . . . . . . . . . 27
2.2.1 EarlyFormsofVisualizations . . . . . . . . . . . . 28
2.2.2 TheAgeofCartographicMaps. . . . . . . . . . . . 30
2.2.3 VisualizationDuringIndustrialization . . . . . . . . 32
2.2.4 AftertheInventionoftheComputer . . . . . . . . . 34
2.2.5 VisualizationToday . . . . . . . . . . . . . . . . . 36
2.3 DataTypesandVisualEncodings . . . . . . . . . . . . . . 38
2.3.1 PrimitiveData . . . . . . . . . . . . . . . . . . . . 39
2.3.2 ComplexData . . . . . . . . . . . . . . . . . . . . 42
2.3.3 MixtureofData . . . . . . . . . . . . . . . . . . . . 48
2.3.4 DynamicData . . . . . . . . . . . . . . . . . . . . 50
2.3.5 Metadata . . . . . . . . . . . . . . . . . . . . . . . 52
2.4 InteractionTechniques . . . . . . . . . . . . . . . . . . . . 53
2.4.1 InteractionCategories . . . . . . . . . . . . . . . . 54
2.4.2 PhysicalDevices . . . . . . . . . . . . . . . . . . . 58
2.4.3 Users-in-the-Loop . . . . . . . . . . . . . . . . . . 61
v
vi Contents
2.5 DesignPrinciples . . . . . . . . . . . . . . . . . . . . . . . 62
2.5.1 VisualEnhancementsandDecorations . . . . . . . . 63
2.5.2 VisualStructuringandOrganization . . . . . . . . . 65
2.5.3 GeneralDesignFlaws . . . . . . . . . . . . . . . . 66
2.5.4 GestaltLaws . . . . . . . . . . . . . . . . . . . . . 68
2.5.5 OpticalIllusions . . . . . . . . . . . . . . . . . . . 71
3 VisualAnalytics 75
3.1 KeyConcepts . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.1.1 OriginandFirstStages . . . . . . . . . . . . . . . . 78
3.1.2 DataHandlingandManagement . . . . . . . . . . . 79
3.1.3 SystemIngredientsAroundtheData . . . . . . . . . 86
3.1.4 InvolvedResearchFieldsandFuturePerspectives . . 88
3.2 VisualAnalyticsPipeline . . . . . . . . . . . . . . . . . . . 91
3.2.1 DataBasisandRuntimes . . . . . . . . . . . . . . . 91
3.2.2 Patterns,Correlations,andRules . . . . . . . . . . . 93
3.2.3 TasksandHypotheses . . . . . . . . . . . . . . . . 97
3.2.4 RefinementsandAdaptations . . . . . . . . . . . . 102
3.2.5 InsightsandKnowledge . . . . . . . . . . . . . . . 104
3.3 ChallengesofAlgorithmicConcepts . . . . . . . . . . . . . 105
3.3.1 AlgorithmClasses . . . . . . . . . . . . . . . . . . 106
3.3.2 ParameterSpecifications . . . . . . . . . . . . . . . 110
3.3.3 AlgorithmicRuntimeComplexities . . . . . . . . . 111
3.3.4 PerformanceEvaluation . . . . . . . . . . . . . . . 112
3.3.5 InsightsintotheRunningAlgorithm . . . . . . . . . 114
3.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.4.1 DynamicGraphs . . . . . . . . . . . . . . . . . . . 117
3.4.2 DigitalandComputationalPathology . . . . . . . . 118
3.4.3 MalwareAnalysis . . . . . . . . . . . . . . . . . . 119
3.4.4 VideoDataAnalysis . . . . . . . . . . . . . . . . . 120
3.4.5 EyeMovementData . . . . . . . . . . . . . . . . . 122
4 UserEvaluation 125
4.1 StudyTypes . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.1.1 Pilotvs.RealStudy . . . . . . . . . . . . . . . . . . 128
4.1.2 Quantitativevs.Qualitative . . . . . . . . . . . . . . 129
4.1.3 Controlledvs.Uncontrolled . . . . . . . . . . . . . 130
4.1.4 Expertvs.Non-Expert . . . . . . . . . . . . . . . . 132
4.1.5 Short-termvs.Longitudinal . . . . . . . . . . . . . 134
Contents vii
4.1.6 Limited-numberPopulationvs.Crowdsourcing . . . 135
4.1.7 Fieldvs.Lab . . . . . . . . . . . . . . . . . . . . . 136
4.1.8 Withvs.WithoutEyeTracking . . . . . . . . . . . . 138
4.2 HumanUsers . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.2.1 LevelofExpertise . . . . . . . . . . . . . . . . . . 139
4.2.2 AgeGroups . . . . . . . . . . . . . . . . . . . . . . 141
4.2.3 CulturalDifferences . . . . . . . . . . . . . . . . . 142
4.2.4 VisionDeficiencies . . . . . . . . . . . . . . . . . . 144
4.2.5 EthicalGuidelines . . . . . . . . . . . . . . . . . . 145
4.3 StudyDesignandIngredients . . . . . . . . . . . . . . . . . 147
4.3.1 HypothesesandResearchQuestions . . . . . . . . . 148
4.3.2 VisualStimuli. . . . . . . . . . . . . . . . . . . . . 149
4.3.3 Tasks . . . . . . . . . . . . . . . . . . . . . . . . . 151
4.3.4 IndependentandDependentVariables . . . . . . . . 153
4.3.5 Experimenter . . . . . . . . . . . . . . . . . . . . . 157
4.4 StatisticalEvaluationandVisualResults . . . . . . . . . . . 158
4.4.1 DataPreparationandDescriptiveStatistics . . . . . 160
4.4.2 StatisticalTestsandInferentialStatistics . . . . . . . 161
4.4.3 VisualRepresentationoftheStudyResults . . . . . 163
4.5 ExampleUserStudiesWithoutEyeTracking . . . . . . . . 167
4.5.1 HierarchyVisualizationStudies . . . . . . . . . . . 168
4.5.2 GraphVisualizationStudies . . . . . . . . . . . . . 169
4.5.3 InteractionTechniqueStudies . . . . . . . . . . . . 171
4.5.4 VisualAnalyticsStudies . . . . . . . . . . . . . . . 172
5 EyeTracking 175
5.1 TheEye . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
5.1.1 EyeAnatomy . . . . . . . . . . . . . . . . . . . . . 178
5.1.2 EyeMovementandSmoothPursuit . . . . . . . . . 179
5.1.3 DisordersandDiseasesInfluencingEyeTracking . . 181
5.1.4 Corrected-to-NormalVision . . . . . . . . . . . . . 183
5.2 EyeTrackingHistory . . . . . . . . . . . . . . . . . . . . . 185
5.2.1 TheEarlyDays . . . . . . . . . . . . . . . . . . . . 186
5.2.2 ProgressintheField . . . . . . . . . . . . . . . . . 188
5.2.3 EyeTrackingToday . . . . . . . . . . . . . . . . . 190
5.2.4 Companies,Technologies,andDevices . . . . . . . 192
5.2.5 ApplicationFields . . . . . . . . . . . . . . . . . . 192
5.3 EyeTrackingDataProperties . . . . . . . . . . . . . . . . . 197
5.3.1 VisualStimuli. . . . . . . . . . . . . . . . . . . . . 199
viii Contents
5.3.2 GazePoints,Fixations,Saccades,andScanpaths . . 202
5.3.3 AreasofInterest(AOIs)andTransitions . . . . . . . 204
5.3.4 PhysiologicalandAdditionalMeasures . . . . . . . 206
5.3.5 DerivedMetrics. . . . . . . . . . . . . . . . . . . . 208
5.4 ExamplesofEyeTrackingStudies . . . . . . . . . . . . . . 209
5.4.1 EyeTrackingforStaticVisualizations . . . . . . . . 210
5.4.2 EyeTrackingforInteractionTechniques . . . . . . . 215
5.4.3 EyeTrackingforText/Label/CodeReading . . . . . 218
5.4.4 EyeTrackingforUserInterfaces . . . . . . . . . . . 221
5.4.5 EyeTrackingforVisualAnalytics . . . . . . . . . . 223
6 EyeTrackingDataAnalytics 229
6.1 DataPreparation . . . . . . . . . . . . . . . . . . . . . . . 230
6.1.1 DataCollectionandAcquisition . . . . . . . . . . . 231
6.1.2 OrganizationandRelevance . . . . . . . . . . . . . 232
6.1.3 DataAnnotationandAnonymization . . . . . . . . 234
6.1.4 DataInterpretation . . . . . . . . . . . . . . . . . . 235
6.1.5 DataLinking . . . . . . . . . . . . . . . . . . . . . 236
6.2 DataStorage,Adaptation,andTransformation . . . . . . . . 237
6.2.1 DataStorage . . . . . . . . . . . . . . . . . . . . . 238
6.2.2 Validation,Verification,andCleaning . . . . . . . . 240
6.2.3 DataEnhancementandEnrichment . . . . . . . . . 241
6.2.4 DataTransformation . . . . . . . . . . . . . . . . . 242
6.3 AlgorithmicAnalyses . . . . . . . . . . . . . . . . . . . . . 243
6.3.1 OrderingandSorting . . . . . . . . . . . . . . . . . 244
6.3.2 DataClustering . . . . . . . . . . . . . . . . . . . . 245
6.3.3 Summarization,Classing,andClassification . . . . . 247
6.3.4 NormalizationandAggregation . . . . . . . . . . . 248
6.3.5 ProjectionandDimensionalityReduction . . . . . . 249
6.3.6 CorrelationandTrendAnalysis . . . . . . . . . . . 250
6.3.7 PairwiseorMultipleSequenceAlignment . . . . . . 252
6.3.8 ArtificialIntelligence-RelatedApproaches. . . . . . 253
6.4 VisualizationTechniquesandVisualAnalytics . . . . . . . . 254
6.4.1 StatisticalPlots . . . . . . . . . . . . . . . . . . . . 256
6.4.2 Point-basedVisualizationTechniques . . . . . . . . 257
6.4.3 AOI-basedVisualizationTechniques . . . . . . . . . 261
6.4.4 EyeTrackingVisualAnalytics . . . . . . . . . . . . 263
7 OpenChallenges,Problems,andDifficulties 267
Contents ix
7.1 EyeTrackingChallenges . . . . . . . . . . . . . . . . . . . 267
7.2 EyeTrackingVisualAnalyticsChallenges . . . . . . . . . . 269
References 273
Index 335
AbouttheAuthor 347