Eye Tracking Methodology: Theory and Practice

335 Pages·2007·7.231 MB·English

by  Andrew Duchowski BSc,PhD (auth.)

Eye Tracking Methodology
Andrew Duchowski
Eye Tracking
Methodology
Theory and Practice
SecondEdition
AndrewDuchowski,BSc,PhD
DepartmentofComputerScience
ClemsonUniversity
Clemson,SC29634
[email protected]
ISBN978-1-84628-608-7 e-ISBN978-1-84628-609-4
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
LibraryofCongressControlNumber:2006939204
(cid:2)c Springer-VerlagLondonLimited2007
Apartfromanyfairdealingforthepurposesofresearchorprivatestudy,orcriticismorreview,as
permittedundertheCopyright,DesignsandPatentsAct1988,thispublicationmayonlybereproduced,
stored or transmitted, in any form or by any means, with the prior permission in writing of the
publishers,orinthecaseofreprographicreproductioninaccordancewiththetermsoflicencesissued
bytheCopyrightLicensingAgency.Enquiriesconcerningreproductionoutsidethosetermsoflicenses
issuedbytheCopyrightLicensingAgency.Enquiries concerningreproductionoutsidethoseterms
shouldbesenttothepublishers.
Theuseofregisterednames,trademarks,etc.,inthispublicationdoesnotimply,evenintheabsenceof
aspecificstatement,thatsuchnamesareexemptfromtherelevantlawsandregulationsandtherefore
freeforgeneraluse.
Printedonacid-freepaper.
9 8 7 6 5 4 3 2 1
SpringerScience+BusinessMedia
springer.com
Preface to the Second Edition
Sincethewritingofthefirstedition,severalimportantadvancementsinthefieldof
eye tracking have occurred in the span of just a few short years. Most important,
eye tracking technologyhas improveddramatically. Due to the increased speed of
computerprocessorsandimprovedcomputervisiontechniques,eyetrackingmanu-
facturershavedevelopeddevicesfallingwithinthefourthgenerationofthefollowing
technologicaltaxonomy.
1. First generation:eye-in-headmeasurementof the eye consisting of techniques
suchasscleralcontactlens/searchcoil,electro-oculography
2. Secondgeneration:photo-andvideo-oculography
3. Thirdgeneration:analogvideo-basedcombinedpupil/cornealreflection
4. Fourth generation:digital video-based combined pupil/cornealreflection, aug-
mentedbycomputervisiontechniquesandDigitalSignalProcessors(DSPs)
Often the most desired type of eye tracking output (e.g., for human–computerin-
teractionusabilitystudies)isestimationoftheprojectedPointOfRegard(POR)of
the viewer, i.e., the (x,y) coordinates of the user’s gaze on the computer display.
First-andsecond-generationeyetrackersgenerallydonotprovidethistypeofdata.
(For second-generation systems, eye movement analysis relies on off-line, frame-
by-framevisualinspectionofphotographsorvideoframesanddoesnotalloweasy
POR calculation.)Combinedvideo-basedpupil/cornealreflection eye trackerseas-
ilyprovidePORcoordinatesfollowingcalibration,andaretodayderigeur.Dueto
theavailabilityoffastanalog-to-digitalvideoprocessors,thesethird-generationeye
trackersarecapableofdeliveringthecalculatedPORinreal-time.Fourth-generation
eye trackers, having recently appeared on the market, make use of digital optics.
Coupledwithon-chipDigitalSignalProcessors(DSPs),eyetrackingtechnologyhas
significantlyincreasedinitsusability,accuracy,andspeedwhiledecreasingincost.
Thestateoftoday’stechnologycanbestbesummarizedbyabrieffunctionalcom-
parisonofolderandnewerequipment,giveninthefollowingtable.If,tothosenew
tothisbookanduninitiatedwitheyetrackingdevices,thecomparisoninthetableis
VI PrefacetotheSecondEdition
Functionaleyetrackercomparison.
LegacySystems State-of-the-Art
Technology Analogvideo Digitalvideo
Calibration 5-or9-point,tracker-controlled Anynumber,application-controlled
Optics Manualfocusing/thresholding Auto-focus
Communication Serial(polling/streaming) TCP/IP(client/server)
Synchronization Statusbyteword APIcallback
notimmediatelysuggestive,considertwousescenariosofbotholdandnewtechnol-
ogypresentedinthenexttable.
Eyetrackerusecomparison.
Typicalsessionwitholdsystem. Typicalsessionwithnewsystem.
1. Logintoconsole. 1. Logintoconsole.
2. Turnoneyetrackingequipment. 2. TurnoneyetrackingPC.
3. Turnoneye/scenemonitors. 3. Runeyetrackingprogram.
4. TurnoneyetrackingPC. 4. Calibrate.
5. Runeyetrackingprogram. 5. Run.
6. Turnoncamera.
7. Turnonilluminationcontrol.
8. Adjusthead/chinrest.
9. Adjustpan/tiltunit.
10. Adjustcamerazoom.
11. Adjustcamerafocus.
12. Adjustpupil/cornealthresholds.
13. Calibrate.
14. Run.
Thedisparityintheuseofoldandnewtechnologiesismainlyduetotheuseofdiffer-
entoptics(camera).Newsystemstendtouseanauto-focusingdigitalcamera,e.g.,
embedded in a flat panel display. Although embedding within a flat panel display
mayrestrictauser’sphysicalpositionsomewhat,itisgenerallypresettooperateata
comfortablerange(e.g.,50–60cmfocaldistance).Unlikeoldersystems,aslongas
theusersitswithinthisdistance,nochinrestsandnofurtherparameteradjustments
are needed.In contrast, older devicesrequiredthe use of a pan/tilt unit to position
thecamera,thecameratobemanuallyfocusedandzoomed,andsoftwaretobeset
toappropriatepupilandcornealreflectiondetectionthresholds.Noneofthesecum-
bersomeoperationsarerequiredwithnewersystems.
Furthermore,oneofthemostimportantfeaturesofthenewtechnology,especiallyfor
applicationdevelopment,isanindividual’sabilitytoself-calibrate.Witholdertech-
nology,wheneveradeveloperwishedtotestanewfeature,sheorhehadtorecruit
a(verypatient)subjectfortesting.Thiswasquiteproblematic.Thenewersystems’
PrefacetotheSecondEdition VII
calibration routines are a much-needed improvement over older (third-generation)
technologythatsignificantlyaccelerateprogramandapplicationdevelopment.
A third-generation eye tracker was used for most of the eye tracking research on
which the first edition of this book was based. The availability of new technology
precipitatedthewritingofthesecondedition.Thesecondeditionthereforefillssev-
eralimportantgapsnotcoveredpreviously,namely:
1. Client/servermodelfordevelopinganeyetrackingclientapplication
2. Client-controlleddisplay,calibration,datacollection
3. Newprogrammingexamples
Beyond updated technical descriptions of client programming, the second edition
alsoincludeswhatthefirsteditionlacked:anoverviewof themethodologybehind
the use of eye trackers, that is, experimentaldesign issues that are often needed to
conduct eye tracking studies. The second edition briefly reviews experimental de-
signdecisions,offerssomeguidelinesforincorporatingeyemovementmetricsinto
astudy(e.g.,usability),andprovidesexamplesofcasestudies.
Finally,thesecondeditionexpandsthethirdpartofthebook:eyetrackingapplica-
tions. A great deal of new and exciting eye tracking work has appeared, undoubt-
edly drivenby the availability of new technology.In fact, there now appearsto be
aratherrefreshingshiftinthereportingofeyetrackingandeyemovementstudies.
Authorsnowtendtounderstatethe“gee-whiz”factorofeyetrackersandtheirtech-
nical machinationsneeded to obtain eye movementdata and are now emphasizing
scientificresultsbolsteredbyobjectiveevidenceprovidedbyusers’gazeandhence
attention.Eyetrackingfinallyappearstobeenteringintomainstreamscience,where
theeyetrackerisbecominglessofanoveltyandmoreofatool.Itishopedthatthis
second edition may inspire readers with the simplicity of application development
now made possible by fourth-generationeye trackers and continue on the road to
newapplicationsandscientificinsights.
AndrewT.Duchowski
Clemson,SC,April2006
Preface to the First Edition
Thescopeofthebookfallswithinafairlynarrowhuman–computerinteractiondo-
main (i.e., describinga particular inputmodality),however,it spans a broad range
ofinterdisciplinaryresearchandapplicationtopics.Thereareatleastthreedomains
thatstandtobenefitfromeyetrackingresearch:visualperception,human–computer
interaction,andcomputergraphics.Theamalgamationofthesetopicsformsasym-
bioticrelationship.Graphicaltechniquesprovideameansofgeneratingrichsetsof
visualstimulirangingfrom2Dimageryto3Dimmersivevirtualworldsandresearch
exploringvisualattentionandperceptioninturninfluencesthe generationofartifi-
cialscenesandworlds.Applicationsderivedfromthesedisciplinescreateapower-
ful human–computerinteraction modality, namely interaction based on knowledge
oftheuser’sgaze.
Recent advancements in eye tracking technology, specifically the availability of
cheaper, faster, more accurate, and easier to use trackers, have inspired increased
eyemovementandeyetrackingresearchefforts.However,althougheyetrackersof-
fer a uniquelyobjective view of overthuman visual and attentionalprocesses, eye
trackershave notyetgainedwidespreaduse beyondwork conductedatvariousre-
search laboratories.Thislack of acceptanceis duein partto two reasons:first, the
useofaneyetrackerinanappliedexperimentalsettingisnotawidelytaughtsubject.
Hence,thereisaneedforabookthatmayhelpinprovidingtraining.Itisnotuncom-
mon for enthusiastic purchasersof eye tracking equipmentto become discouraged
withtheirnewlyboughtequipmentwhentheyfinditdifficulttosetupandoperate.
Only a few academic departments (e.g., psychology, computer science) offer any
kindofinstructionintheuseofeyetrackingdevices.Second,toexacerbatethelack
oftrainingineyetrackingmethodology,evenfewersourcesofinstructionexistfor
system development.Settingup aneye trackinglab andintegratingthe eye tracker
intoanavailablecomputersystemfordevelopmentofgaze-contingentapplications
isafairlycomplicatedendeavor,similartothedevelopmentandintegrationofvirtual
realityprograms.Thusfar,itappearsnotextbookotherthanthisoneexistsproviding
thistypeoflow-levelinformation.
X PrefacetotheFirstEdition
The goal of this bookis to providetechnicaldetails for implementationof a gaze-
contingentsystem,couchedinthetheoreticalcontextofeyemovements,visualper-
ception,andvisualattention.Thetextstartedoutastheauthor’spersonalnotesonthe
integrationofacommercialeyetrackerintoavirtualrealitygraphicssystem.These
technicalconsiderationscomprise the middle chaptersof the bookand include de-
tailsofintegratingacommercialeyetrackerintobotha3Dvirtualenvironment,and
a 2D image display application. The surroundingtheoretical review chapters grew
fromnotesdevelopedforan interdisciplinaryeye trackingmethodologycourseof-
feredtobothundergraduatesandgraduatesfromfourdisciplines:psychology,mar-
keting, industrialengineering,and computerscience. An early form of these notes
waspresentedasashortcourseattheAssociationforComputingMachinery(ACM)
SpecialInterestGrouponGraphics’SIGGRAPHconference,23–28July2000,New
Orleans,LA.
Overview
Asofthesecondedition,thebookisdividedintofourparts,presentedthematically
in a top-down fashion, providing first an introduction to the human visual system
(Part I), then briefly surveying eye tracking systems (Part II), then discussing eye
tracking methodology (Part III), and finally ending by reviewing a number of eye
trackingapplications(PartIV).
In the first part, “Introductionto the Human Visual System (HVS),” the bookcov-
ers the concept of visual attention, mainly from a historical perspective. The first
chapterfocusesonthedichotomyoffovealandperipheralvision(the“what”versus
the“where”).Thischaptercoverseasilyobservableattentionalphenomena.Thenext
chaptercoversthe neurologicalsubstrate of theHVS presentingthe low-levelneu-
rologicalelementsimplicatedin dynamichumanvision.Thischapterdiscussesthe
primarydualpathways,theparvo-andmagno-cellularchannels,whichlooselycor-
respondtotheflowofvisualinformationpermittedbytheretinalfoveaandperiph-
ery.Following this description of the visual “hardware”,observablecharacteristics
ofhumanvisionaresummarizedinthefollowingchapteronvisualperception.Here,
resultsobtainedmainly frompsychophysicsare summarized,distinguishingfoveal
and peripheral visual perception. The first part ends by discussing the mechanism
responsible for shifting the fovea, namely eye movements. Having established the
neurologicalandpsychophysicalcontextforeyemovements,the followingchapter
onthetaxonomyandmodelsofeyemovementsgivesthecommontermsforthemost
basic of eye movementsalong with a signal-analyticdescription of recordableeye
movementwaveforms.
Thesecondpartofthebook,“EyeTrackingSystems,”presentsabriefsurveyofthe
main types of available eye tracking devices, followed by a detailed technical de-
scriptionoftherequirementsforsysteminstallationandapplicationprogramdevel-
opment. These details are mainly applicable to video-based,corneal-reflectioneye
PrefacetotheFirstEdition XI
trackers, the most widely available and most affordable type of eye trackers. This
part of the book offers information for the development of three general systems:
oneforbinocular3Deyetrackinginvirtualreality,oneformonocular2Deyetrack-
ingovera2Ddisplay(e.g.,atelevisionmonitoronwhichgraphicalinformationcan
be displayed),and one for binocular2D eye tracking on the desktop. The descrip-
tionsofthefirsttwoformersystemsareverysimilarbecausetheyarebasedonthe
samekindof(older)eyetrackinghardware(ISCANinthisinstance).Thelattersys-
temdescriptionisbasedonmoderneyetrackingtechnologyfromTobii.Bothsystem
descriptionsincludenotesonsystem calibration.Thispartof the bookendswith a
descriptionofdatacollectionandanalysisindependentofanyparticulareyetracking
hardware.
The fourth part of the book surveys a number of interesting and challenging eye
trackingapplications.Applicationsidentifiedinthispartaredrawnfrompsychology,
humanfactors,marketingandadvertising,human–computerinteractionandcollab-
orativesystems,andcomputergraphicsandvirtualreality.
Howto ReadThis Book
The intended audience for this bookis an interdisciplinaryone, aimed particularly
at those interested in psychology, marketing, industrial engineering, and computer
science.Indeed,thistextismeantforundergraduatesandgraduatesfromthesedis-
ciplines enrolled in a course dealing with eye tracking, such as the eye tracking
methodologycoursedevelopedbytheauthoratClemsonUniversity.Inthiscourse,
typically all chapters are covered,but not necessarily in the order presented in the
text.Insuchacourse,theorderofchaptersmaybeasfollows.
First,PartIVispresentedoutliningvariouseyetrackingapplications.Normally,this
partshouldgivethereadermotivationfordesignandimplementationofasemester-
longeyetrackingproject.Coverageofthispartofthebookisusuallysupplemented
by readings of research papers from various sources. For example, papers may be
selectedfromthefollowingconferences.
• TheproceedingsoftheEyeTrackingResearch&Applications(ETRA)confer-
ence
• TheproceedingsoftheACMSpecialInterestGrouponHuman–ComputerInter-
action(SIGCHI)conference(HumanFactorsinComputing)
• Transactionson Graphics, the proceedingsof the annualAssociation for Com-
puting Machinery (ACM) Special Interest Group on Graphics and Interactive
Techniques(SIGGRAPH)conferenceseries
• TheproceedingsoftheHumanFactorsandErgonomicsSociety(HFES)
Tospeedupdevelopmentofaneyetrackingapplication,PartIIfollowsthepresenta-
tionofPartIV,dealinginthetechnicaldetailsofeyetrackerapplicationdevelopment.
XII PrefacetotheFirstEdition
Thetypesofapplicationsthatcanbeexpectedofstudentswilldependmainlyonthe
programmingexpertiserepresentedby membersofinterdisciplinarystudentteams.
Forexample,intheeyetrackingmethodologycourseatClemson,teamsareformed
byjoiningcomputersciencestudentswith oneormoreoftheotherrepresentatives
enrolled in the class (i.e., from marketing, psychology, or industrial engineering).
Although all group members decide on a project, students studying the latter sub-
jectsaremainlyresponsibleforthedesignandanalysisoftheeventualeyetracking
experiment.
Givencommencementofaneyetrackingapplication,PartIIIisthencovered,going
overexperimentaldesign.Inthecontextoftheusabilitymeasurementframework,the
eye tracking methodology course advocates performance measurement, and there-
forefocusesonlaboratoryexperimentsandquantitativedataanalysis.
PartIofthetextiscoveredlast,givingstudentsthenecessarytheoreticalcontextfor
the eye tracking pilot study. Thus, although the book is arranged “top-down”, the
courseproceeds“bottom-up”.
The book is also suitable for researchers interested in setting up an eye tracking
laboratory and/or using eye trackers for conducting experiments. Because readers
withthesegoalsmayalsocomefromdiversedisciplinessuchasmarketing,psychol-
ogy,industrialengineering,andcomputerscience,notallpartsofthebookmaybe
suitable for everyone. More technically oriented readers will want to pay particu-
larattentiontothemiddlesectionsofthebookwhichdetailsysteminstallationand
implementationof eyetrackingapplicationsoftware.Readersnotdirectlyinvolved
with such low-level details may wish to omit these sections and concentrate more
onthetheoreticalandhistoricalaspectsgiveninthefrontsectionsofthebook.The
latterpartofthebook,dealingwitheyetrackingapplications,shouldbesuitablefor
allreadersinasmuchasitpresentsexamplesofcurrenteyetrackingresearch.
Acknowledgments
ThisworkwassupportedinpartbyaUniversityInnovationgrant(#1-20-1906-51-
4087),NASAAmestask(#NCC2-1114),andNSFCAREERaward#9984278.
Thepreparationofthisbookhasbeenassistedbymanypeople,includingKeithKarn,
RoelVertegaal,DorionListon,andKeithRaynerwhoprovidedcommentsonearly
editionsofthe text-in-progress.Laterversionsofthedraftwerereviewedbyexter-
nalreviewerstowhomIexpressmygratitude,fortheircommentsgreatlyimproved
thefinalversionofthetext.SpecialthanksgotoDavidWoodingforhiscarefuland
thoroughreviewofthetext.
IwouldliketothanktheteamatSpringerforhelpingmecomposethetext.Thanks
goto BeverlyFordandKarenBorthwickforeggingme onto writethe textandto