Doing better statistics in human-computer interaction

254 Pages·2019·1.12 MB·English

by  Cairns, Paul

DOING BETTER STATISTICS
IN HUMAN–COMPUTER
INTERACTION
Eachchapterofthisbookcoversspecifictopicsinstatisticalanalysis,
such as robust alternatives to t-tests or how to develop a question-
naire.Theyalsoaddressparticularquestionsonthesetopicsthatare
commonly asked by HCI researchers when planning or completing
theanalysisoftheirdata.Thebookpresentsthecurrentbestpractice
in statistics, drawing on the state-of-the-art literature that is rarely
presented in HCI. This is achieved by providing strong arguments
thatsupportgoodstatisticalanalysiswithoutrelyingonmathematical
explanations.Itadditionallyofferssomephilosophicalunderpinning
forstatistics,sothatreaderscanseehowstatisticsfitwithexperimental
designandthefundamentalgoalofdiscoveringnewHCIknowledge.
paul cairns is a Reader in Human–Computer Interaction at
the University of York, UK, and Scholar-in-Residence for The
AbleGamers Charity, which helps people with disabilities combat
social isolation by making videogames more accessible. He has
taught statistics at all levels of education for nearly twenty years.
His particular research interest is in players’ experiences of digital
games,andhisexpertiseinexperimentalandstatisticalmethodswas
developedthroughworkinginthisarea.

DOING BETTER STATISTICS
IN HUMAN–COMPUTER
INTERACTION
PAUL CAIRNS
UniversityofYork
UniversityPrintingHouse,Cambridgecb28bs,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,ny10006,USA
477WilliamstownRoad,PortMelbourne,vic3207,Australia
314–321,3rdFloor,Plot3,SplendorForum,JasolaDistrictCentre,NewDelhi–110025,India
79AnsonRoad,#06–04/06,Singapore079906
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781108482523
doi:10.1017/9781108685139
©CambridgeUniversityPress2019
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2019
PrintedintheUnitedKingdombyTJInternationalLtd.PadstowCornwall
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
isbn978-1-108-48252-3Hardback
isbn978-1-108-71059-6Paperback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracy
ofURLsforexternalorthird-partyinternetwebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
ForHB,myground,myskyandmysunshine,
andforSamson,whomwemiss.

Contents
ListofFigures page xi
ListofTables xiv
Acknowledgements xv
GettingStarted 1
parti whyweusestatistics 9
1 HowStatisticsSupportScience 11
1.1 TheProblemofInduction 12
1.2 SevereTesting 15
1.3 EvidenceinHCI 17
1.4 NewExperimentalisminHCI 18
1.5 BigData 21
1.6 Conclusions 22
2 TestingtheNull 25
2.1 TheBasicsofNHST 26
2.2 Goingbeyondp-Values 30
2.3 NHSTandSevereTesting 33
2.4 HonestyinStatistics 35
3 ConstrainingBayes 38
3.1 DefiningProbability 40
3.2 Plausibility 42
3.3 UnconstrainedBayes 44
3.4 TheBayesianCritiqueofFrequentism 47
3.5 BeingCareful:AResponsetotheCritique 48
3.6 So,FrequentistorBayesian? 50
4 Effects:WhatTestsTest 53
4.1 Location 55
4.2 Dominance 58
4.3 Variation 59
4.4 EstimationandSignificance 60
vii
viii Contents
4.5 Big,SmallandZeroEffects 62
4.6 ChoosingTeststoSeeEffects 64
partii howtousestatistics 69
5 PlanningYourStatisticalAnalysis 71
5.1 Principle1:Articulation 73
5.2 Principle2:Simplicity 75
5.3 Principle3:Honesty 77
5.4 Conclusions 79
6 ACautionaryTail:WhyYouShouldNotDoaOne-TailedTest 80
6.1 ATaleofTwoTails 81
6.2 One-TailBad,Two-TailsBetter 82
7 IsThisNormal? 86
7.1 WhatMakesDataNormal? 86
7.2 TheProblemsofNon-normalData 90
7.3 TestingforNormality 91
7.4 Implications 93
8 SortingOutOutliers 95
8.1 DetectingOutliers 96
8.2 SourcesandRemediesforOutliers 98
8.2.1 ErrorsinData 99
8.2.2 MischievousParticipants 99
8.2.3 FaultyStudyDesign 100
8.2.4 NaturalVariation 101
8.3 Conclusions 102
9 PowerandTwoTypesofError 104
9.1 TypeIandTypeIIErrors 105
9.2 DefiningPower 106
9.3 PowerandSampleSizes 108
9.4 PowerandtheQualityofTests 110
9.5 Summary 112
10 UsingNon-ParametricTests 114
10.1 TheMechanicsofRanks 115
10.2 AnalysingErrors 117
10.2.1 TypeIErrors 117
10.2.2 TypeIIErrors 119
10.3 PracticalUse 120
10.4 ReportingNon-ParametricTests 122
10.5 Summary 123