User Experience, Performance, and Social Acceptability: Usable Multimodal Mobile Interaction

165 Pages·2012·54.39 MB·English

User Experience, Performance, and
Social Acceptability: Usable
Multimodal Mobile Interaction
Julie R. Williamson
Doctor of Philosophy
University of Glasgow
January 2012

Declaration
Ideclarethatthisthesiswascomposedbymyselfandthattheworkcontainedthereinismy
own,exceptwhereexplicitlystatedotherwiseinthetext.
(JulieR.Williamson)
3
4
Abstract
This thesis explores the social acceptability of multimodal interaction in public places with
respecttoacceptance,adoptionandappropriation. Previousworkinmultimodalinteraction
has mainly focused on recognition and detection issues without thoroughly considering the
willingness of users to adopt these kinds of interactions in their everyday lives. This thesis
presentsanovelapproachtouserexperiencethatistheoreticallymotivatedbyphenomenol-
ogy,practicedwithmixed-methods,andanalysedbasedondramaturgicalmetaphors.
In order to explore the acceptance of multimodal interfaces, this thesis presents three studies
thatlookatusers’initialreactionstomultimodalinteractiontechniques:asurveystudyfocus-
ingongestures,anon-the-streetuserstudy,andafollow-upsurveystudylookingatgesture
andvoice-basedinteraction. Theinvestigationofmultimodalinteractionadoptionisexplored
throughtwostudies: aninsituuserstudyofaperformativeinterfaceandafocusgroupstudy
usingexperienceprototypes. Thisthesisexplorestheappropriationofmultimodalinteractionby
demonstratingthecompletedesignprocessofamultimodalinterfaceusingtheperformative
approachtouserexperiencepresentedinthisthesis.
Chapter 3 looks at users’ initial reactions to and acceptance of multimodal interactions. The
resultsofthefirstsurveyexploredlocationandaudienceasfactorstheinfluencehowindivid-
uals behave in public places. Participants in the on-the-street study described the desirable
visualaspectsofthegesturesasplayful,cool,orembarrassingaspectsofinteractionandhow
gestures could be hidden as everyday actions. These results begin to explain why users ac-
cepted or rejected the gestures from the first survey. The second survey demonstrated that
the presence of familiar spectators made interaction significantly more acceptable. This re-
sultindicatesthatperformativeinteractioncouldbemademoreacceptablebyinterfacesthat
supportcollaborativeorsocialinteraction.
Chapter4exploreshowusersplaceinteractionsintoausabilitycontextforuseinrealworld
settings. In the first user study, participants took advantage of the wide variety of possible
performances,andcreatedawidevarietyofinput,fromhighlyperformativetohiddenactions,
basedonlocation. Theabilityofthisinterfacetosupportflexibleinteractionsallowedusersto
demonstratethethepurposedoftheiractionsdifferentlybasedontheimmediatelyco-located
spectators. Participantsinthefocusgroupstudydiscussedhowtheywouldgoaboutplacing
multimodalinteractionsintorealworldcontexts, usingthreeapproaches: relationshiptothe
device, personal meaning, and relationship to functionality. These results demonstrate how
usersviewinteractionwithinausabilitycontextandhowthatmightaffectsocialacceptability.
Chapter 5 examines appropriation of multimodal interaction through the completion of an
entiredesign process. The resultsofaninitial surveywereused asabaselineof comparison
from which to design the following focus group study. Participants in the focus groups had
similarmotivesforacceptingmultimodalinteractions,althoughthewaysinwhichthesewere
expressedresultedinverydifferentpreferences. Thedesiretousetechnologyinacomfortable
andsatisfyingwaymeantdifferentthingsinthesedifferentsettings. Duringthe‘inthewild’
userstudy,participantsadaptedperformanceinordertomakeinteractionacceptableindiffer-
entcontexts. Insomecases,performancewashiddeninpublicplacesorsharedwithfamiliar
spectatorsinordertosuccessfullyincorporateinteractionintopublicplaces.
5
6
Contributing Publications
Williamson,J.R.SendMeBubbles: MultimodalPerformanceandSocialAcceptability. InExtended
Abstracts (Student Research Competition) of ACM CHI 2011 (Vancouver, Canada). ACM
Press.
Rico,J.,andBrewster,S.GestureandVoicePrototypingforEarlyEvaluationsofSocialAcceptability
inMultimodalInterfaces. InProceedingsofICMI2010(Beijing,China). ACMPress.
Rico, J., Jacucci, G., Reeves, S., Hansen, L., andBrewster, S.DesigningforPerformativeInterac-
tionsinPublicSpaces. InProceedings(Workshops)ofUbicomp2010. ACMPress.
Rico, J.andBrewster, S.UsableGesturesforMobileInterfaces: EvaluatingSocialAcceptability. In
ProceedingsofACMCHI2010(Atlanta,GA,USA),ACMPressAddison-Wesley,pp887-896.
Rico, J. Evaluating the Social Acceptability of Multimodal Mobile Interactions. In Extended Ab-
stracts(DoctoralConsortium)ofACMCHI2010(Atlanta,USA),ACMPressAddison-Wesley.
Rico, J. and Brewster, S. Gestures all around us: differences in social acceptability perceptions of
gesturebasedinterfaces. InProceedingsofMobileHCI2009(Bonn,DE).ACMPress.
7
8
Acknowledgements
IwouldliketothankmyadvisorStephenBrewster,whohasalwaysgivenmeexcellentguid-
ance in my research and supported me in a wide variety of research endeavours. I am con-
stantlyinspiredbySteve’senthusiasmandsheertalentinhiswork. Thisthesiswouldnotbe
possible without his supervision and guidance. I would also like the thank my second su-
pervisorMatthewChalmersforalltheinterestingandchallengingdiscussionsthathelpedto
shapethisresearch.
IwouldlikethethankthethesiscommitteeSteveBenford,QuintinCutts,andRonPoet,forall
oftheirtimespentreviewingthisthesisandtheirguidancetoimprovethefinalsubmission.
IcouldnothavecompletedthisresearchwiththefinancialsupportfromtheNationalScience
FoundationGraduateResearchFellowship,andgenerousdonationsfromNokiaandHPLabs.
Many thanks go to Paul Dourish for giving me my first taste for research and helping me to
find the topics that are most interesting to me and to the LUCI Lab for lots of good times. I
thankmycolleaguesatHPLabsinBangalorewhowelcomedmeandguidedandsupported
myresearchintocrossculturalcomparisonsofsocialacceptability.
Ihavealsobeenluckytohaveanamazingmentorwhoencouragedme,gavemeconfidence,
andhelpedmeseethegoodthingsaboutmyself. Andre´ vanderHoekhasbeenanamazing
mentorsincemyundergraduateyearsandbeenawonderfulinfluenceonmethroughoutmy
postgraduatelife.
IwouldliketothankthefriendsandcolleaguesattheUniversityofGlasgowthatmakework-
ingandlivingheresoamazing. ForallmyfriendsintheDepartment/SchoolofComputing
Science, thank you for being so awesome, having such great chat, and commiserating when
necessary.
I would like to thank John Williamson, who has given me love and support throughout this
processandwhowillinglyreadthisentirethesismorethanonce.
Iamthankfulformyfamilyandtheirlongdistancesupportthroughthegoodtimesandthe
bad,especiallymymother.
Inallofthis,ImustthanktheseriesofmiraculouscircumstancesthatbroughtmetowhereI
amtoday,fromaseriesofbrokencamerasthatchangedmylifeforever,thefortunaterandom
paththatbroughtmetoInformaticsatUCI,totheintroductionfromPaulthatbroughtmeto
Glasgowandhelpedmefindmyselfandmyhappiness.
9
10