Evaluating Re-authentication Strategies for Smartphones

17 Pages·2016·1.01 MB·English

Ask Me Again But Don’t Annoy Me:
Evaluating Re-authentication Strategies
for Smartphones
Lalit Agarwal, Hassan Khan, and Urs Hengartner, University of Waterloo
https://www.usenix.org/conference/soups2016/technical-sessions/presentation/agarwal
This paper is included in the Proceedings of the
Twelfth Symposium on Usable Privacy and Security (SOUPS 2016).
June 22–24, 2016 • Denver, CO, USA
ISBN 978-1-931971-31-7
Open access to the Proceedings of the
Twelfth Symposium on Usable Privacy
and Security (SOUPS 2016)
is sponsored by USENIX.
Ask me again but don’t annoy me: Evaluating
re-authentication strategies for smartphones
Lalit Agarwal, Hassan Khan and Urs Hengartner
CheritonSchoolofComputerScience
UniversityofWaterloo
Waterloo,ONCanada
{lagarwal, h37khan, urs.hengartner}@uwaterloo.ca
ABSTRACT usersareauthenticatingusingtext-basedpasswords[29]. To
Re-authenticating users may be necessary for smartphone mitigate these usability issues, researchers have proposed
authenticationschemesthatleverageuserbehaviour,device severaltechniquesthatreducetheauthenticationburdenby
context, or task sensitivity. However, due to the unpre- leveraging user behaviour [21, 32, 37], device context [16,
dictablenatureofre-authentication,usersmaygetannoyed 24, 25] or the sensitivity of launched apps [17].
whentheyhavetousethedefault,non-transparentauthen-
While these schemes reduce the authentication burden on
tication prompt for re-authentication. Weaddressthis con-
theuser,theymayrequiremid-taskre-authentication. Sch-
cern by proposing several re-authentication configurations
emes that leverage user behaviour need re-authentication
with varying levels of screen transparency and an optional
in case of a behaviour mismatch against the current phone
timedelaybeforedisplayingtheauthenticationprompt. We
user. Similarly, device context-based schemes may need to
conductuserstudieswith30participantstoevaluatetheus-
establish a user’s identity in case a contextual source (e.g.,
abilityandsecurityperceptionsoftheseconfigurations. We
ambient noise) changes. Taking the sensitivity of launched
find that participants respond positively to our proposed
apps into account for authentication may also require mid-
changes and utilize the time delay while they are antici-
task re-authentication. For instance, some users have indi-
pating to get an authentication prompt to complete their
cated that for a messenger app only opening old messages
current task. Though our findings indicate no differences
should trigger re-authentication [17].
in terms of task performance against these configurations,
we find that the participants’ preferences for the configu- Preliminaryevaluationsshowthatusersliketheconvenience
rations are context-based. They generally prefer the re- offered by these schemes [4, 16, 17, 19, 24]; however, a
authentication configuration with a non-transparent back- fieldstudyofbehaviour-basedauthenticationshowsthatre-
groundforsensitiveapplications,suchasbankingandphoto authentications are a potential issue [19]. More specifically,
apps, while their preferences are inclined towards conve- the evaluated scheme used a (simulated) behaviour-based
nient, usable configurations for medium and low sensitive authenticationschemethatfocusedontheuser’stouchinput
appsorwhiletheyareusingtheirdevicesathome. Wecon- behaviour. Whenever re-authentication was required, the
cludewithsuggestionstoimprovethedesignofourproposed user’s current task was interrupted and a re-authentication
configurationsaswellasadiscussionofguidelinesforfuture prompt with dark background, similar to the standard An-
implementations of re-authentication schemes. droid authentication prompt, appeared immediately. Non-
surprisingly the unpredictability of a re-authentication and
1. INTRODUCTION thecontextswitchduetothetaskinterruptionwereannoy-
The increased usage of smartphones to access personal and ing to some users.
corporatedatarequiresauthenticationatmultiplelevels. A
Whilere-authenticationisunavoidabletoprecludemisuseof
device-levelauthenticationscheme,suchasaPINorfinger-
adeviceoranapp,theunpredictabilityofre-authentication
printrecognition,isrequiredtoprotectaccesstothedevice
can be reduced by delaying the transition between the cur-
while text-based passwords may be required to further es-
rent task and the re-authentication prompt through a fade-
tablish identity for social networking, banking or enterprise
ineffect. Duringthefade-in,theuserisallowedtocontinue
apps. Existing studies have shown that the short and fre-
interacting with their current task on the device. In addi-
quentnatureofsmartphonesessionscreatesusabilityissues
tion to the fade-in effect, the re-authentication prompt can
for device-level authentication schemes [17] whereas con-
beconfiguredtohavevaryinglevelsoftransparencytopro-
strained keyboards on smartphones are a bottleneck when
vide a visual of the user’s current task in the background.
The fade-in effect should reduce the unpredictability of the
re-authentication and a visual of the current task of the
user should reduce the context switch overhead due to re-
authentications. Together these controls have the potential
toprovideincreasedusabilityatthecostofreducedsecurity.
Copyrightisheldbytheauthor/owner. Permissiontomakedigitalorhard
In this paper, we evaluate different configurations of ex-
copiesofallorpartofthisworkforpersonalorclassroomuseisgranted
withoutfee. plicitauthenticationschemes(suchasPINsorpattern-locks)
SymposiumonUsablePrivacyandSecurity(SOUPS)2016,June22–24, whenusedforre-authentication. Ourfocusisonthefade-in
2016,Denver,Colorado.
USENIX Association 2016 Symposium on Usable Privacy and Security 221
effectandthetransparencyofthere-authenticationprompt. ing their explicit input. Various IA schemes have been pro-
We choose behaviour-based authentication as a target use posedthatauthenticateusersthroughtheirtouchinputbe-
case to evaluate the different configurations; however, our haviour[13,21,37],keystrokebehaviour[8,10,14],gaitbe-
findings can be generalized to other authentication propos- haviour [12, 27] or device usage behaviour [32, 33]. Several
als that require re-authentications. In addition to the re- IAproposalshavebeenshowntoprovideover95%accuracy
authenticationconfigurationusedinthepreviouswork[19], [13, 21, 37] and researchers have proposed to use them as a
weselectthreeconfigurationsofexplicitauthenticationsch- primaryauthenticationmechanismforuserswhodonotlock
emes for re-authentication: (i) The authentication prompt their device or as a secondary authentication mechanism to
appearsimmediately(nofade-in)andthebackgroundofthe compliment the existing primary authentication schemes.
authentication prompt is transparent to provide a visual of
TherearescenarioswhenanIAschemeisunsureaboutthe
theuser’scurrenttaskinthebackground; (ii)theauthenti-
identity of the user. This uncertainty may be caused by
cation prompt appears immediately and the background of
an adversary using the device or it could be the result of
theauthenticationpromptgraduallytransitionsfromtrans-
a false reject. False rejects occur when legitimate users are
parenttoopaqueforimprovedsecurity;and(iii)theauthen-
misclassified as adversaries. When an IA scheme is unsure
tication prompt appears after a four second fade-in delay
about the identity of the user, it uses an explicit authenti-
andthebackgroundoftheauthenticationpromptgradually
cationmechanismtore-authenticatetheuser. Furthermore,
transitions from transparent to opaque.
if an IA scheme relies on the input behaviour of the user,
We perform lab experiments using synthetic tasks to evalu- the false rejects can occur mid-task and re-authentication
atethesecurityperception,easeofuse,obstructivenessand requires interrupting the current task of the user [19].
annoyance of PIN and pattern-lock-based re-authentication
Context-aware authentication: Several schemes have
basedonthedefaultconfigurationfromtheearlierstudy[19]
been proposed that leverage device context to reduce au-
(asabaseline)andthemodifiedconfigurations. Inaddition
thentication overhead [16, 24, 25, 28]. These schemes rely
to these qualitative usability metrics, we collect quantita-
on a variety of contextual sources, including location, prox-
tivedataonthetaskefficiencyandthetaskerrorratefora
imitytoWiFiandBluetoothdevices,andambientlightand
multifaceted evaluation of these configurations. Finally, we
noise. An evaluation of CASA [16] shows that it can re-
conduct interviews to gather participants’ perceptions on
duceexplicitauthenticationsby68%andalabstudyofthe
thesensitivityofdifferentkindsofappsandofparticipants’
scheme proposed by Riva et al. [28] indicates that it can
preferred configuration of the re-authentication prompt for
reduce the number of explicit authentications by 42%.
different apps and different environments.
Context-aware schemes can be deployed to sense and assist
Our study was completed by 30 participants. Though our
in authentication only when users begin their interaction
findingsindicatenodifferencesfortheuserperformance(in
withthedevice. However,toprecludeattacksfrominformed
terms of task efficiency, task error rate, and context switch
attackers (such as friends and coworkers), a continuous au-
overhead) against these configurations, participants found
thentication scenario is more suitable. For instance, a con-
all three modified configurations to be less annoying and
tinuousproximitysensingschemewillnotallowaninformed
less obstructive as compared to the default configuration.
maliciouscoworkertounlockthedeviceattheworkplaceand
The modified configurations were also at least as easy to
thenmovetoasecludedplacetoaccesspersonaldataonthe
useasthedefaultconfiguration. Asexpected,theperceived
device. Since such scenarios may arise with the legitimate
security level of the modified configurations was quite low
user of the device (e.g., the device owner moves out of the
when comparedto the default configuration. Whilethe low
proximityrangewhileusingthedevice,oranambientnoise
perceived level of protection was a bottleneck in the adop-
sensor may switch off), the device owner may be subjected
tionofthemodifiedconfigurationsinhigh-riskenvironments
to mid-task re-authentication.
and for sensitive content, a significant number of partici-
pantspreferredtheproposedconfigurationsoverthedefault App-specific authentication: Hayashi et al. [17] show
configurationforlesssensitivecontentandforlow-riskenvi- that all-or-nothing access to smartphones does not align
ronments. Wealsocommunicatesuggestionsbythepartici- with user preferences. They find that while the majority
pantsonhowtoimprovethedesignofourproposedconfigu- of the users prefer to be authenticated for select apps only,
rationsandwediscussguidelinesforfutureimplementations for a subset of apps the users want some functionality to
of re-authentication schemes. be available always and some functionality to be available
after authentication. For instance, browsing existing en-
2. MOTIVATION tries(suchascontacts)inanappshouldalwaysbeavailable
Implicit factors have been proposed to reduce authentica- while modifying or deleting entries should require authen-
tion overhead on the web [2], personal computers [22] and tication. Similarly, looking at recent messages should not
smartphones[17,25,32]. Ourfocusisonsmartphones. The require authentication while browsing old messages should
implicit factors for authentication on smartphones leverage require user authentication. These scenarios require mid-
behavioural biometrics [32], device context [16, 24, 25] or task re-authentication of the user.
thesensitivityoflaunchedapps[17]. Wenextdescribeeach
of these three implicit factors and their potential need to 2.2 NeedforBetterRe-authenticationSchemes
re-authenticate a smartphone user.
User studies on IA show that users find IA to be more con-
2.1 Re-authenticationScenarios venient and easier to use than traditional authentication
Implicit authentication (IA): IA uses behavioural bio- schemes [4, 19]. Evaluations of the context-aware schemes
metrics to conveniently authenticate users without requir- show that the reduced authentication overhead is found to
222 2016 Symposium on Usable Privacy and Security USENIX Association
be useful and the users indicated that they would use the enteranincorrectcharactertoauthenticatewhenthephone
evaluated scheme if it was available on their devices [16, vibrates [6].
24]. A similar positive experience was reported for an app
These approaches are promising; however, they may intro-
sensitivity based authentication scheme [17].
duce confounding factors as they have not been adopted
While users agree that these schemes are useful and are in- widely. The missing experience of the participants with
terested in adopting them, most of these evaluations have thesenewconfigurationdesignmayaffecttheirusabilityper-
notinvestigatedtheeffectofre-authenticationswiththeex- ceptions. Since several usability issues can be traced to the
ceptionofKhanetal. intheirusabilitystudyoftouchinput- unpredictabilityandcontext-switcheffectsofre-authenticat-
based IA [19]. Khan et al. find that for 35% of the partic- ion[19],weperformexperimentstoinvestigatewhetherthe
ipants, re-authentications due to false rejects were a source unwantedeffectsstemmingfromunpredictabilityandcontext-
ofannoyance. Theparticipantsfoundthere-authentications switches can be minimized for widely deployed authenti-
to be frustrating due to their unpredictable nature and the cation mechanisms. Therefore, the main objective of this
accompanying context-switch due to authentication inter- study is to investigate whether widely deployed authenti-
rupts. Thecontextswitchwasalsoresponsibleforreducing cation schemes can be modified to make them more usable
the overall task completion time of the participants. forre-authenticationscenarioswithoutsignificantlycompro-
mising on security.
Since unavoidable re-authentications are a potential issue
in the adoption of IA, we investigate whether the unpre-
dictable nature and the context-switch due to authentica- 3.2 ConfigurationParameters
tion interrupts can be reduced by modifying how a user is We introduce two configuration parameters for existing au-
re-authenticated. We assume that our concepts can miti- thentication prompts: time delay and screen transparency
gate these usability issues and thus reduce barriers to the and define the possible values for each of the parameter.
adoption of novel authentication schemes that require re- The time delay represents the time it takes between the
authentication. transition from the current task of the user to the appear-
ance of the re-authentication prompt. This variable sup-
3. STUDYDESIGN&OBJECTIVES ports two possible values: immediate lock (Imm-Lock) and
Inthissection,wefirstoutlinedifferentapproachesthatcan gradual lock (Grad-Lock). In the Imm-Lock case, the re-
be used for re-authentication. We then provide the ratio- authentication prompt appears immediately (without any
nale for our selection of a slightly modified version of the delay)whereasfortheGrad-Lockcase,there-authentication
existing authentication prompts through two configuration prompt appears after a predefined interval with a fade-in
parameters: time delay and screen transparency. Finally, effect. During this fade-in, the user can continue to inter-
we outline the security and usability trade-offs introduced act with the current task. The two possible values provide
by these parameters, our constructions of re-authentication different usability and security trade-offs: the secure Imm-
prompts with different configurations of these parameters Lock bars the user from interacting with the current task,
and the usability expectations from our constructions. while the less secure Grad-Lock is not abrupt and provides
the user with an opportunity to interact with the current
3.1 Re-authenticationApproaches task during the fade-in effect thereby potentially allowing
Several re-authentication schemes are possible. During the the user to reduce the effect of interruption. For example,
design phase, we considered the following: the user can finish reading a sentence.
Split-screenconfiguration: Inthisconfiguration,theau- Forourexperiments,wechoseafoursecondtimedelay. Our
thenticationpromptandthecurrentusertaskequallyshare selectionwasbasedontheresultsfrompreviousstudiesand
the screen space (screenshots are provided in Appendix B). our experiments with both shorter and longer delays. Fer-
This enables the user to authenticate within a timeout pe- reira et al.’s [11] study on understanding micro-usage pat-
riod with their task in sight. However, it is difficult to en- terns for various smartphone apps revealed that 40% of the
sure that the authentication prompt is displayed at a loca- applicationusagelastslessthan15secondsandissufficient
tionthattheuserisfocusingon. Incasetheauthentication for a user to read or reply to a message. In a study con-
prompt appears in the location where the user is focusing ducted by Yan et al. [38], they find that 50% of the smart-
on, it results in the aforementioned usability issues. Never- phone interactions last fewer than 30 seconds. With such
theless,thisapproachisworthexploringoncegazetracking briefperiodsofinteractions,itisthereforenecessarytolock
solutions for smartphones have matured [23, 26]. the device quickly to prevent any misuse. For the grace pe-
riod, we considered and tested delays between two to seven
Alternate authentication mechanisms: Alternate au-
seconds. During our empirical tests with four participants,
thenticationmechanismshavebeenproposedtocountersho-
we found that the four seconds delay period allowed the
ulder-surfingattacks,whichreducethesizeoftheauthenti-
participants to prepare for re-authentication prompts. The
cation prompt [20] or allow the user to enter the PIN using
shorter delay values did not provide the users with enough
simpleupanddowngestures[35]. Similartothesplit-screen
time to prepare for the re-authentication prompt, whereas
configuration, a challenge for these approaches is the iden-
the longer delay values made the users anxious in anticipa-
tification of the most suitable placement of the authentica-
tion of the re-authentication prompt.
tion prompt for re-authentication. Another option is to use
mechanisms that provide security using obscurity. For in- Thescreen transparency variableaffectsthevisibilityofthe
stance, De Luca et al. [7] have proposed a mechanism that currenttaskbyconfiguringthebackgroundofthere-authen-
allows users to enter the secret discretely through the back ticationprompttobeinstantaneouslydark(Imm-Dark,see
of the device. In another proposal, the user is expected to Figure 1a), gradually fade from transparent to dark (Grad-
USENIX Association 2016 Symposium on Usable Privacy and Security 223
Dark,seeFigure1b)orremaintransparent(Imm-Trans,see turns into dark. However, this configuration also al-
Figure 1c and 1d). Similar to the time delay variable, the lows the user to continue interacting with the current
three possible states of screen transparency provide vary- task for a grace-period of four seconds before the re-
ing degrees of security and usability. The Imm-Dark state authentication prompt appears. During the grace pe-
is the most secure one because it hides sensitive data dis- riod, the brightness of the current task is reduced to
playedinthecurrenttask;however,thecontext-switchover- indicate the forthcoming re-authentication prompt to
headshouldbethemostinthiscasesincetheuser’staskis the user. After the re-authentication prompt appears,
not visible anymore. The Imm-Trans state covers the other the users can no longer interact with their task.
extreme where sensitive data displayed in the current task
remains visible behind the re-authentication prompt; how- 3.4 StudyAims
ever, the context-switch overhead should be the least since Weexpectthefollowingpropertiesfromourre-authentication
the user’s task remains visible while the user is interacting prompt configurations:
with the re-authentication prompt. The Grad-Dark state
Imm-Dark-Imm-Lockisthemostobstructivetherefore
providesagraceperiodduringwhichtheusercanauthenti- •
it should be the most annoying. Furthermore, since it
catetoresumethetaskathand;however,iftheuserfailsto
providesnovisualcluesonthecurrenttaskoftheuser,
do so in a configurable amount of time, the background of
task efficiency should be reduced.
the re-authentication prompt becomes dark thereby hiding
the user’s current task. Imm-Trans-Imm-Lock also immediately locks out the
•
userbutitspresentationofthere-authenticationprompt
3.3 Re-AuthenticationPromptConfigurations is less intrusive and it provides visual clues on the
current task of the user. Therefore, it should be less
The four configurations of re-authentication prompts that annoying and more task efficient as compared to the
weconstructusingthedifferentmeaningfulcombinationsof Imm-Dark-Imm-Lock configuration.
the two configuration parameters are as follows: Grad-Dark-Imm-Lock has similar properties as Imm-
•
Trans-Imm-Lockbutitprovidesadditionalsecurityby
1. Immediate Dark, Immediate Lock (Imm-Dark-
making the current task of the user invisible after a
Imm-Lock): Weevaluatethedefaultlockschemeon
predefined time interval. Therefore, it should score
most Android smartphones to establish a baseline for
similar to Imm-Trans-Imm-Lock in terms of usability
when it is used for re-authentication. In this con-
with a relatively better security perception.
figuration the re-authentication prompt appears im-
mediately with a dark background, which completely Grad-Dark-Grad-Lockenablestheusertointeractwith
•
hides the content of the current task, and the user the current task for a grace period and this may in-
can no longer interact with the current task. The re- crease the task efficiency of the users. However, the
authentication prompt asks the user to enter a PIN user may not take advantage of the grace period and
or pattern-lock and the user is able to access the cur- instead wait for the re-authentication prompt to ap-
rent task again only after correctly answering the re- pear, which may increase the anxiety and annoyance
authentication prompt. This configuration was also of the user.
used in the earlier work by Khan et al. [19], as dis-
In the rest of this paper, we evaluate whether the four re-
cussed in 2.2.
authenticationpromptconfigurationsprovidetheaforemen-
§
2. ImmediateTransparent,ImmediateLock(Imm- tioned usability properties.
Trans-Imm-Lock): There-authenticationpromptap-
4. STUDYDESIGN
pears immediately in this configuration and the user
Inthissectionweoutlineourdesignofauserstudytoeval-
can no longer interact with the current task. How-
uate the four re-authentication prompt configurations. To
ever, the background of the re-authentication prompt
measure the properties of each configuration, we perform a
remainstransparent,whichallowsuserstoobservethe
lab-basedevaluationwhereparticipantsareinvitedtoexpe-
contents of their task.
rienceeachconfigurationbyperformingpredefinedsynthetic
3. Gradual Dark, Immediate Lock (Grad-Dark-
tasks. After the users experience these configurations, they
Imm-Lock): Inthisconfiguration,there-authenticat-
are asked to rate and provide qualitative feedback in terms
ion prompt appears immediately and the user can no
of usability, security perception and their willingness to use
longerinteractwiththecurrenttask. Furthermore,the
these configurations. In addition to the user feedback, we
backgroundofthere-authenticationpromptisinitially
measure the task efficiency, context switch overhead, and
transparent and the contents of the current task are
task error rate against each configuration. Our evaluation
visible. Then,thebackgroundofthere-authentication
andfeedbacksetuparedesignedtoelicittheefficacyofthese
prompt gradually fades into a dark screen and hides
configurations for re-authentication in different scenarios.
the contents of the current task from the user. If
OurstudywasreviewedandreceivedapprovalfromtheIRB
the user manages to authenticate before the screen
ofouruniversity. Wenowprovidedetailsofourstudydesign
has darkened completely, this configuration keeps the
in terms of experimental setup and our methodology.
user’s current task visible in the background.
4. GradualDark,GradualLock(Grad-Dark-Grad- 4.1 Apparatus
Lock): In terms of task visibility, this configuration Whileseveralusecasesexistforre-authentication(see 2.1),
§
is similar to the Grad-Dark-Imm-Lock configuration wechooseIAastherepresentativeusecaseinthisworkbe-
described above. That is, the background of the re- causeitwaseasiertoexplainandconductthantheotherre-
authenticationpromptisinitiallytransparentandthen authenticationcasesoutlinedinthepaper. OurchoiceofIA
224 2016 Symposium on Usable Privacy and Security USENIX Association
(a) Imm-Dark (b) Grad-Dark (c) Imm-Trans (d) Imm-Trans
Figure 1: The proposed configurations with varying values for screen transparency. Figures (a), (b) and (c) show the three
possiblevalueswhenapattern-lockbasedre-authenticationpromptisused. Figure(d)showsasamplevalueforaPIN-based
re-authenticationprompt. FortheGrad-Darkconfiguration,thebackgroundofthere-authenticationpromptgraduallyturns
from transparent into dark.
isalsomotivatedbythepriorworkofKhanetal.[19]inthe teraction of the users with the app can be measured, which
IAdomainthathighlightstheissueswithre-authentications enables us to compute several metrics in terms of context-
incaseoffalserejects. Toensurethateachparticipantexpe- switchoverheadanderrorsmadebytheusers. Fortheemail
riencesacertainnumberoffalserejects,weuseasimulated activity,sincetheemailscontainsensitivematerial,theusers
IA scheme, as was also done by Khan et al. In particular, performing the email activity should consider the security
our scheme simulates IA schemes based on a user’s touch implications of a re-authentication prompt configuration in
input or keystroke behaviour. addition to its usability aspects.
For our experiments, we select two widely used authentica- TheseactivitieswerebundledintwoseparateAndroidapps,
tionmechanismsonAndroid: a4-digitPINandtheAndroid which allowed users to perform tasks. We define a task as
pattern-lock(withthesameconstraintsonpossiblepatterns completingthetextentryortheemailactivityalongwitha
asinAndroid). Theuserinterfaceofbothschemeswassim- mid-task re-authentication of the user using either the PIN
ilar to the Android lock screens (see Figure 1). or the pattern-lock in one of the four configurations. For
thetextentrytask,theuserswereinterruptedatpredefined
Thefourre-authenticationpromptconfigurationsintroduced
intervals, which were triggered based on the key presses by
in 3.3 are evaluated using two synthetic activities — a
§ theusers. Thenumberofkeypressesrequiredtotriggerre-
text entry activity and an email activity (screenshots are
authenticationchanged acrossdifferenttextentryactivities
provided in Appendix A). We choose these activities since
for each user but it stayed constant across users for those
they represent common smartphone activities (i.e., reading
tasksforresultstobecomparable. Similartothetextentry
andcomposingemailsandtextmessagesorinteractingwith
task, the users were interrupted with a re-authentication
social media apps).
prompt after a predefined number of swipes for the email
task. Theappswereinstrumentedtogatherthetimestamps
Text entry activity: This activity displays a 12-
• ofevents,includinginputeventsbytheuserandthedisplay
digit number to the participants. It also contains a
and dismissal events of the re-authentication prompts. The
textboxandtheusersareaskedtoenterthedisplayed
appsalsocollectedtheerrorsmadebytheusersforthetext
numberinthetextboxusingthenumerickeyboardof
entry activity and during the re-authentication. We also
the device.
logged the user interactions, including the keystrokes and
Email activity: Intheemailactivity,usersareasked screen touch events, during the grace period for the Grad-
•
toreadanemailinanemailapp. Theuserinterfacefor Dark-Grad-Lock configuration. The data collected by the
theemailappdeveloped forthisactivitylookssimilar apps was instrumental in computing the task completion
to the Android Gmail app. Once a participant has rate,contextswitchoverheadandtheerrorrateagainsteach
read the email, they are asked to answer a multiple re-authentication prompt configuration.
choice question related to the email on a laptop. The
emails composed for this activity contained sensitive 4.2 EvaluationMethodology
data,whichemphasizedtheneedtoprotecttheemails We evaluate the four re-authentication prompt configura-
from adversaries (see Figure 10b for an example). tions using the text entry and email tasks. Each scheme
was evaluated in a round that consisted of four text en-
The design of the text entry activity ensures that the in- try tasks and two email tasks. Each user was subjected to
USENIX Association 2016 Symposium on Usable Privacy and Security 225
five rounds and in each round a different re-authentication leagues starts using their device when it is left unat-
prompt configuration was evaluated. For the first round, tended. For this scenario, the apps on the device may
the participants performed the tasks without any authenti- be used for a limited time by someone known by the
cation,whichallowedustoestablishabaseline. Thepartic- smartphone owner.
ipants were allowed to take a break between each task and
Home Scenario: In this scenario, we asked the par-
eachround. Theorderofthefourre-authenticationprompt •
ticipants to consider that their spouse accesses their
configurations was randomly chosen for the participants.
device while it is left unattended or when they are
The participants shortlisted for this study were invited for asleep. Thenumberofadversariesislimitedinthissce-
an hour long lab-based study. The participants were first nario as compared to the others and the users may or
asked to fill a demographic survey, which asked about their maynotwanttoprotecttheirdatafromtheirspouse.
age, gender, and current occupation. They were then asked
Aresearcherpresentedthescenariostotheparticipantsand
to fill a security preferences survey. In terms of security
was available during the interview to answer any questions
preferences, we asked the participants about their device
participants may have. Participants were given sufficient
locking habits, their preferred authentication scheme, and
timetoconsiderthepresentedscenarios. Foreachscenario,
the adversaries that they wanted protection against. These
theparticipantsweretoldthatthere-authenticationprompt
pre-study surveys are provided in Appendix D. After the
wouldgetactivatedincasethesystemnoticesanysuspicious
pre-study surveys, the participants were introduced to IA,
activity. Wealsoremindedthemoffalserejectsandthefact
thepossibilityoffalserejectsinIA,thetasksandappsused
that they may be subjected to re-authentication while they
duringthestudy,andthedifferentre-authenticationprompt
are using the device. In order to inquire about the security
configurations. Theparticipantswerealsotoldthatfalsere-
perceptionofanevaluatedre-authenticationpromptconfig-
jects were simulated for the purpose of this study. We gave
uration, the participants were told that for the purpose of
participantstheoptiontoselecttheirpreferredlockscheme
these scenarios, they should consider that only IA is pro-
(PIN or pattern-lock) and a corresponding secret for the
tectingtheirdevice. Sincedifferentusersmayhavedifferent
study. We did not assign participants a specific scheme to
security preferences for each configuration and each usage
avoid any bias due to their inexperience with it. This de-
scenario, we initially asked the users to establish the sen-
sign decision prohibited us to counterbalance the authen-
sitive nature of the apps and usage scenarios. Then the
tication methods. The authentication times varied across
participants were asked to provide feedback in terms of se-
participants. Tocaterforthis,wereportwithin-subjectrel-
curity perception, usability and preferred re-authentication
ativedifferencesinsteadofabsolutevalues. Theparticipants
promptconfigurationforeachofthefourappsundereachof
experienced the different configurations in multiple rounds.
thethreedeviceusagescenarios. Thefeedbackquestionnaire
Afterthecompletionofeachround,theywereaskedtorate
is provided in Appendix E.
theusabilityandperceivedsecurityoftheconfigurationthat
they experienced and to give an overall ranking in terms of Finally,attheendofthestudy,weconductedashortsemi-
their preferences by taking both the usability and the secu- structured interview (provided in Appendix F) to gain in-
rity of the evaluated configuration in account. Participants sight into participants’ overall impression of the configura-
were also asked to indicate their preferences for the eval- tions that they evaluated.
uated configurations under different device usage scenarios
and were subjected to a semi-structured interview to gain 5. RESULTS
furtherinsightintotheirfeedback. Aresearcherwaspresent The data collected through the user studies and the inter-
to respond to any questions the participants had. views were recorded and analyzed. The audio responses of
the participants were transcribed by one of the researchers.
4.3 UserFeedback
We report both the quantitative and the qualitative results
The evaluated schemes trade off security for usability and
fromthestudyinthissection. Forstatisticalsignificance,we
since different users have different security preferences for
usedpairedt-testswhencomparingcontinuousdataforthe
differentappsanddifferentscenarios,weseekfeedbackfrom
within-subjects condition such as the inter-stroke rate for
theusersagainstfourappsforthreedifferentscenarios. Pre-
each user between grace and non-grace periods. We used
vious studies have shown that users prefer a strict security
one-way ANOVA when comparing continuous data for the
setting for financial and email apps, which contain highly
within-subjects condition for the four authentication con-
sensitive data, whereas they prefer a relatively relaxed se-
figurations (e.g., context-switch overhead). We used chi-
curity setting for contacts and other utility apps [17]. We
squaredtestswhencomparingparticipants’responsestocat-
sought feedback from the users for four apps: a banking
egorical Likert-type questions.
app,anemailapp,aphotosapp,andacontactsapp. These
apps are commonly used and contain varying levels of sen-
5.1 StudyParticipants
sitive data of the smartphone user. The participants were
We advertised the study through our university-wide mail-
asked to consider the following device usage scenarios with
ing list and through the graduate student research portal
the aforementioned apps available on the device.
of our university. The study was advertised with the title
BusScenario: Theparticipantshadtoconsiderasit- “Evaluating authentication schemes for smartphones” and
• uation where they are traveling on a bus and they ac- werecruitedonlythoseuserswhohadpriorexperiencewith
cidentally leave their smartphone behind. A stranger using smartphones. Participants received $10 for their par-
picks up their device and starts using it. ticipation for an hour of study.
Office Scenario: This scenario asks the participants We recruited 30 participants for the study (see Table 1 for
•
toconsideraworkenvironmentwhereoneoftheircol- theirdemographics). Alltheparticipantswerestudentsfrom
226 2016 Symposium on Usable Privacy and Security USENIX Association
N=30
Gender 60% Females
40% Males
Age 33% Under 20 years
57% 21-25 years
7% 26-30 years
3% 31-35 years
Lock 26 (87%) Yes
device? 4 (13%) No
13/26 Pattern-lock
Authentication 5/26 PIN (4 digits)
Figure 2: Task completion overhead time for the text entry
scheme 6/26 Fingerprint
activityrelativetotheBASE ROUND(errorbarsrepresent
2/26 Password
95% confidence interval).
Protecting 25/26 Strangers
from? 16/26 Friends
14/26 Room-mate
pletion overhead is the additional time taken to complete a
14/26 Coworker
textentrytaskascomparedtotheBASE ROUNDinwhich
3/26 Spouse, own children
a user is not interrupted to re-authenticate. For the task
completionoverhead,weonlytakeintoaccountthetexten-
Table1: Demographicinformationandthedevicelockusage tryactivitysincetheemailsusedfortheemailactivitywere
pattern of the participants. ofadifferentnatureandlengthduringeachround. Ourgoal
is to find if there are any re-authentication prompt config-
urations that assist the users in completing their text entry
our university. The majority of our participants (87%) re- tasks faster.
ported that they locked their device. The security prefer-
Wefoundthatonaverageuserstook3-4secondslongerwhen
ences of participants who locked their devices are provided
they had to re-authenticate during a text entry task (see
inTable1. Weaskedthefourparticipantswhodidnotlock
Figure 2). A one-way between subjects ANOVA was con-
their devices for their reason to do so: two indicated that
ducted to compare the effect of the four configurations on
they had nothing to protect, two wanted their emergency
thetaskcompletionoverhead,whichindicatednosignificant
contacts to be available and one considered authentication
differences across the four configurations (F(3,116)=2.31,
to be inconvenient (multiple answers were possible).
p=0.08).
5.2 QuantitativeResults Discussion: OurexpectationthattheImm-Dark-Imm-Lock
Outof30participants,18participantschosetouseapattern- configurationislessefficientascomparedtothemodifiedre-
lockduringthestudy,whiletheremainingparticipantschose authenticationpromptconfigurationsturnsouttobeincor-
to use a PIN. Participants were subjected to five rounds in rect. Though, we did not find any significant differences in
total. During the first round, participants were not inter- theperformanceoftheconfigurations,theparticipantsmen-
rupted for re-authentication. This round was used to es- tionedduringthestudythattheyfeltthattheirperformance
tablish a baseline and we use the term BASE ROUND to wasaffectedduringtheImm-Dark-Imm-Lockconfiguration:
refer to it. For the remaining rounds, participants tested
one of the four configurations in each round. The order of
”Itkindoffreaksmeoutbecauseitistoosudden,
the configurations was random during the four rounds.
it slows down whatever I was doing.”(P4)
During each round, participants completed four text entry
tasks and two email tasks. They re-authenticated once for
every email and text entry task during all rounds except 5.2.2 Effectoncontextswitchoverhead
BASE ROUND. The high rate of re-authentication is not Context switch overhead for the text entry task is defined
representative of a real-world scenario; however, our moti- as the time taken by the users to resume their text entry
vation was to get participants acquainted with the configu- task once they have re-authenticated. The context switch
rationsandtocollectsufficientdatatoevaluatethemetrics overheadisrepresentedbythetimeintervalbetweenthedis-
used in this section. During the study each participant re- missalofthere-authenticationpromptandthefirstkeypress
authenticatedthemselves16timesduringthetextentryac- onthetextentrytaskoncethere-authenticationprompthas
tivity (four times per configuration) and eight times during disappeared. It was not possible to compute this metric for
the email activity (twice per configuration). In total, 120 the email task because after re-authenticating a user would
re-authentication events, 120 text entry tasks and 60 email complete reading the email text visible on the screen be-
tasks were logged per configuration by our apps. fore interacting with the device. Our expectation was that
a visual of the user task in the background would reduce
5.2.1 Effectontaskcompletionoverhead thecontextswitchoverhead. Toconfirmthis,weconducted
The task completion time is the time taken by the users a one-way between subjects ANOVA to compare the effect
to complete a text entry or an email task. It also includes of the four configurations on the context switch overhead.
the time taken by the users to re-authenticate themselves However,theresultsindicatenosignificantdifferencesacross
while evaluating one of the configurations. The task com- the four configurations (F(3,116)=1.15, p=0.33).
USENIX Association 2016 Symposium on Usable Privacy and Security 227
Discussion: Whilenostatisticallysignificantdifferenceswere
observed,duringtheinterviews,mostusersfoundtheImm-
Dark-Imm-Lock configuration to be abrupt and reported
thatitwasdifficulttoresumetheirtaskafterre-authentication:
”Ilostmyplace[context]onwhatIwasdoingbe-
fore[thelockappeared],soitismyleastfavourite.
It would be too frustrating for me for everyday
use, so I would rather take the risk.”(P9)
”You can’t prepare for what’s going to come. It
takes more time to pick up after unlock”(P10)
Figure3: Inter-keyintervalforthetextentryactivity(error
bars represent 95% confidence interval). The top bar rep-
5.2.3 Effectofgraceperiod
resents the inter-key interval for the Grad-Dark-Grad-Lock
We allowed a grace period of four seconds for the Grad-
configuration during the grace period.
Dark-Grad-Lockconfiguration. Duringthegraceperiodthe
participants could continue working on their task for four
seconds before getting locked out. We observe that all par-
ticipants took advantage of this grace period by continuing
their work during the text entry activity. The average task
completiontimefortheGrad-Dark-Grad-Lockconfiguration
was 13 seconds and we found that on average users entered
38% of the text during the four second grace period with
someusersenteringupto60%ofthetotaltextinthegrace
period. A similar trend was observed for the email task
where 23% of the swipe events occurred during this period
(averagetimetocompletetheemailtaskfortheGrad-Dark- Figure 4: User perceptions of the security of the four re-
Grad-Lock configuration was 41 seconds). authentication prompt configurations.
We find that the inter-key intervals (time interval between
two consecutive key presses) of the users reduced signifi-
5.3 QualitativeFeedback
cantly for the Grad-Dark-Grad-Lock configuration during
For the apps evaluated in this work, 100%, 73%, 60% and
the grace period. The average inter-key interval of users
30%oftheparticipantsconsideredthebanking,email,photo
reduced by almost 60% during the grace period when com-
andcontactsapptobesensitive,respectively. Theresponses
pared to the average inter-key interval during the task (see
to the pre-study question regarding the adversaries that
Figure 3). A paired t-test was conducted to compare the
the participants (who used protection) wanted protection
inter-key interval between the grace and non-grace period
againstindicatethatdifferentscenariosrequiredifferentlev-
for the same text entry activity for each user. The results
elsofprotection. Almostalluserswantedprotectionagainst
show that inter-key intervals are significantly different be-
strangers, which corresponds to the bus scenario. Corre-
tween the grace and non-grace period (t(29) = 2.1, p =
sponding to the office scenario, 54% of participants wanted
0.04).
protectionagainstco-workers. Ontheotherhandonly11%
Discussion: Our results indicate that participants took ad- ofparticipantsconsideredthattheyneededprotectionagainst
vantage of the grace period by attempting to quickly com- family members, which corresponds to the home scenario.
plete the text entry activity. They typed faster than their
Wenowpresentthefindingsfromthefeedbackofthepartic-
normal speeds during the grace period.
ipantsregardingtheusabilityandsecurityperceptionsofthe
configurations for each app in the different usage scenarios.
5.2.4 Effectontaskerrorrate
Incasetheinputoftheusersmismatchedthedisplayedtext 5.3.1 Securityperceptions
for the text entry task, we counted it as an error (with at
Figure 4 shows the security perceptions of the participants
most one error per task). Our results indicate that users
for each re-authentication configuration. Significantly more
made errors in 77 out of 600 text entry tasks. However, a
(57% more) participants thought that the Imm-Dark-Imm-
one-way between subjects ANOVA for the task error rate
Lock configuration wasmore secure than theother configu-
acrossthefourconfigurationsandBASE ROUNDindicates ration (χ2(3) = 151,p < 0.001). Imm-Dark-Imm-Lock im-
nosignificantdifferences(F(4,145)=1.51,p=0.2). Similarly,
mediately hides the content on the screen to prevent the
whileparticipantsmadeerrorsin43outof240emailtasks,
leakage of any sensitive information. Some participants in-
the differences were not significant across the different con-
dicated that they would take advantage of this increased
figurations (F(4,28)=0.28, p=0.84).
security at the cost of usability for some apps:
Discussion: The task error rate among the configurations
werecomparable. Thoughtheinter-keyintervaloftheusers ”If I am sending an important email, I do not
duringthegraceperiodreducedsignificantly,itdidnotaffect want anybody else to look at it even for a sec-
the task error rate compared to the other authentication ond. It is annoying but it would be the most
configurations. beneficial.”(P13)
228 2016 Symposium on Usable Privacy and Security USENIX Association
Figure5: Userpreferenceoftheconfigurationsforthebank-
Figure6: Userpreferencesfortheconfigurationsforthepho-
ing app in different scenarios. 1 represents the most pre-
tos app in different scenarios. Only users who consider the
ferred configuration while 5 represents the least preferred
photos app as sensitive are included (N=18). 1 represents
configuration(errorbarsrepresent95%confidenceinterval).
themostpreferredconfigurationwhile5representstheleast
preferredconfiguration(errorbarsrepresent95%confidence
interval).
This was followed by the Grad-Dark-Imm-Lock configura-
tion,whichwasconsideredtobesecureby33%ofthepartic-
ipants. Wefoundthatonly13%and7%oftheparticipants
the participants who considered the photos app to be sen-
consideredtheImm-Trans-Imm-LockandGrad-Dark-Grad-
sitive preferred the Imm-Dark-Imm-Lock configuration for
Lock configurations to be secure. As expected, the visible
thebusscenario(Figure6). Fortheofficescenario,thepar-
taskinthebackgroundisperceivednegativelybymostusers
ticipants who were very concerned about protecting their
in terms of security. The Grad-Dark-Grad-Lock configura-
photos preferred configurations that obscured or gradually
tionprovidesaccesstothe devicefora shortperiodof time
obscuredtheapp,preventingitfrombeingaccessedbytheir
and participants felt that their content was vulnerable dur-
co-workers:
ingthisperiod. Wenowexplorewhethertheconfigurations
thatwereperceivedtobelesssecurewereconsideredappro-
priate for some usage scenarios. “I won’t care about my photos with respect to a
strangerbutinofficewhereitsmoreprofessional
“Ilikedtheideathathowthelockappearsatthe environmentwiththepeopleIknow,Iwouldin-
start [during Grad-Dark-Imm-Lock], so if it is crease the security of the scheme.”(P12)
someone else, they can’t enter any text message
“Ihavealotofphotosthatareverypersonaland
and they can’t send anything compared to the
I don’t want them [strangers] to see any part of
last scheme [Grad-Dark-Grad-Lock] where they
them.”(P6)
can do anything if they are fast enough”(P4)
“Imighthavealreadysharedalotofphotoswith
mypartner,soIwouldpreferacomfortablelock
TheImm-Dark-Imm-Lockconfigurationwasperceivedmost
scheme.”(P6)
secure and all participants indicated that they would only
considerusingthisconfigurationfortheirbankingappona
bus and at the office (see Figure 5). On the other hand, for For the contacts app, the participants were willing to use
the home scenario, users had different preferences. 40% of configurations that provided device access for a period be-
theusersindicatedthattheywouldstillonlyconsiderusing forelockingthemout. Theywanteditsobecausethiswould
theImm-Dark-Imm-Lockconfigurationforthebankingapp allow a stranger to contact them in case they lost their de-
at home while 23% of the users indicated that they would vice. The participants were less concerned about securing
preferusingtheGrad-Dark-Imm-Lockconfigurationinstead. their contacts at home or office because they felt that they
Some of the user comments shed more light on the user shared contacts with individuals at these locations.
preferences for the banking app:
“If someone picked up my phone and they are
“Bankingwouldbeverysensitive,soIwantitto
looking at my contacts, they could try to re-
get dark as quickly as possible.”(P9)
turn it to me through someone in my contacts,
“Even with my partner, I won’t feel completely soIwouldchoosesomethingexcepttheonethat
securewithmybankingappopenedonmyphone turns dark immediately.”(P7)
thatiswhyIwouldpreferimmediatedark.”(P4)
“For contacts, now there is an issue of privacy
because these are people which they [office col-
The feedback from the users was inconclusive for the email leagues]mightalsoknow,soitisimportantthat
app and there is no one configuration that users signifi- Iprotecttheirinformationbutatthesametime
cantly prefer over the other for the different usage scenar- I don’t want it to be very inconvenient for me
ios. On the other hand, for the photos app, the majority of when I look at the contacts.”(P2)
USENIX Association 2016 Symposium on Usable Privacy and Security 229