PrefMiner: Mining User's Preferences for Intelligent Mobile Notification Management

12 Pages·2016·0.99 MB·English

PrefMiner: Mining User’s Preferences
for Intelligent Mobile Notification Management
AbhinavMehrotra RobertHendley MircoMusolesi
UniversityofBirmingham UniversityofBirmingham UniversityCollegeLondon
UniversityCollegeLondon UnitedKingdom UnitedKingdom
UnitedKingdom [email protected] [email protected]
[email protected]
ABSTRACT anyimportantinformation[23]. However,theirwillingnessis,
Mobile notifications are increasingly used by a variety of inasense,exploitedbymobileapplicationsasthesetrigger
applicationstoinformusersaboutevents,newsorjusttosend a plethora of notifications continuously [24]. Given the po-
alerts and reminders to them. However, many notifications tentiallylargenumberofnotifications,usersdonotacceptall
are neither useful nor relevant to users’ interests and, also ofthemastheirreceptivityreliesonthecontenttypeandthe
forthisreason,theyareconsidereddisruptiveandpotentially senderofthemessages[24,25]. Usersmostlydismiss(i.e.,
annoying. swipeawaywithoutclicking)notificationsthatarenotuseful
orrelevanttotheirinterests[19,33]. Someexamplesofsuch
Inthispaperwepresentthedesign,implementationandevalu-
notificationsarepromotionalemails,gameinvitesonsocial
ationofPrefMiner,anovelinterruptibilitymanagementsolu-
networksandpredictivesuggestionsbyapplications. Atthe
tionthatlearnsusers’preferencesforreceivingnotifications
same time, past studies have shown that users get annoyed
basedonautomaticextractionofrulesbyminingtheirinter-
byreceivingirrelevantorunwantednotificationswhichcould
actionwithmobilephones. Thegoalistobuildasystemthat
resultinuninstallingthecorrespondingapplication[17,33].
isintelligibleforusers, i.e., notjusta“black-box”solution.
Rulesareshowntouserswhomightdecidetoacceptordis- Theabovefindingsprovideevidencethat,inordertoreduce
cardthematrun-time. ThedesignofPrefMinerisbasedona the level of disruption, an interruptibility management sys-
largescalemobilenotificationdatasetanditseffectivenessis tem should not just try to deliver notifications at opportune
evaluatedbymeansofanin-the-wilddeployment. moments but also stop notifications that are not useful, or
are uninteresting or irrelevant for the user. However, most
AuthorKeywords ofthepreviousstudiesproposeinterruptibilitymanagement
Notifications;Interruptibility;Context-awareComputing. mechanismsthatleveragetheconceptofanticipatorycomput-
ing[29]topredictopportunemomentsbyusingcontext[21,
18,28,26]andcontent[24]. Inthiswork,forthefirsttime,we
ACMClassificationKeywords
designanintelligentinterruptibilitymanagementmechanism
H.1.2. ModelsandPrinciples: User/MachineSystems;H.5.2.
thatlearnsthetypesofinformationusersprefertoreceivevia
InformationInterfacesandPresentation(e.g. HCI):UserIn-
notificationsindifferentsituations. Anotherimportantaspect
terfaces
is usability: in order to tackle this problem we implement
amechanismforminingassociationrules[9]andmakethe
INTRODUCTION
discoveredrulestransparenttouserssothattheycancheck
Today’smobilephonesarehighlypersonaldevicescharacter-
theirappropriateness.
izedbyalways-onconnectivityandhigh-speeddataprocessing.
Theseaffordancesmakeitauniqueplatformforapplications Inordertotrainandevaluatetheproposedintelligentnotifi-
harnessingtheopportunityofreal-timeinformationdelivery. cationmechanism,wefirstexploitthedatasetsofreal-world
Avarietyofapplicationsareavailableontheappstoresthat mobilenotificationscollectedduringtheMyPhoneandMe
enableuserstosubscribetonumerousinformationchannels study [25]. We construct the association rules by using the
andactivelyreceiveinformationthroughnotifications[3,4]. combinationsofnotificationtitlesandcontextmodalitiesin-
cluding activity, time and location. Through an extensive
Previousstudieshaveshownthatusersarewillingtotolerate
evaluation,weshowthatbyusingnotificationtitlesandthe
someinterruptionsfromnotifications,sothattheydonotmiss
user’s location we can predict the notifications that will be
dismissedbytheuserwithaprecisiongreaterthan91%.More-
over,weshowthattheuser’sactivityandhourofthedaydo
Permissiontomakedigitalorhardcopiesofallorpartofthisworkforpersonalor
notcontributetotheimprovementofthepredictionaccuracy.
classroomuseisgrantedwithoutfeeprovidedthatcopiesarenotmadeordistributed
forprofitorcommercialadvantageandthatcopiesbearthisnoticeandthefullcitation
onthefirstpage.CopyrightsforcomponentsofthisworkownedbyothersthanACM Wethenfunneledourfindingsintoapracticalimplementation
mustbehonored.Abstractingwithcreditispermitted.Tocopyotherwise,orrepublish, ofaninterruptionmechanismformobiledevices–MyPref,
topostonserversortoredistributetolists,requirespriorspecificpermissionand/ora
anintelligentnotificationlibraryfortheAndroidOS.MyPref
[email protected].
UbiComp’16,September12-16,2016,Heidelberg,Germany enablesanapplicationtodiscoverpersonalizedrulesforthe
©2016ACM.ISBN978-1-4503-4461-6/16/09...$15.00
DOI:http://dx.doi.org/10.1145/2971648.2971747
user’spreferencesandpredicttheirreceptivitytonotifications withothers,forexamplefordeferringthenotificationsuntil
containingdifferenttypesofinformation. TheexpressiveAPI therighttime/context.
ofthelibraryallowsadevelopertoconfiguretheprediction
settingsincludingthefeaturesusedforminingrules,making
it also extensible and generalizable. Moreover, through an WhyMineAssociationRulesOverotherTypesofMachine
extensiveevaluationofmemory,batteryandtimeoverheads, LearningModels?
weshowthatMyPrefisalight-weightandresourceefficient Oneofthemajorissuesindesigninginterruptibilitymanage-
librarythatiscapableofmining1500notificationsamplesin mentsystemsisrelatedtotheirtestingandevaluation. Infact,
aminutebyconsumingonly11.737mAHbatterycharge. ifthenotificationmechanismsarenotcorrect,theymightlead
to rejecting or deferring important notifications. Therefore,
WepresentthedesignandimplementationofPrefMiner,an
inordertobuildusablesystems,wehavetoinvolveusersto
interruptibility management application that is built on top
adjusttheinterruptibilitymanagementmechanisms,inorder
oftheMyPreflibrary. Toachieveusers’trusttheapplication
toachievethegoalofreducinginterruptionswithoutcompro-
showsthediscoveredrulestothemandasksfortheirconsent
misingthereceptionofanyusefulandimportantinformation.
foreachruletoactivateit.
Theinterruptibilitymanagementmechanismsproposedinthe
Finally,unlikepreviousinterruptibilitystudies,wedonotre-
previousstudiesrelyonmachinelearningmodelsformaking
strictourselvestoevaluatingtheinterruptibilitymanagement
predictionsthatmightbedifficulttounderstandandtranslate
mechanismonthecollecteddataortomanageourowngener-
directlyintohuman-readablerules[21,28,24]. Thesemodels
atednotifications. Instead,inordertoevaluatePrefMiner,we
aregoodforlearningquicklywithahighaccuracybutitis
conductedanin-the-wild 15-dayexperiment. Weshowthat
nearlyimpossibleforuserstounderstandthesemodelsand
with16subjectsPrefMinersuggested179rulesand56.98%
providetheirfeedback. Therefore,werelyonminingassocia-
oftheseruleswereaccepted. Duringthestudyperiodtheapp
tionrulesthatcanbeeasilyunderstoodbyusersascompared
filterednotificationswitharecallof45.81%andaprecision
to other prediction models. This allows us to get feedback
of100%asallfilteringruleswereacceptedbytheusers.
fromusersabouttherulesthatshouldnotbeusedforstopping
notificationsontheirphones. Itisworthnotingthatasimple
MOTIVATIONOFKEYDESIGNCHOICES
machinelearningtechniquesuchasthedecisiontree[34]is
In this section we discuss the key motivations and design
likelytofindafewmorerulesthantheassociationrules,how-
choices of our solution, namely (i) the choice of learning
ever,thedecisiontree-basedrulesmostlyhavelowreliability
users’preferenceswithrespecttodifferenttypesofinforma-
becausetheyrefertoverysmallsetsofdatainstances[27].
tiondeliveredthroughnotificationsratherthanmodellingtheir
interruptibility;(ii)thechoiceofimplementinganalgorithm
basedonassociationrulesinsteadofalternativemachinelearn-
MININGUSERPREFERENCES
ingalgorithms.
In this section we discuss how we extract notification rules
byminingassociationrulesbasedondifferentcombinations
WhyMineUser’sPreferencesforDifferentTypesofInfor-
ofnotificationtitlesandcontextmodalitiesincludingactivity,
mation?
timeandlocation.
Thekeyobjectiveofaninterruptibilitymanagementsystem
istodelivertherightinformationattherighttime. Mostof
thepreviousinterruptibilitystudieshaveproposeddifferent RemovingReminderNotifications
approachestomodelinterruptibilityforpredictingopportune Thefirststepconsistsofidentifyingaparticularclassofnoti-
moments [21, 18, 28, 24]. However, these studies do not ficationsthatarealwaysdismissedbuttheyshouldbeshown
suggestwhattodowithnotificationsthatarriveatinopportune in any case to users. As discussed earlier, notifications are
moments. Shouldwedeferthem? Orshouldwecompletely dismissed if they are not found to be useful or relevant to
dismissthem? theuser’sinterest[19,33]. However,somenotificationsare
dismissedbecausetheydonotrequireanyfurtheractionfrom
Inapreviousstudy[13],Clarksuggestedthatusers’negative
theuser. Thesenotificationsshouldnotbeautomaticallyfil-
responsetoaninterruptioncanbeoftwotypes: (a)acknowl-
tered,sincetheymightberelevantforusers,eveniftheyare
edgeitandagreetohandleitlater; (b)declineit(explicitly
alwaysdismissed. Werefertosuchnotificationsasreminder
refusingtohandleit).BasedonClark’ssuggestionswehypoth-
notificationsintherestofthepaper. Alarm, calendarevent
esizethataninterruptibilitymanagementmechanismshould
andbatterystatusnotificationsaresomecommonexamplesof
takeanorthogonalbutequallyimportantapproachbylearn-
remindernotifications.
ing the different types of interruptions that users explicitly
refusebydismissingnotifications. Inthiswaysuchamecha- Moreformally,inordertodefineremindernotifications,we
nismcanidentifythenotificationsthatarenotusefulforthe introduceasimpledefinitionofclickrate(CR):
usersandstoptheoperatingsystemfromtriggeringalertsfor
these types of notification. Moreover, our hypothesis is in Numberofacceptednotifications
line with the findings of Fischer et al. [19, 33], i.e., users’ CR= ∗100 (1)
Totalnumberofnotifications
receptivity relies on the usefulness and their interest in the
deliveredinformation. Wewouldliketostressagainthatthis Ifanapplication’sclickrate(CR)iszerothenallnotifications
isanorthogonalmechanismthatcanbeusedinconjunction fromthatapplicationaretreatedasremindernotifications.
NotificationClassification (iii) Removalofstopwords: stopwordsarethecommonwords
Inordertomodelusers’preferenceswithrespecttoreceiving (suchas‘a’,‘the’,‘is’and‘are’)thatgenerallyhavequite
different types of information, we categorize each notifica- highfrequencyinthetext. Weremovethemtoensurethat
tionbasedontheinformationcontainedinit. Arecentstudy theydonotaffectcontent-bearingkeywordsintheclustering
proposesanapproachformodellinginterruptibilitybyusing algorithm[12].
informationtype,socialcircleandcontextinformation[24]. (iv) Removalofthesenderandapplicationnames: weremove
Theauthorsof[24]assumethatallnotificationstriggeredby
senderandapplicationnamesbecausetheycanleadtheclas-
an application are of the same type and categorize them by
sifiertoclusternotificationsbasedonthesenderorapplica-
using the type of application that triggered them. In other
tionnames. Therefore,foreachnotificationweremove(if
words,theyclassifynotificationsatanabstractlevel,e.g.,chat,
present)thenameoftheapplicationbywhichitistriggered.
email,systemsandsoon.
Moreover, to remove sender names we find and remove
Instead,wedonotlimitourselvestocategorizeanotification allthewordsthatarenotpresentintheenglishdictionary
basedonthetypeofapplicationthattriggereditbecausean thatcomeswiththeqdapDictionariespackage[5]. Itis
application can generate notifications that contain different worthnotingthatinthecaseofchatandemailnotifications
types of information. A user might be interested to receive thatcontainonlynames,wedonotremovenamesandthus
somebutnotalltypesofnotificationstriggeredbyaspecific allowthealgorithmtoclassifysuchnotificationsbasedonly
application. Forexample,aFacebooknotificationaboutanew onthesendernames.
postonauser’stimelinemightnotbeconsideredasdisruptive (v) Stemmingofwords: itisastandardizationmethodtoavoid
as a game invite notification. In other words, users would having multiple versions of words referring to the same
notwantaninterruptibilitymanagementsystemtocompletely conceptbyreducingaworddowntoitsroot. Forinstance,
stopFacebooknotifications,butonlythosethatareannoying, thewords‘comment’,‘commented’,‘comments’and‘com-
suchasgameinvites. menting’areallstemmedto‘comment’).
Therefore,inordertoclassifynotifications,weperformclus- ConstructingaClassifier
teringbyconsideringtheirtitlesbymeansofDBSCAN[16], Inordertotraintheclassifier,weuseabagofwordsapproach
adensitybasedalgorithm1. Anotificationtitleisashortsen- andcreateaDocument-TermMatrix(DTM)–amatrixthat
tencethatgivesaglimpseoftheinformationcontainedinit. describesthefrequencyofterms(i.e.,words)thatoccurina
Thefollowingaresomeexamplesofnotificationtitles: “Sign collectionofdocuments(i.e.,notificationtitlesinourcase). In
in to a Wi-Fi network”, “Time to Work”, “Today is Alice’s aDTM,rowscorrespondtodocumentsinthecollectionand
birthday”.Itisworthnotingthatinsomecasesthenotification eachtermisassociatedtoacolumn.
titlecontainsthesendernamealongwithothertext(suchas
“Alicecommentedonyourpost”)ormerelythesendername Topreventtheproblemofoverfittingtheclassifier[35],we
(suchas“Alice”). Generally,thesendernameisattachedto computethetermfrequency(TF)andremovethetermsthat
thetitlesofnotificationstriggeredbychatandonlinesocial haveaTF lowerthanTFthreshold. TheTF andTFthreshold are
networkingapplications. definedasfollowing:
Theclusteringofnotificationsiscarriedoutthroughthefol- Numberofnotificationsinwhichthewordoccurs
lowing steps: i) cleaning notification titles; ii) constructing TF=
Totalnumberofnotifications
aclassifier; andiii)clusteringnotifications. Wediscussthe (2)
detailsofeachstepindetailbelow.
Numberofparticipationdays
TF = (3)
CleaningNotificationTitles threshold N*Totalnumberofnotifications
Notificationtitlesareshortsentencesphrasedin awaythat
theyareeasilyunderstoodbyusers. Themostimportantstep According to the above equation the value of TFthreshold en-
for analyzing these titles is to first clean them in order to suresthatatleastonenotificationcontainingthetermistrig-
removethenon-informativedata. geredinN days.
Toanalyzethenotificationtitleswefollowthestandardprocess Finally,aDBSCAN-basedclassifierisconstructedbyusing
ofcleaningtextinthefollowingway: theaboveDTM.Toclusterthedatatheclassifierrequirestwo
parametersasinputs: (i)MP,definedastheminimumnumber
(i) Conversionofthethetexttolower-case: thisensuresthat ofpointsrequiredtoformadenseregionand(ii)ε,definedas
thelower-caseandtheupper-caseversionsofthesameword themaximumdifference(i.e.,numberofnon-matchingwords)
areconsideredthesame. betweenthenotificationtitlesofacluster.
(ii) Removalofpunctuationandnumbers:theseelementsofthe
ClusteringNotifications
textdonotcontainusefulinformationfortheclassification
Since the notification titles are relatively short-length sen-
task,butatthesametimetheymightinfluenceit.
tences, it is possible that notifications from different appli-
cations contain similar words that can lead the classifier to
1Weareawarethatnotificationclusteringcouldbeimprovedbyusing
cluster them together. In order to prevent this problem, for
theentirenotificationcontent(i.e.,includingalsotheheaderofnotifi-
cations).However,duetoprivacyconcernsthenotificationheaders eachapplicationwecreateaseparateclassifierbyusingthe
werenotrecordedinthedatasetweusedforourevaluation[25]. notifications generated by that application. For example, if
there are 15 notifications from 3 applications, we create 3
clustering models and each model is trained using notifica- Filtered Remaining
tionsofseparateapplications. Finally,whentheclassifiersare
constructed,wepredicttheclassesofallnotifications.
3000
nt
u
ConstructingAssociationRules Co2000
Inordertodiscoverrulesabouttheuser’spreferencesforre- ms
ceivingnotifications,weusetheAISalgorithm[9]–amethod Ter1000
forminingdatatodiscoverstatisticalrelationshipsbetween
0
variables. Thealgorithmscansthedatatofindthefrequent
1 2 3 4 5 6 7
itemsetsandcomputestheirsupportvalue(discussedlaterin N−value
thissection). Finally,itfiltersoutthelistofitemsetswhose Figure1. Filteredandremainingtermsfornotificationclusteringwith
supportvalueisgreaterthanthegivensupportthreshold. differentvaluesofNinTFthreshold.
AnassociationruleisrepresentedasX →Y,whereX isde-
finedastheantecedentandY astheconsequent.Thealgorithm DatasetandEvaluationSettings
generatesruleswiththeconsequentcontainingonlyoneitem. In order to evaluate the proposed solution, we use the data
ThismeansthatrulescanbeintheformofX1∪X2→Y but collectedbymeansofsmartphonesduringtheMyPhoneand
notintheformofX →Y1∪Y2. Mestudy[25]. Fromtheoriginaldataset, wetookasubset
of data considering only the users who participated for at
To better understand the concept of association rules let us
least14days. Wealsoremovedthenotificationswhichwere
consideranexamplewheretheuser:(i)alwaysdismissTwitter
notmappedwiththeuser’sactivity,locationandarrivaltime.
notificationsforwhotofollow;(ii)acceptsFacebookbirthday
Moreover,thedatasetcontainsanattributetoidentifywhether
reminder notifications only in the morning while she is at
anotificationisclicked,dismissedorhandledonanotherde-
home;(iii)doesnotacceptWhatsAppnotificationsfromAlice
vice. Wedidnotconsidernotificationsthatwerehandledon
whileatwork. AssumingthatnotificationsabouttheTwitter
another device. Consequently, the final dataset used in our
suggestion,FacebookbirthdayreminderandWhatsAppfrom
analysiscomprises11,185notificationsfrom18users.
AliceareclassifiedintheclassesN ,N andN respectively,
1 2 3
the following association rules would represent the user’s Weevaluatethediscoveredrulesforpredictingusers’response
preferencesinthiscase: to notifications by using a k-fold cross validation approach
{N1}→{Dismiss} withthevalueofkas10. Thepredictionresultsarecomputed
{N2,Home,Morning}→{Accept} separatelyforeachuserandweaggregatethembycomputing
{N2,Home,Afternoon}→{Dismiss} themeanandthestandard-errorwith95%confidencelimits.
{N2,Home,Evening}→{Dismiss} Wedividethesetofnotificationsbelongingtoeachuserintoa
{N2,Home,Night}→{Dismiss} trainingsetandatestset,wherethetrainingsetcontains90%
{N2,Work}→{Dismiss} ofthedataandtherestisconsideredasthetestset.
{N ,Other}→{Dismiss}
2
{N ,Home}→{Accept}
3 DefiningConfigurations
{N ,Other}→{Accept}
3 Asdiscussedearlier,weusetheTF topreventtheprob-
threshold
{N ,Work}→{Dismiss}
3 lemofoverfittingtheDBSCAN-basedclassifierwithsparse
ForanassociationruleX →Y,wedefinethetwoparameters: (i.e.,infrequent)terms. Inordertofindanoptimalvalueof
TF wecreateaDTMforallnotificationtitlesinthe
Threshold
• Support: theratiobetweenthenumberoftimesX andY datasetandcomputethenumberoftermsfilteredbysetting
co-occurandthenumberofdata-instancespresentinthe TF with different values of N ∈[1,7]. Here, we do
Threshold
givendata. Itcanberepresentedasthejointprobabilityof notconsidervaluesofN greaterthan7becausewebelieveit
X andY: P(X,Y); wouldnotbeusefultoincludenotificationtermsthatdonot
arriveatleastonceinaweek.
• Confidence: theratiobetweenthenumberoftimesY co-
occurswithX andthenumberoftimesX occursinthegiven As shown in Figure 1 there are 3642 unique terms in the
data. Itcanberepresentedastheconditionalprobabilityof dataset. BysettingN=1,therearechancesofunderfittingthe
X andY: P(Y |X). modelbecause3371termsareremovedandonly271terms
remainintheDTM.Sincethereisnotmuchdifferenceinthe
An association rule is created only when it has at least the
numberoffilteredtermswithN∈[2,7],weuseN=2forour
minimumsupport(S )andconfidence(C ).Itisworthnot-
min min analysis,whichalsoensuresthateachtermisseenbytheuser
ingthatdecreasingthevaluesofeithersupportorconfidence
atleastonceintwodays.
couldresultindiscoveringmorerules[9].
Inordertosatisfytheaboveconstraint,foreachuserwesetthe
EVALUATIONOFTHERULE-BASEDMECHANISM valueofMPtoD/2(Dindicatesthenumberofparticipation
Inthissectionwediscusstheimplementationandevaluationof days). This ensuresthatthere areat least D/2 notifications
themechanismforminingindividual-basedassociationrules ineachcluster. Inotherwords,notificationsofeachcluster
aboutusers’preferences. arrive at least once in two days. Similarly, for discovering
100 100 100
90 90 90
80 80 80
%)70 %)70 %)70
Result (34560000 PRreeccaisllion Result (34560000 PRreeccaisllion Result (34560000 PRreeccaisllion
20 20 20
10 10 10
0 0 0
70 80 90 70 80 90 70 80 90
Confidence Value (%) Confidence Value (%) Confidence Value (%)
(a) (b) (c)
100 100
90 90
80 80
%)70 %)70
Result (34560000 PRreeccaisllion Result (34560000 PRreeccaisllion
20 20
10 10
0 0
70 80 90 70 80 90
Confidence Value (%) Confidence Value (%)
(d) (e)
Figure2.Predictionresultsforassociationrulesdiscoveredbyusingnotificationresponsealongwith:(a)AR1:notificationtype;(b)AR2:notification
typeandactivity;(c)AR3:notificationtypeandarrivaltime;(d)AR4:notificationtypeandlocation;(e)AR5:notificationtype,activity,arrivaltime
andlocation.
association rules we set S (i.e., minimum support value) 3. AR3: byusingnotificationresponsewithnotificationtype
min
equaltoD /(2∗N ),whichensuresthatthenotifi- andarrivaltime;
participate count
cationinteractionpatterncoveredineachrulehasoccurredat
4. AR4: byusingnotificationresponsewithnotificationtype
leastonceintwodays. Here,D indicatesthenumber
participate andlocation;
ofparticipationdaysoftheuserandN referstothetotal
count
numberofnotificationscollectedforthatuser. Moreover,we 5. AR5: byusingnotificationresponsewithnotificationtype,
setε to1sothateachnotificationtitlewillhaveatleastN−1 activity,arrivaltimeandlocation;
(N referstothenumberofwordsinatitle)wordssimilarto
Forallapproacheswerestricttheconsequenttocontainonly
othernotificationtitlesinitscluster.
the notification response and the antecedent is restricted to
FeatureSelection nevercontainthenotificationresponse. Weintroducethiscon-
Inordertodiscoverassociationrulesabouttheuser’sprefer- straintbecauseweareonlyinterestedtopredicttheacceptance
enceswerelyonthefollowingfeatures: ofanotification,thereforeotheritemsintheconsequentwould
beofnouseandjustaddextracomputationalload.
(i) notification response: the user’s response (i.e., click or
dismiss)toanotification;
PredictionResults
(ii) notificationtype: theidentifieroftheclustertowhichthe
Inthissectionwepresenttheaccuracyoftheassociationrules
notificationbelongs;
discoveredwithdifferentvaluesofC forallfiveapproach.
min
(iii) arrivaltime: thearrivaltimeofthenotificationconsidering Inordertoassessthediscoveredassociationrules,wecom-
fourtimeslots–morning(6-12),afternoon(12-16),evening parethepredictedresponsewiththeactualresponse(i.e.,the
(16-20)andnight(20-24and0-6); groundtruth)andcomputetheaccuracyintermsof:
(iv) activity: the user’s physical activity (includes still, walk-
• Recall: ratiobetweenthenumberofnotificationsthatare
ing,running,bikingandinvehicle)whenthenotification
correctly predicted as dismissed and the total number of
arrived;
notificationsthatareactuallydismissed.
(v) location: theuser’slocationwhenthenotificationarrived;
Notethatwedonotincludethealertmodalityofnotifications • Precision:ratiobetweenthenumberofnotificationsthatare
forminingrulesbecauseweareinterestedinfindingtheinfor- correctlypredictedasdismissedandthetotalnumberofno-
mationthatusersprefertoreceivevianotificationsindifferent tificationsthatarepredicted(bothcorrectlyandincorrectly)
situationsirrespectiveoftheiralertmodalities. asdismissed.
Byusingdifferentcombinationsofthesefeaturesweconstruct InFigure2wepresentthepredictionresultsfortheassocia-
theassociationrulesaccordingtothefiveapproaches: tionrulesconstructedbyusingallfiveapproaches. Theresults
showthatincreasingtheconfidenceofassociationrulesde-
1. AR1: byusingnotificationresponsewithnotificationtype; creasestherecallbutimprovestheprecision. Thisimpliesthat
2. AR2: byusingnotificationresponsewithnotificationtype byincreasingtheconfidenceafewerbutmorereliablerules
andactivity; arediscovered.
100 l l l Confidence l l l l l l l l l l l
40 l 65% l
l
Precision (%) 90 Clonf67788i50505d%%%%%encel l l l l Recall (%)2300 778899050505%%%%%% l l l l l
l 10
90% l
95% l l l l l
l l l l l l l 0 l l
1 3 5 7 9 11 13 15 17 19 21 1 3 5 7 9 11 13 15 17 19 21
Day Day
Figure3.Predictionresultsforassociationrules(withnotificationtitleandlocation)usingtheonlinelearningapproach.
TheassociationrulesconstructedwithapproachesAR1and usefulnotifications. Therefore,whiledesigningitweshould
AR2donotshowasignificantdifference. TherecallofAR2 aimtohavefewerfalse-negatives(i.e.,incorrectlypredicting
isslightlyhigherwhentheconfidenceisbelow70%,butthe a notification as non-interesting for the user) that could be
precisiondropsaswell,whichmeanssomeextrabutunreliable achievedbyensuringthattheprecisionremainscloseto100%.
rulesarediscovered. Ontheotherhand,theassociationrules However, we could consider a precision of around 90% as
constructedwithAR3achievesbetterrecallthanAR1andAR2 acceptablebecauseitmightbepossiblethatacoupleofnotifi-
butitsprecisionconsistentlyremainslower(i.e.,under85%) cationshavebeenclickedbymistakeortokilltimewhenthe
ascomparedtootherapproaches. Thisimpliesthatthereisno userwasbored.
significantcontributionofactivityandarrivaltimeintermsof
Atthesametime,theinterruptibilitymanagementmechanism
predictinguser’spreferencesforreceivingnotifications.
shouldalsoachieveasignificantvalueforrecallinorderto
The approach AR4 (i.e., the association rules that are con- proveitsefficacyinfilteringnotificationsthatarenotusefulor
structedbyusingthenotificationresponse,typeandlocation) relevanttotheuser’sinterest.Wecouldnotobtainahighrecall
performsbetterthatotherapproachesintermsofbothrecall valuebecausenotalldismissednotificationsarenon-useful.
andprecision. Itsrecallgoesuptoaround43%withoutdrop- Instead,somenotificationsaredismissedbecausetheydonot
pingtheprecisionbelow79%. Evenbycombiningallfeatures requireanyfurtheractions. Thoughwehavealreadyfiltered
togetherintheapproachAR5,theresultsdonotimprove. As outremindernotificationsbeforetheanalysis,therecouldbe
compared to approach AR4, there is a negligible increment notificationsfromotherapplicationswhichareuseful,butdo
intherecalloftheapproachAR5withabigdropinitspre- not require any action, such as a chat message “See you at
cision. Consequently, our results provide evidence that the 7pm”andaconfirmationemail“Ok,bye!”.
user’spreferenceforreceivingnotificationsdoesnotdepend
Weobservethat,byusingtheconfidenceof70%,theapproach
ontheactivityandarrivaltime,butonthetypeofinformation
AR4isabletoachievearound42%recallwithaprecisionof
itcontainsandthelocationoftheuser.
84%. However,thedropof16%precisionvaluebringsarisk
It is worth noting that the high standard-error in the results thatthemechanismcouldpredictsomefalse-negatives. So,
demonstratesthatourmechanismdoesnotworkconsistently weshouldcompromisebyhavingaround35%recallwitha
forallusers. Itisduetothefactthatusershavedifferentpref- precision of more than 90% at a confidence of 80%. Here,
erencesforreceivingnotifications. Moreover,theupper-limit even the 35% recall implies that we are able to filter a big
ofthestandard-errorforprecisioncloseto100%demonstrate portionofnonusefulnotificationsandthusreducedisruption.
thatforsomeusersourmechanismisabletoveryaccurately
predictnotificationsthatarenotinterestingorusefulforthem. ONLINELEARNING
Intheprevioussectionweevaluatedourmechanismbyusing
Anotherinterestingobservationisthattherecallofapproach
thebatchlearningmethodinwhichtheassociationrulesare
AR3isalwayshigherthanthatofAR1andAR2,butitspre-
minedbyusingstaticdata. However,inarealworldscenario
cision is instead always lower. We believe that AR3 attains
suchtrainingdataisnotinitiallyavailableandtherefore,asso-
higherrecallbecauseusersfollowcircadianrhythmsanddur-
ciationrulescannotbediscovereduntilasufficientamountof
ingdifferenttimeperiodsofadaytheyarepresentatcertain
datahasbeencollected.
locations(e.g.,atworkintheafternoonandathomeduring
night)[15]. Therefore,inapproachAR3rulesareessentially Inordertoaddressthisissue,wecanuseanonlinelearning
associatedtotheirbehavioratcertainlocationsratherthanthe methodinwhichtheassociationrulesareperiodicallymined
timeofnotifications’arrival.Moreover,theprecisionmightbe fromthedatathatiscollectedgradually. Evenifthissolution
affectedasusers’mobilitypatternmightvaryonsomedays. does not remove the problem of initial bootstrapping com-
pletely,thepredictionaccuracyimprovesgraduallyasmore
OptimizingtheSystemforHighPrecision andmoretrainingdatabecomesavailable. Moreover,byusing
Thekeyrequirementfordeployinganinterruptibilitymanage- anonlinelearningmethodasystemcanadaptitselfaccording
mentmechanismin-the-wildisthatitshouldneverstop/defer tothepotentialchangesinuser’sbehaviorovertime.
Inordertoevaluatethismethod,weiterativelyconstructas- locally,thelibraryalsopreservesuser’sprivacysincenodata
sociationruleswithallthenotificationscollectedbytheend istransmittedtoaback-endserver. Itoffersflexibilitytode-
ofeachdayandevaluatetheserulesbyusingnotificationsof velopers by allowing them to define notification clustering
thefollowingday. Forexample,ondayN amodelisbuiltby configurations(i.e.,TF ,ε andMP)aswellastherule
Threshold
usingnotificationsfromday1toN−1. Moreover,similarto miningconfigurations(i.e.,support,confidenceandfeatures
theevaluationbyusingbatchlearningmethodweconfigure tobeusedforminingrules).
S as(N−1)/(2∗N ), whereN indicatesthetotal
min count count Finally,thelibrarylearnstheuser’spreferencesforreceiving
numberofnotificationscollecteduntilldayN−1. Thisensues
notifications and returns the discovered rules as output. In
thatthenotificationinteractionpatterncoveredineachrule
ordertoenableanoverlyingapplicationtofacilitatethetrans-
hasoccurredatleastonceintwodaysoratleast(N−1)/2
parencyofthepredictionmechanismtotheusers,thelibrary
timesinN−1days.
makestheruleshumanunderstandablebyreplacingeachno-
Figure3showsthepredictionresultsfortheindividualized tificationtypewiththemostfrequent3 wordsoftherelevant
rulesiterativelyconstructedoneachdaywithdifferentvalues notificationcluster(werefertothesewordsaskeywordsinthe
ofC byusingtheonlinelearningmethod. RuleswithC restofthepaper).
min min
as80andbelowstartfilteringnotificationsfromthe3rd day
For instance, let us consider some examples of hypothetic
and by the 7th day they achieve the recall of 25-35% and
keywordsinnotificationclusters: thekeywordsfromtheclus-
precision around 90%. Instead, by the 7th day other rules
ter of Facebook’s birthday reminder notifications (such as
could achieve the recall of 10% and precision above 90%.
“TodayisAlice’sbirthday.” and“AliceandChrishavebirth-
Interestingly, rules with C as 95% never become stable.
min daystoday. Helpthemhaveagreatday!”) wouldbe“today”
Thiscanbeduetothefactthattheserulesfilternotifications
and“birthday”. ThekeywordfromtheclusterofGooglePlay
withhighreliabilitywhichcanbeconfirmedbytheirprecision
Store’sappupdatenotifications(suchas“2applicationsup-
always remaining over 90%. On the other hand, rules with
dated.” and“3updatesavailable.”) wouldbe“update”. The
C as65-70%and80-90%becomestablebythe12th and
min keywordsfromtheclusterofsystems’sWiFiavailabilitynoti-
9thdayrespectively.
fications(suchas“Wi-Finetworksavailable”and“Verizon
Itisworthnotingthatgiventhedurationofthedataset,itisnot Wi-Fiavailable”)wouldbe“Wi-Fi”and“available”.
possibletoevaluatetheadaptationofthealgorithmandcon-
Theoverlyingapplicationcanpresenttheserulestousersin
firmiftherulesmaintaintheirstabilityovertime. Itmightbe
ordertoaskfortheirconsentandusetherulesthatareaccepted
possiblethatusersdivergefromtheirnotificationinteraction
by them to filter notifications. Later in the section, we will
behaviourandthesystemneedstoadaptaccordingly. There-
show a potential approach to take the consent of users for
fore,weenvisagethatthesystemshoulduseaslidingwindow
usingthepredictedrulesinanapplication.
approach, i.e.,thealgorithmshouldbetrainedonthelastL
daysinordertobeabletoadapttochangesinuser’sbehavior. Moreover,producingruleswithkeywordsinsteadofcluster
identifierswouldreducethecomputationtimeatthearrival
MYPREFLIBRARY ofnotifications. Thismightbefineforapplicationsthatare
predictingtheacceptancefortheirownnotifications.However,
Overview
if an application is managing notifications from third-party
Starting from the mechanisms and evaluation we presented
applicationthenthepredictionprocessshouldbeveryquick
intheprevioussection,weimplementedtheMyPreflibrary
sothat,ifrequired,itcancancelanotificationbeforeitalerts
– anintelligent interruptibilitymanagement librarythat can
theuser(viasound,vibrationorLEDlight).
predict the type of notifications that users would prefer to
receiveontheirmobilephonesinspecificcontexts.Thelibrary
learns the user’s preferences for receiving notifications by EvaluationoftheMyPrefLibrary
miningassociationruleswiththenotificationandcontextdata In this section we quantify the performance of the MyPref
thatissuppliedtoitandpredictstheuser’sreceptivitytothe library. We use a Nexus 6 phone with 3 GB of RAM and
subsequentnotifications. TheMyPreflibraryisimplemented a quad-core 2.7GHz Krait 450 CPU, running a clean slate
fortheAndroidOSandreleasedasanopensourceproject2. Android5.0OSfortheperformanceevaluationofthelibrary.
Thegoalistoprovidedeveloperswithapracticalgenerictool
forintelligentrule-basednotificationsthatcanbeintegratedin SourceCodeandMemoryFootprint
anyapplication,hidingatthesametimethecomplexityrelated MyPrefisimplementedasalight-weightAndroidlibrarycon-
tothepredictionmechanisms. sistingof15Javaclassesbuiltwith4077linesofcode. We
developedastubapplicationontopofthelibrarytoevaluate
The MyPref library abstracts the functionalities of the pro-
thememoryfootprintthataccountsto7.559MB(including
posed interruptibility mechanism through a set of intuitive
WekaandSnowballstemminglibraries). Weusedthird-party
APIprimitives. Theabstractionsincludeclusteringofnotifica-
measurementtools,namelyCountLinesofCode[2]andAn-
tions,miningassociationrulesandpredictingtheacceptance
droidDalvikDebugMonitorServer[1],toevaluatethesource
ofnotifications. ThelibraryreliesonWekaforAndroid[7]
codeandmemoryfootprintofthelibrary.
andtheSnowballstemminglibrary[6]forclusteringnotifica-
tionslocallyonthephone.Sincethecomputationisperformed
3Weconsiderthemostfrequentafterremovingallthestopwords
2https://github.com/AbhinavMehrotra/PrefMiner andstemmingtheremainingwords.
H)
A
m 60
on (11.5
ettery Charge Consumpti1101..50 l l l l l l l l l l l l l llAAAAARRRRR12345l Time (seconds)2400 l l l l l l l l l l l l l llAAAAARRRRR12345l
B 100 300 500 700 900 1100 1300 1500 100 300 500 700 900 1100 1300 1500
Notification Count Notification Count
Figure4. Batterychargeconsumedforminingrulesusingdifferentap- Figure5.Timetakenforminingrulesusingdifferentapproacheswitha
proacheswithdifferentnumberofnotificationsasaninput. varyingnumberofnotificationsininput.
EnergyConsumption
ThePrefMinerapplicationisbuiltontopoftheMyPreflibrary
Afundamentalaspectindesigningthisclassoflibrariesisthe
discussedandevaluatedintheprevioussectionandusesthe
energyusageanditsimpactonthephone’sbatterylife. We
notificationtypeandlocationdata(i.e.,theapproachAR4as
analyzetheenergyconsumptionofthelibrarybyvaryingthe
discussedabove)forminingtheassociationrules. Theappli-
amountofdataintheinput.Wecharacterizethebatterycharge
cation continuously collects the notification data and binds
consumptionforminingassociationrulesforallapproaches
theuser’scurrentlocationtoeachdatainstance. Therulesare
(AR1,AR2,AR3,AR4andAR5). WeusePower-Tutor[36]for
constructed every day when the phone is in charging mode
takingthebatterymeasurements.
andnotinusesothattheapplicationdoesnotdirectlyaffect
AsshowninFigure4,thebatteryconsumptionincreasesal- users’mobileexperience.
mostlinearlyastheamountofdatausedforminingtherules
As shown in Figure 6.a the newly discovered rules are pre-
increases. However,asthenumberoffeaturesincreasesthe
sented to the users in a human-readable format to get their
energyconsumptionincreasesbutnotdramatically. Thisim-
consent. Toconvertaruleintoahuman-readableformat,we
pliesthatmostoftheenergyisconsumedforclusteringthe
usetheapplicationname,thenotificationclusteridentifier(i.e.,
notificationtypesandlessenergyisrequiredforminingthe
thekeywordsprovidedbythelibraryasareplacementforthe
rules.
notification type) and location. Some possible examples of
Moreover, the battery consumption slightly varies for ap- suchrulesarethefollowing: “StopnotificationsfromFace-
proachesAR2,AR3andAR4evenwhentheyhavethesame bookthatcontain‘candy’and‘crush’wordsinthetitle.” and
numberoffeaturesused. Thereasonforthisdifferencecould “StopnotificationsfromWhatsAppthatcontain‘Alice’wordsin
be that these approaches use features which have different thetitleandarriveatWORK.”
number of classes. For instance, location has three classes:
Users can accept the rules, which they think are correctly
home,workandother,whereasactivityhasfiveclasses: still,
discovered,forfilteringoutnotificationsontheirphones,by
walking,running,bikingandinvehicle. Increaseinthenum-
clicking“Yes”. Iftheuserclickson“Never”forarule,the
berofclassesinafeaturewouldrequiremoreiterationsfor
applications stores that rule as a blacklisted rule and never
discoveringrulesandthusconsumemorebattery.
showsitagaininthefuture. Toensurethis,aftereveryrule
TimeComplexity miningprocess,theapplicationremovesallblacklistedrules
Wecomputethetimerequiredforminingassociationrulesby fromthenewlydiscoveredrules. Moreover,iftheuserclicks
usingallfiveapproachesfordifferentamountsoftrainingdata. on“NotNow”,theruleisre-proposedduringthenextiteration
ThetimecomplexityofthealgorithmisO(N ). oftheminingprocess. Finally,oncearuleisacceptedbya
count
user, it becomes active (see Figure 6.c) and the application
AsshowninFigure5andasexpected,thetimecomplexity
startsfilteringoutallthesubsequentnotificationsaccording
linearlyincreasesastheamountofdataincreases. Quiteinter-
totherulesthatarecurrentlyactive. Moreover,theusercan
estingly,thedifferenceintermsoftimerequiredforthecom-
clickonthe"ViewFilteredNotifications"button(shownin
putationforthevariousapproachestakenintoconsideration
Figure 6.a) in order to view the list of filtered notifications
increasesasthenumberofnotificationsincreases. Moreover,
alongwiththeirtimeofarrivalandthetriggeringapplication.
italsoincreasesasthenumberoffeaturesusedgetslarger. For
instance,thelibrarytakesnotmorethan61secondsformining
associationrulesfrom1500notificationsforanyapproach. DeploymentofPrefMiner
ThePrefMinerapplicationwaspublishedontheGooglePlay
IN-THE-WILDEVALUATION Storeandadvertisedthroughsocialmediaandotherchannels
In this section we present an in-the-wild evaluation of in our University. We ran the study for the duration of 15
PrefMiner (see Figure 6), a mobile application that is able daysfollowedbyanexitquestionnaire(seeTable1). Over-
learntheuser’spreferenceforreceivingdifferenttypesofno- all,18peopleparticipatedinthestudywithoutanymonetary
tificationsandfilteringoutthenotificationsthatarenotuseful, incentive. However, oneuserdidnotansweranyquestions
uninterestingandirrelevanttotheuser. aboutthesuggestedrulesandanotheruserdismissedallthe
45%
35
Accepted
unt2350 Rejec7te7d% 73%73% 64% 50% 34% 20%
Co20 50%
ules 15 80%79% 75% 57% 47% 25%
(a) R
(b) 10 67%
5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
User
Figure7. UsersresponsetotherulessuggestedbythePrefMinerappli-
cation.
(a) (b)
60
%)
(d) all (40
(c) ec
R
er
Filt20
(c) (d)
Figure6. PrefMinerapplication:(a)mainscreen,(b)self-rulecreation, 0
(c)activerules,(d)pendingrules. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
User
Figure8.PerformanceofPrefMinerinfilteringnotifications.
rules. Therefore,weconsideredonlytheremaining16users
Question AverageResponse
forevaluatingtheapplication. Q1.Ifoundtheappusefulforlearningmypref-
4.25
erencestofilternotifications.
Tomaketheapplicationinterestingfortheusersweallowed
Q2.Theappfilteredmostofthenotificationsthat
4.33
them to manually create their own rules. As shown in Fig- Ididn’twanttoreceive.
ure6.b,amanualrulecanbecreatedbydefiningtheapplica- Q3.Theappincorrectlyfilterednotificationsthat
1.58
tion(selectedfromalistofinstalledapplications),keywords Iwantedtoreceive.
andlocation. Inordertoensurethatthemanualrulesdonot Table1.Exitquestionnairewiththeaverageresponsefromusers.
affectourexperimentwerestricttheusersfrommanuallycre-
ate any rule during the period of the study (i.e., the first 15
Duringthestudytheapplicationalsocollectedthenotifications
daysaftertheinstallationoftheapplication). Notethat,for
thatarrivedontheuser’sphone. Ourresultsshowthateach
privacyreasons,theuserhastogiveexplicitpermissionafter
dayauserreceivesaround71notifications(withthestandard
theinstallationasrequiredbytheAndroidoperatingsystemto
deviation equal to 22 notifications). We compute the filter
allowPrefMinertomanagenotifications. Moreover,toensure
recallaspercentageoffilterednotificationsoverthesumof
thattheuserisawareofdatacollection,theapplicationalso
thenotificationsdismissedbytheusersandthefilteredones
showsadetaileduserconsentform.
(i.e.,allthenotificationsautomaticallyormanuallyfiltered).
Theresultsshowthatthefilterrecallachievedbytheaccepted
rules is 45.81%. More specifically, the average number of
In-the-wildEvaluationResults notificationsthataresuccessfullyfilteredeverydayis12(with
DuringthestudyPrefMinersuggested179rulestothepartic- thestandarddeviationequalto8). InFigure8weshowthe
ipants out of which 102 rules (i.e., 56.98%) were accepted. filterrecallofeachuser. ForsomeusersPrefMinerisableto
Figure7showsthecountforacceptedanddismissedrulesfor identifyaround60%ofthenotificationsthatusersdonotwant
each user that are sorted according to their rule acceptance toreceive. Ontheotherhand,thefilterrecallforsomeusers
percentage. Thegraphshowsthattherearesomeuserswho isfairlylow(approx. 20%). However,itisworthnotingthat
acceptedmostoftherulesandsomewhoacceptedonlyafew thesystemisoptimizedforhighprecisionsothatitdoesnot
rules. Overall,around70%oftheusersaccepted50%(and filteroutanyimportantnotifications. Moreover,foreachday
above)ofthesuggestedrules. Inordertofindwhyandwhat onaverage7notificationsperuserareclassifiedasreminders.
rulesweredismissedbyusersweanalyzedalldismissedrules.
Ourresultsshowthatmostofthedismissedrulesareaboutthe ExitQuestionnaire
“communication applications” including WhatsApp, Gmail Attheendofthe15-daystudy,theapplicationaskedusersto
and Hangouts. The other most dominant app category for fillinanexitquestionnaire(showninTable1)thatcanbeused
dismissedrulesis“systemapplications”includingBluetooth toquantifytheusefulnessandaccuracyofPrefMineraccord-
Share,AndroidSystemandGooglePlayStore. ingtotheusers. Userswereallowedtoregistertheirresponse
on a 5 point Likert scale (1:Strongly disagree - 5:Strongly 33]. Forexample,Pielotetal.[33]showthatusersconsider
agree). Out of the 16 participants only 12 participants reg- notificationsfromcommunicationapplications(suchasmes-
isteredtheirresponsetothequestionnaire. Table1liststhe sengersandemailclients)asimportant. Thesenotifications
averageresponsetothethreequestionsprovidedbythepartici- are perceived as less annoying and are less likely to be dis-
pants. Theresultsdemonstratethatusersfoundtheapplication missedcomparedtonotificationsfromotherapplications. Felt
usefulforfilteringnotificationsandaccordingtothemitfil- etal.[17]foundthattheuser’sperceptiontowardsmobileno-
teredmostoftheunwantednotificationscorrectly. tificationsvariesstrongly. Ifapplicationstriggernotifications
thatarenotconsideredrelevant,userstendtogetannoyedand
DISCUSSIONANDFUTUREWORK deletethem. Fischeretal.[19]showthatusers’receptivityis
In this paper we have presented PrefMiner that learns the influencedbytheirgeneralinterestinthenotificationcontent,
user’s preference and accordingly manages the subsequent entertainmentvalueperceivedinitandactionrequiredbyit,
notifications. To the best of our knowledge, this is the first but not the time of delivery. Mehrotra et al. [25] show that
studybasedonboththeanalysisofareal-worldnotification notificationscontainingimportantorusefulcontentareoften
content along with context and the in-the-wild deployment accepted,despitethedisruptioncausedbythem.
of a customizable notification system. We believe that our
Atthesametime,paststudieshaveproposedinterruptibility
approachformakingpredictiontechniquestransparenttothe
managementmechanismsfordeliveringnotificationsatthein-
users helps an interruptibility management system to build
ferredopportunemomentsbyusingtheuser’scontext[22,21,
users’trustsinceitreducestheriskofstoppinganyimportant
20,18,32,31,28]andthenotificationcontent[24].In[21]Ho
notification. It still reduces the burden from users by stop-
andIntillesuggestedthatthetransitionbetweentwophysical
ping45.81%ofnotificationsthatarenotconsideredusefulor
activities(suchassittingandwalking)canbeusedasoppor-
importantand,thus,minimizestheperceiveddisruption.
tunemomentsfordeliveringnotifications. Pielotetal.[31]
However, the proposed interruptibility management mecha- proposedamodelthatcanpredictwhetherauserwillviewa
nism has some minor limitations. Firstly, it does not focus notificationwithinafewminuteswithaprecisionofapproxi-
ondeferringnotificationsatinopportunemoments. Instead,it mately81%.PejovicandMusolesi[28]proposedamechanism
aimstostopunnecessarynotificationsfromalertingtheuser thatreliesonthecontextualinformation(includingactivity,
andthusreducetheoveralldisruption. Asafutureplan,we locationandtimeofday)topredictopportunemomentsfor
plantodesignandevaluateourmechanismforlearningthe deliveringnotifications. Mehrotraetal.[24]suggestedusing
momentsatwhichnotificationsshouldbedeferredandforhow boththecontextualinformationandthenotificationcontent
long. Secondly,ourmechanismisnotabletoaccuratelydetect formodelinginterruptibility. However,theauthorsassumed
remindernotifications. Instead,itfindsonlytheapplications thatanapplicationtriggersonlyasingletypeofnotifications.
forwhichallnotificationsaredismissedandlabelsthemas Instead, in this work, for the first time, we design an inter-
reminderapplicationsandremindernotifications. Webelieve ruptibilitymanagementsystemthatclassifiesnotificationsinto
thatifwecanaccuratelyidentifyandremoveallreminderno- different classes based on the information they contain and
tifications,ourinterruptibilitymanagementmechanismwould learnstheuser’spreferencesforreceivingthetypesofinfor-
beabletodiscovermoreaccuraterulesandthustheoverall mationindifferentsituations.
accuracycanbeimproved. Thismightbebasedonthemanual
annotationofcertaintypeofnotifications, suchasstandard
CONCLUSIONS
Androidbatterywarningmessages. Inthispaperwehavepresentedanovelsolutionforintelligent
Anotherlimitationisthatwecouldnotinfertheuserresponse notificationmanagementbasedontheautomaticextractionof
tonotificationsthatarehandledfromotherdevices. Thus,we rulesthatreflectuser’spreferences. Thegoaloftheproposed
havetodiscardsuchnotifications. Webelievethatbybeing approachistomakenotificationsintelligibletousers. Wefirst
able to detect the user response for such notifications, our evaluateourproposedmechanismwithalarge-scaledataset
mechanismcanstartmakingstablepredictioninfewerdays ofnotificationscollectedduringaninterruptibilitystudy. Our
andmightalsodiscoversomeinterestingrulesacrossdevices. resultsshowthatbyusingthenotificationtitleandtheuser’s
Forthisreason,anotherextensionofthisstudyisthedesign location,wecanpredictifamessagewillbedismissedbya
ofarule-basedmanagementsystemacrossdevices. userwithaveryhighprecision.
Finally,thecurrentimplementationoftheMyPreflibrarycan WehavealsodiscussedthedesignofanopensourceAndroid
only support mobile phones that are configured to use the libraryimplementingtheinterruptibilitymechanismandthe
Englishlanguage. Thisisbecausethesephonesreceivethe implementationofthePrefMinerapplicationbuiltontopof
notificationtitlesinEnglishandthecurrentimplementation it. Throughanin-the-wilddeployment,wehaveshownthat
of our library could classify only the text in this language. PrefMinerrepresentsaveryeffective,yettransparent,solution
We plan to evolve our library’s capability to support other forinterruptibilitymanagementformobiledevices.
languagesinthefuture.
ACKNOWLEDGEMENTS
RELATEDWORK This work was supported through the EPSRC Grant
The area of mobile interruptibility has received increasing “MACACO:Mobilecontext-AdaptiveCAchingforContent-
attentioninthepastyears. Previousstudieshaveexploredvar- Centricnetworking”(EP/L018829/2).
iousinterestingaspectsoftheproblem[8,11,10,14,25,30,