Personalized Task Recommendation in Crowdsourcing Systems

116 Pages·2016·2.529 MB·English

by  David Geiger (auth.)

Progress in IS
David Geiger
Personalized Task
Recommendation
in Crowdsourcing
Systems
Progress in IS
More information about this series at http://www.springer.com/series/10440
David Geiger
Personalized Task
Recommendation in
Crowdsourcing Systems
DavidGeiger
UniversityofMannheim
Mannheim
Germany
ThisbookisbasedonadoctoralthesissubmittedtotheUniversityofMannheim.
ISSN2196-8705 ISSN2196-8713 (electronic)
ProgressinIS
ISBN978-3-319-22290-5 ISBN978-3-319-22291-2 (eBook)
DOI10.1007/978-3-319-22291-2
LibraryofCongressControlNumber:2015950152
SpringerChamHeidelbergNewYorkDordrechtLondon
©SpringerInternationalPublishingSwitzerland2016
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilarmethodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt
fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthis
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained
hereinorforanyerrorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
Springer International Publishing AG Switzerland is part of Springer Science+Business Media
(www.springer.com)
Abbreviations
API Applicationprogramminginterface
CI Confidenceinterval
CSS CascadingStyleSheets
DDD Domain-drivendesign
DOM DocumentObjectModel
HIT HumanIntelligenceTask
HTML HypertextMarkupLanguage
HTTP HypertextTransferProtocol
IETF InternetEngineeringTaskForce
IP InternetProtocol
IS Informationsystems
JAX-RS JavaAPIforRESTfulWebServices
JS JavaScript
JSON JavaScriptObjectNotation
REST RepresentationalStateTransfer
RFC RequestforComments
UI Userinterface
URI UniformResourceIdentifier
URL UniformResourceLocator
W3C WorldWideWebConsortium
v
ThiSisaFMBlankPage
Contents
1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 ResearchContext. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 ResearchObjective. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 ResearchApproach. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . 5
2 CrowdsourcingSystems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 ASocio-TechnicalPerspective. . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2 OrganizationalFunctions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2.1 TypologyDevelopment. . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.2 SystemArchetypes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3 CurrentStateofPersonalizedTaskRecommendation. . . . . . . . . . . 15
3.1 Methodology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Findings. . . . .. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . 17
3.2.1 RecommenderContext. . . . . . . . . . . . . . .. . . . . . . . . . . . 17
3.2.2 RecommenderTechniques. . . . . . . . . . . . . . . . . . . . . . . . 18
3.2.3 RecommenderEvaluation. . . . . . . . . . . . . . . . . . . . . . . . 21
3.3 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3.1 ApplicationsandLimitations. . . . . . . . . . . . . . . . . . . . . . 23
3.3.2 SourcesofIndividualKnowledge. . . . . . . . . . . . . . . . . . . 24
3.3.3 TheRoleofCapabilitiesandContributions. . . . . . . . . . . . 25
3.3.4 ConnectingExternalKnowledgeSources. . . . . . . . . . . . . 26
3.3.5 TheRightRecommendationTechnique. . . . . . . . . . . . . . 27
3.3.6 TheUtilityofPersonalizedTaskRecommendation. . . . . . 28
4 DesignofaThird-PartyTaskRecommendationService. . . . . . . . . . 31
4.1 Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.2 Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.2.1 Domain-DrivenDesign. . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2.2 PortsandAdapters. .. . . . . . . .. . . . . . . .. . . . . . .. . . . . 34
vii
viii Contents
4.3 DomainModel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.3.1 Contributors,Tasks,andInteractions. . . . . . . . . . . . . . . . 37
4.3.2 RecommendationGeneration. . . . . . . . . . . . . . . . . . . . . . 39
4.3.3 IdentityManagement. . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.4 ApplicationServices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.5 Adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.5.1 ExtensionAPI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.5.2 TaskSynchronization. . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.5.3 AppEngineInfrastructure. . . . . . . . . . . . . . . . . . . . . . . . 47
4.6 BrowserExtension. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.6.1 UserInterface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.6.2 SynchronizingContributorData. . . . . . . . . . . . . . . . . . . . 56
4.7 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5 PersonalizedTaskRecommendationintheField. . . . . . . . . . . . . . . 61
5.1 PilotStudy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.2 ContributorSurvey. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.2.1 QuestionnaireDesign. . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.2.2 DataCollection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.2.3 DataAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.2.4 Discussion. .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . 73
5.3 OnlineEvaluation. . . .. . . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . 74
5.3.1 IntensityofUse. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.3.2 PredictiveCapability. . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
5.3.3 Discussion. .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . 79
6 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
AppendixA:FindingsofSystematicLiteratureReview. . . . . . . . . . . . . . 83
AppendixB:RecommenderPerformanceOptimizations. . . . . . . . . . . . . 89
AppendixC:UniquePropertyService. . . . . . . . . . . . . . . . . . . . . . . . . . . 91
AppendixD:ContributorSurvey. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Figures
Fig.2.1 Componentsofacrowdsourcingsystem............................. 9
Fig.2.2 Thefourarchetypesofcrowdsourcingsystems..................... 12
Fig.4.1 Tacticaldesignpatternsindomain-drivendesign................... 33
Fig.4.2 PortsandAdapters..................................................... 35
Fig.4.3 Metacrowd’scontent-basedrecommendationalgorithm........... 42
Fig.4.4 ThemetacrowdextensionintheChromeWebStore............... 53
Fig.4.5 Userinterfaceofthemetacrowdbrowserextension................ 54
Fig.4.6 Metacrowd’sprivacypolicy........................................... 54
Fig.5.1 Exemplaryforumannouncement..................................... 62
Fig.5.2 Surveyrecruitmenttask............................................... 65
Fig.5.3 Platformexperienceandactivityfrequencydistributions.......... 67
Fig.5.4 Searchchannelfrequencydistributions.............................. 68
Fig.5.5 Searchcriteriafrequencydistributions............................... 70
Fig.5.6 Searchtimefrequencydistributions.................................. 71
Fig.5.7 Perceivedusefulnessfrequencydistribution......................... 72
Fig.5.8 Initialsubmissionhistorysizeperuser.............................. 75
Fig.5.9 Numberofexaminedorrejectedrecommendationsperuser...... 75
Fig.5.10 Meandifferencesinthelognumberofdatapointsperuser....... 78
Fig.C.1 Exemplaryuniquevalueentities...................................... 92
Fig.D.1 Contributorsurvey:welcomepage................................... 95
Fig.D.2 Contributorsurvey:partone(platformexperience)................ 95
Fig.D.3 Contributorsurvey:parttwo(searchbehavior)..................... 96
Fig.D.4 Contributorsurvey:partthree(metacrowd)......................... 97
ix