ebook img

Service-Oriented Crowdsourcing: Architecture, Protocols and Algorithms PDF

101 Pages·2012·2.168 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Service-Oriented Crowdsourcing: Architecture, Protocols and Algorithms

SpringerBriefs in Computer Science Series Editors Stan Zdonik Peng Ning Shashi Shekhar Jonathan Katz Xindong Wu Lakhmi C. Jain David Padua Xuemin Shen Borko Furht V. S. Subrahmanian Martial Hebert Katsushi Ikeuchi Bruno Siciliano For furthervolumes: http://www.springer.com/series/10028 Daniel Schall Service-Oriented Crowdsourcing Architecture, Protocols and Algorithms 123 Daniel Schall Siemens Corporate Technology Vienna Austria ISSN 2191-5768 ISSN 2191-5776 (electronic) ISBN 978-1-4614-5955-2 ISBN 978-1-4614-5956-9 (eBook) DOI 10.1007/978-1-4614-5956-9 SpringerNewYorkHeidelbergDordrechtLondon LibraryofCongressControlNumber:2012950384 (cid:2)TheAuthor(s)2012 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionor informationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserofthe work. Duplication of this publication or parts thereof is permitted only under the provisions of theCopyrightLawofthePublisher’slocation,initscurrentversion,andpermissionforusemustalways beobtainedfromSpringer.PermissionsforusemaybeobtainedthroughRightsLinkattheCopyright ClearanceCenter.ViolationsareliabletoprosecutionundertherespectiveCopyrightLaw. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. While the advice and information in this book are believed to be true and accurate at the date of publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with respecttothematerialcontainedherein. Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) Preface Crowdsourcinghas emergedasan important paradigmin human problem solving techniques on the Web. More often than noticed, programs outsource tasks to humans which are difficult to implement in software. Service-oriented crowd- sourcing enhances these outsourcing techniques by applying the principles of service-orientedarchitecture(SOA)tothediscovery,composition,andselectionof ascalablehumanworkforce.Thisbookprovidesbothananalysisofcontemporary crowdsourcing systems such as Amazon Mechanical Turk and a statistical description of task-based marketplaces. In the following, a novel mixed service- oriented computing paradigm is introduced by providing an architectural descriptionoftheHuman-ProvidedServices(HPS)frameworkandtheapplication of social principles to human coordination and delegation actions. Then, the previously investigated concepts are extended to business process management integration including the extension of XML-based industry standards such as WS-HumanTask and BPEL4People and the instantiation of flexible processes in crowdsourcing environments. Vienna, August 2012 Daniel Schall v Acknowledgments The work presented in this book provides a consolidated description of the author’s research in the field of human computation and crowdsourcing techni- ques. He started investigating crowdsourcing techniques in 2005 at Siemens Corporate Research in Princeton, NJ, USA. In 2006, he started his doctoral studies at the Vienna University of Technology (TU Wien) where he was employed as a project manager and research assistant. At that time he was involved in the EU FP6 project inContext (interaction and context-based tech- nologies for collaborative teams) and defined a number of key principles such as the notion of Human-Provided Services and algorithms for context-sensitive expertise mining. In the following, the author worked as a Senior Research Scientist also at TU Wien where he was the principle investigator of efforts related to crowdsourcing techniques and mixed service-oriented systems. During this time period he was involved in a number of projects including the EU FP7 projects collaboration and interoperability for networked enterprises (COIN) and compliance-driven models, languages, and architectures for services (COMPAS). During his time at TU Wien, he published more than 50 scientific publications in highly ranked journals and renown magazines including the IEEE Transaction on Services Computing, IEEE Computer, IEEE Internet Computing, Data and Knowledge Engineering, Distributed and Parallel Databases, Social Network Analysis and Mining, Information Systems, as well as numerous world class conferences including the International Conference on Business Process Man- agement, the International Conference on Services Computing, the International Conference on Advanced Information Systems Engineering, the International Conference on Social Informatics, the International Conference on Self-Adaptive and Self-Organizing Systems, or the International Conference on Engineering of ComplexComputerSystems.Thefinalizationofthisbookwascarriedoutwhilethe authorhasalreadybeenwithSiemensCorporateTechnology—aresearchdivision ofthe Siemens AG. vii Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Task Marketplaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 SOA for Crowdsourcing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.4 Adaptive Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 Outline. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2 Crowdsourcing Task Marketplaces. . . . . . . . . . . . . . . . . . . . . . . . 7 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3 Basic Model and Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3.1 System Context Overview . . . . . . . . . . . . . . . . . . . . . . 10 2.3.2 Marketplace Task Statistics . . . . . . . . . . . . . . . . . . . . . 11 2.4 Clustering and Community Detection. . . . . . . . . . . . . . . . . . . . 14 2.4.1 Clustering Approach . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.4.2 Community-Based Ranking Model . . . . . . . . . . . . . . . . 16 2.5 Crowdsourcing Broker Discovery . . . . . . . . . . . . . . . . . . . . . . 17 2.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.6.1 Community Discovery and Ranking . . . . . . . . . . . . . . . 19 2.6.2 Recommendation of Crowdsourcing Brokers . . . . . . . . . 22 2.7 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 27 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3 Human-Provided Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.3 HPS Interaction Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.3.1 HPS Activity Model . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.3.2 Hierarchical Activities. . . . . . . . . . . . . . . . . . . . . . . . . 36 ix x Contents 3.3.3 Task Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.3.4 Task Execution Model. . . . . . . . . . . . . . . . . . . . . . . . . 38 3.4 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.4.1 HPS Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.4.2 Data Collections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.4.3 Interactions and Monitoring . . . . . . . . . . . . . . . . . . . . . 43 3.5 Expertise Ranking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.5.1 Context-Sensitive Interaction Mining. . . . . . . . . . . . . . . 45 3.5.2 Hubs and Authorities. . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.5.3 Personalized Expert Queries. . . . . . . . . . . . . . . . . . . . . 47 3.5.4 Ranking Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.6.1 SOA Testbed Environment. . . . . . . . . . . . . . . . . . . . . . 51 3.6.2 Performance Aspects. . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.6.3 Quality of Expertise Rankings . . . . . . . . . . . . . . . . . . . 54 3.7 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 56 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4 Crowdsourcing Tasks in BPEL4People. . . . . . . . . . . . . . . . . . . . . 59 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.3 Service-Oriented Crowdsourcing. . . . . . . . . . . . . . . . . . . . . . . 62 4.3.1 Task-Based Crowdsourcing Markets . . . . . . . . . . . . . . . 62 4.3.2 Approach Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.4 Non-Functional Properties in B4P . . . . . . . . . . . . . . . . . . . . . . 65 4.4.1 Human Tasks in B4P. . . . . . . . . . . . . . . . . . . . . . . . . . 66 4.4.2 Basic Model and Extensions. . . . . . . . . . . . . . . . . . . . . 67 4.5 Social Aggregator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.6 Task Segmentation and Matching . . . . . . . . . . . . . . . . . . . . . . 73 4.6.1 Hierarchical Crowd Activities. . . . . . . . . . . . . . . . . . . . 73 4.6.2 Social Interactions. . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 4.6.3 Ranking Coordinators . . . . . . . . . . . . . . . . . . . . . . . . . 76 4.7 Implementation and Evaluation. . . . . . . . . . . . . . . . . . . . . . . . 81 4.7.1 SOA-Based Crowdsourcing Environment. . . . . . . . . . . . 81 4.7.2 Social Network Generation . . . . . . . . . . . . . . . . . . . . . 85 4.7.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 4.7.4 Overall Findings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 4.8 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 90 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Acronyms AMT Amazon Mechanical Turk API Application Programming Interface BPEL Business Process Execution Language B4P Business Process Execution Language 4 People BPM Business Process Management CFL Crowd Flow HIT Human Intelligent Task HPS Human Provided Service NFP Nonfunctional Property PFL Process Flow RFS Request For Support SBS Software-Based Service SOA Service-Oriented Architecture WSDL Web Services Description Language WSHT Web Services Human Task XML Extended Markup Language xi Chapter 1 Introduction Abstract Thischaptergivesanintroductiontohumancomputationandcrowdsourc- ingtechniques.Next,thekeyfeaturesofhumantaskmarketplacessuchasAmazon MechanicalTurkarebrieflyoutlined.Inthefollowing,service-orientedcrowdsourc- ingismotivatedbygivinganexample.Finally,adaptiveprocessesinthecontextof crowdsourcingarediscussedandanoutlineofthebookisgiven. 1.1 Overview The shift toward the Web 2.0 allows people to write blogs about their activities, shareknowledgeinforums,writeWikipages,andutilizesocialplatformstostayin touchwithotherpeople.Task-basedplatformsforhumancomputationandcrowd- sourcing, including CrowdFlower [7], Google’s Smartsheet [17], or Yahoo’s Pre- dictalot[11]enableaccesstothemanpowerofthousandsofpeopleondemandby creatinghuman-tasksthatareprocessedbythecrowd.Human-tasksincludeactivities suchasdesigning,creating,andtestingproducts,votingforbestresults,ororganizing information.Thenotionofcrowdsourcingdescribesanonline,distributedproblem solvingandproductionmodelwithincreasinglyinterestedbusinesspartiesinthelast coupleofyears[6].Crowdsourcingfollowstheopenworldassumption[9]wherein peersinteractandcollaboratewithoutbeingorganizedonamanagerial/hierarchical model[5].Thousands ofindividualsmaketheirindividualcontributionstoabody ofknowledgeandproducethecoreofourinformationandknowledgeenvironment. One of the main motivations to outsource activities to a crowd is the potentially considerable spectrum of returned solutions. Furthermore, competition within the crowdensuresacertainlevelofquality. Accordingto[18],therearetwodimensionsinexistingcrowdsourcingplatforms. Thefirstcategorizesthefunctionoftheplatform.Currentlythesecanbedividedin communities(i)specializedonnoveldesignsandinnovativeideas,(ii)dealingwith code development and testing, (iii) supporting marketing and sales strategies, and D.Schall,Service-OrientedCrowdsourcing,SpringerBriefsinComputerScience, 1 DOI:10.1007/978-1-4614-5956-9_1,©TheAuthor(s)2012 2 1 Introduction (iv)providingknowledgesupport.Anotherdimensiondescribesthecrowdsourcing mode. Community brokers assemble a crowd according to the offered knowledge and abilities that bid for activities. Purely competition based crowdsourcing plat- forms operate without brokers in between. Depending on the platform, incentives forparticipationinthecrowdareeithermonetaryorsimplecredit-oriented.Evenif crowdsourcingseemsconvenientandattractsenterpriseswithascalableworkforce andmultilateralexpertise,thechallengesofcrowdsourcingareadirectimplication ofhuman’sad-hoc,unpredictablebehaviorandavarietyofinteractionpatterns. 1.2 TaskMarketplaces Task-basedcrowdsourcingplatformssuchasAmazonMechanicalTurk[2](AMT) enable businesses to access the manpower of thousands of people on demand by postinghuman-taskrequestsonAmazon’sWebsite.Todate,AMTprovidesaccess to the largest group of workers available for processing Human Intelligent Tasks (HIT).CrowdsourcingplatformslikeAMTtypicallyofferauserportaltomanage HITs.Suchtasksaremadeavailableviaamarketplaceandcanbeclaimedbywork- ers.Inaddition,mostplatformsofferapplicationprogramminginterfaces(APIs)to automatethemanagementoftasks.However,fromtheplatformpointofview,there is currently very limited support in helping workers to identify relevant groups of tasksmatchingtheirinterests.Also,asthenumberofbothrequestersissuingtasks andworkersgrowsitbecomesessentialtodefinemetricsassistinginthediscovery ofrecommendable requesters.Somerequestersmayspamtheplatformbyposting unusabletasks.Astudyfrom2010showedthat40%oftheHITsfromnewrequesters arespam[10]. 1.3 SOAforCrowdsourcing Service-oriented architecture (SOA) is an emerging paradigm to realize extensi- blelarge-scalesystems.Asinteractionsandcompositionsspanningmultipleenter- prises become increasingly commonplace, organizational boundaries appear to be diminishing in future service-oriented systems. In such open and flexible enter- priseenvironments,peoplecontributetheircapabilitiesinaservice-orientedmanner. Weconsidermixedservice-orientedsystems[12,13]basedontwoelementarybuild- ingblocks:(i)Software-BasedServices(SBS),whicharefullyautomatedservices and(ii)Human-ProvidedServices(HPS)[14]forinterfacingwithpeopleinaflexi- bleservice-orientedmanner.Herewediscussservice-orientedenvironmentswherein servicescanbeaddedatanypointintime.Byfollowingtheopenworldassumption, humans actively shape the availability of HPSs by creating services. Interactions betweenHPSsareperformedbyusingWebservice-basedtechnology(XML-based SOAPmessages).

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.