Table Of Content

Cross-language Wikipedia Editing of Okinawa, Japan ScottA.Hale OxfordInternetInstitute,UniversityofOxford 1StGiles,Oxford,UKOX13JS [email protected] ABSTRACT hand,increasedlanguagediversitymayalsoincreasetherisk ThisarticleanalyzesuserswhoeditWikipediaarticlesabout of fragmenting content and users across languages, particu- Okinawa,Japan,inEnglishandJapanese. Itfindstheseusers larlyifmultilingualuserswhowouldhaveusedthesiteina 5 are among the most active and dedicated users in their pri- large,internationallanguagemoveexclusivelytosmallerlan- 1 mary languages, where they make many large, high-quality guageeditions.OnquestionandanswerplatformStackOver- 0 edits. However, when these users edit in their non-primary flow,theriskoffragmentation—thatis,theriskthatmultiple 2 languages, they tend to make edits of a different type that language editions of the site would result in fewer users to r are overall smaller in size and more often restricted to the any one particular language edition and therefore less high- a M narrow set of articles that exist in both languages. Design qualityanswers—hasbeencitedasoneofthereasonsforthe changes to motivate wider contributions from users in their platformtoremainmonolingual.1 2 non-primary languages and to encourage multilingual users Key to the trade-off between the potential increase in other- 2 totransfermoreinformationacrosslanguagedividesarepre- languageusersandtheriskoffragmentationacrosslanguages sented. aretherolestechnologyandusersplayinfacilitatingtheflow ] Y of information between languages. Previous research has ACMClassificationKeywords suggestedmultilingualuserswhoreadandcontributecontent C H.5.4 Information Interfaces and Presentation (e.g. HCI): in multiple languages may share novel information between . Hypertext/Hypermedia; H.5.3 Information Interfaces and s languagesandbroadenthescopeofinformationavailableto Presentation(e.g. HCI):GroupandOrganizationInterfaces c monolingualusersonline[10,15,16,18]. Researchtodate, [ GeneralTerms however,haseitherexamineddifferencesincontentbetween 2 languages[6, 8, 18]orexamineduserbehavior[10, 15, 16], HumanFactors,Design v butnotboth. Thus,while15%oftheactiveeditorsonWiki- 7 pediacontributetomultiplelanguageeditions[16],itremains AuthorKeywords 5 unclearwhatcontentmultilingualuserscontribute,howmuch SocialMedia;InformationDiscovery;SocialNetwork 6 content they contribute, and how valuable the content they Analysis;InformationDiffusion;Cross-language; 0 contributeis. Thesequestionsareimportantinunderstanding 0 Wikipedia;Multilingual thecurrentrolesmultilingualusersplayintransferringinfor- . 1 mation between languages as well as gaining insight for de- INTRODUCTION 0 signingmultilingualplatformsinwaysthatmaximizecross- Allowinguserstocontributecontentinmultiplelanguageson 5 language information transfer. This paper starts to address user-generatedcontentplatformsresultsinalargedifference 1 thisgapbyexaminingthecontentcontributedbymultilingual : inthecontentavailableindifferentlanguages. WithinWiki- v pedia, for example, over 74% of concepts have an article in users in their primary and non-primary languages on one of i thelargestmultilingualuser-generatedcontentplatformson- X only one language edition, and more than 95% of concepts line,Wikipedia. Thefindingsleadtoamorein-depthunder- appear in six or fewer languages [18]. This finding is not r standing of the role multilingual users play on multilingual a uniquetoWikipedia: onTwitterthereisalsosurprisinglylit- user-generatedcontentplatformsandsuggestplatformdesign tleoverlapbetweenthetopdomainnamesandhashtagsused strategies. in tweets of different languages [19]. User-generated con- tentplatformsfaceatrade-offonallowingtheuseofmultiple languages. Ontheonehand,increasedlanguagediversityin- BACKGROUNDANDRELATEDWORK creases the potential number of contributors as monolingual EachlanguageeditionofWikipediaexhibitsaself-focus:that individuals from multiple languages can join. On the other is, each edition has, in general, more information about the regionswherethelanguageoftheeditionisspokenandless informationabouttheregionswherethelanguageisnotspo- ken [18]. Even when corresponding articles for a common conceptexistacrossmultiplelanguageeditions,thereisasur- prising diversity in the topics covered in each language edition’s article on that concept [5]. For example, the Spanish- 1http://blog.stackoverflow.com/2009/07/ (cid:13)c ScottA.Hale2014.Thisistheauthor’sversionofthework.Itispostedherefor non-english-question-policy/ yourpersonaluse.Notforredistribution.ThedefinitiveversionwaspublishedinCHI 2015,http://dx.doi.org/10.1145/2702123.2702346. 1 languagearticleonpsychologycontainsasectiononimpor- zerobytes. Usingthecontentofedits,thisstudycalculatesa tant contributions from Latin America that is not found in morenuancedmeasureofeditsizeusingthenumberofwords otherlanguageeditions[18]. added,thenumberofwordsremoved,andtheamountofre- organization performed. A second dimension by which to TheglobalnatureoftheInternetallowsforthepossibilitythat compare contributions is the types of content changes users expatriates, language-learners, and other-culture/language make in their non-primary languages. The study of 2010 enthusiasts (who Zuckerman [31] has termed xenophiles) Haitianearthquakeblogsfoundthatthesharingofimagesand canspreadinformationbetweenlanguagesonuser-generated videos was a large motivation for crossing language bound- contentplatforms. Inaone-monthstudyofeditstothetop46 ariesamongbloggers[14],whichsuggestsadding,removing, languageeditionsofWikipedia,Hale[16]foundthatapprox- or otherwise modifying images might be more prevalent in imately 15% of active Wikipedia users edited multiple lan- users’non-primarylanguages. guageeditionsoftheencyclopedia. Thesemultilingualusers were distributed across all language editions, but smaller- Thefinalresearchquestionposedinthispaperaskshowvalu- sized editions with fewer users had a higher percentage of ablearethecontributionsbymultilingualusersintheirnon- multilingualuserscomparedtolarger-sizededitions.Theper- primarylanguages? (RQ3). Giventhediversityininforma- centageofmultilingualusersprimarilyeditingeachlanguage tion between languages on Wikipedia [18] as well as online edition was found to negatively correlate with the self-focus moregenerally,editsbymultilingualusershavethepotential of the 15 editions studied by Hecht and Gergle [18]: that tointroducetrulynovelandvaluableinformationandsources is, editions with more multilingual users exhibited less self- from one language into articles in another language. Con- focus[16]. nections across languages may serve as “bridges” and have been compared to the concept of “weak ties” in social net- Multilingualusersarewellsituatedtoactasbridgesorgate- work analysis [10, 13], where ample scholarship has shown keepers and transfer content between languages; however, weak ties to be of critical importance to the spread of infor- past work points to a more nuanced picture of the extent to mation, withbenefitsascribedtoboththeindividualandthe which multilingual users actually fulfill this role. Studies of system/networkasawhole[7]. multilingualusersonTwittershowtheyareinstructuralpo- sitionstoactasbridgesacrosslanguagegroups[10,15]and The study of multilingual users on Wikipedia also found a thatapproximately11%ofactiveTwitteruserswriteinmulti- positivecorrelationbetweenmultilingualismandthenumber plelanguages[15].However,astudyofmultilingualuserson of edits users’ made in their primary languages: users more Twitter in Switzerland, Qatar, and Quebec using LDA topic active in their primary languages were more likely to edit modelingfoundthatmultilingualusersoftenfocusedondif- in multiple languages [16]. This suggests that multilingual ferenttopicsindifferentlanguages[20]. Thissuggeststhese users overlap to some extent with the group of very active multilingual users may not be bridging across languages as elite or power users on Wikipedia, on which, like on many muchastheirstructuralpositionssuggest. Asimilarsituation otherplatforms,muchoftheworkisdonebyasmallpercent- maybepresentinWikipedia,where43%ofthemultilingual age of very active users [29, 21]. Monolingual studies have usersstudiedbyHale[16]editedarticlesaboutdifferentcon- foundthattheseusersareresponsibleforadisproportionately ceptsintheirprimaryandnon-primarylanguages. large percentage ofthe content in the encyclopedia [21]and areoverwhelminglyresponsibleforthecontentthatisviewed The first research question of this paper, therefore, simply mostfrequently[29]. Itremainsunclear,however,howmuch askswhatarticlesdomultilingualseditintheirnon-primary contenttheseuserscontributeandhowlongitpersistswhen languages? (RQ1). This paper compares the articles mul- theyareeditingintheirnon-primarylanguages. tilingual users edit in their non-primary languages with the articles edited by other users to understand the scope within In order to investigate both users and content effectively, which multilingual users may transfer information between this paper focuses on content and multilingual user contri- languages. The results show that multilingual users edit a butionsinEnglishandJapaneseinarelativelycontainedge- narrower set of articles in their non-primary languages and ographic area: Okinawa, Japan. Okinawa is an archipelago suggest design interventions such as cross-language content ofsmall, sub-tropicalislandshometoalargenumberofna- recommendationsystemscouldbroadenthescopeofarticles tive Japanese speakers and a large number of native English userseditintheirnon-primarylanguages. speakers.2 Geographically closer to Taipei than Tokyo, the islandswereoncepartofaprosperousindependentkingdom Beyond what articles users edit in a non-primary language, built on trade in the region. After formal incorporation into thispaperalsoaskswhattypesofeditsdomultilingualusers Japan at the end of the 1800’s, the islands were separated makeintheirnon-primarylanguages? (RQ2)inordertoun- fromJapanattheendofWorldWarIIandadministeredbythe derstandthenatureandtheextentoftheinformationthatmul- United States until 1972. Since that time, the US has main- tilingualuserstransferbetweenlanguages. Afirstdimension tainedastrongpresence, withhalfofallUSpersonnel(mil- by which to compare contributions is size. Using the meta itary, contractors, dependents) in Japan under the US Status data available from Wikipedia, Hale [16] analyzed the dif- ofForcesAgreementlocatedinOkinawa. Thisaccountsfor ference in the size of articles before and after each edit in bytes, but that measure is not the most reliable as an edit that adds a large block of text but also removes a different 2TheauthorhaslivedinOkinawaandcanspeakJapaneseandEn- block of text could result in a size difference very close to glish. 2 justunder50,000individuals[24]withmilitaryfacilitiesoc- CorrespondingarticlesintheEnglishandJapaneselanguage cupyingjustover18%ofthelandareaofthelargestandmost editions were found using the October 2013 database dump populatedisland[26]. from WikiData.6 Launched in 2012, WikiData centralizes allinterlanguagelinks(and,increasingly,statisticsandother Previous qualitative studies of Wikipedia have found differ- structureddata)inonelocationandavoidssomepreviousis- ences in the editing behavior of users editing different lan- sueswithout-of-dateorconflictinginterlanguagelinkswhen guageeditionsofWikipedia—suchascorrelationswithHof- the links were separately maintained in each language edi- stede’s cultural dimensions [28]. These studies have not ex- tionofWikipedia. Foreachnon-anonymoususer,theCentral amined the roles played by users who edit in multiple lan- Authorizationdatabasewasqueriedwiththeusernametode- guages as is done in this article, but these previous studies termine if the username was a global account connected to do underscore the importance of studying users in multiple multiple language editions. If it was, the database for each languages before making generalizations. Nonetheless, En- language edition the user edited was queried to get the total glish and Japanese are good initial languages to study given numberofeditsperlanguagethattheusermadesincecreat- thattheyareamongthemost-usedlanguagesonline,notonly ingtheaccount. Thelanguageofeachuser’smosteditededi- on Wikipedia [16], but also on Twitter [15]. Furthermore, tionisreferredtoastheuser’sprimarylanguagethroughout speakers in each language play vastly different roles in in- thispaperwhilethelanguagesofanyothereditionseditedby terlanguageconnections[15,16]. Intheone-monthstudyof theuser arereferredtoas theuser’snon-primarylanguages. editstoWikipedia,Japanesewasamajoroutlierwithonly6% Users belonging to the (ro)‘bot’ group or having the ‘bot’ of the primary editors of the Japanese edition editing a sec- templateontheiruserpagesaswellasusersthathadbeensus- ondedition[16]. Incontrast,Englishwasverycentralinthe pendedformaliciouseditingwereremovedinordertofocus cross-languagemovementsofusers: whennon-Englishusers onthebehaviorofgood-faith,humaneditors.7 editedasecondedition,thateditionwasmostfrequentlyEn- glish[16]. Measuresofeditsizeandvalue Users contribute value to Wikipedia in many ways. For DATA example, Kriplean et al. [22] found 42 different types of Finding the subset of all articles related to a particular ge- contributions to Wikipedia through an analysis of barnstars ographic region on Wikipedia involves a trade-off between (personalized tokens of appreciation given to fellow users). directrelevanceandcompleteness(or,usingtheterminology Thetypesofcontributionstheyidentifiedincludedprogram- of information retrieval, between precision and recall). Us- mingtools/bots,designingtemplates,performingadministra- ingtheWikimediaLabs3 infrastructure,threearticlesamples tive functions, teaching, and leadership. Welser et al. [30] wereextractedinOctober2013. Thegeotagsampleincluded similarly identified multiple user roles by analyzing the dis- allarticleswithgeographicinformation(geotags)physically tribution of users’ edits across the different namespaces on placing the articles in Okinawa.4 The category sample in- Wikipedia(articles,articletalkpages,userpages,etc.). cluded all articles in any category that contained the word Inordertohaveonequantitativemeasureofthevalueofed- “Okinawa” for the English edition or “沖縄” (Okinawa) for itstoarticles,thispaperuseseditpersistence,followingpast theJapaneseedition. Finally,thearticlelinksampleincluded work analyzing Wikipedia in one language [1, 2, 3, 29]. In allarticlescontaining alinktoan articlestartingwith“Oki- particular,thispaperusesthealgorithmsdevelopedbyAdler nawa”fortheEnglisheditionortoanarticlestartingwith“沖 et al. [1, 2, 3] for their WikiTrust content-driven reputation 縄”(Okinawa)fortheJapaneseedition. Alleditstoeachar- system for Wikipedia to compute the extent to which each ticleineachsamplewerethendownloadedfromthedatethe editsurvivesthroughthenextsixrevisionstothearticle(edit article was created until October 2013 using the Wikipedia persistence). WikiTrust accounts for some of the complexi- API.5 tiesofWikipediaincludingdifferencescreatedbyrewording The article samples were filtered to only include articles in and rearranging text, and it also correctly attributes text that the main, article namespace (i.e. not talk pages, user pages, isdeletedandthenrestoredtothefirstauthorwhocontributed etc.). Thearticlelinksamplewasalsofilteredtoonlyinclude it(ratherthantotheauthorwhorestoredit)[2]. Editpersis- articlesthatmentionedOkinawainthemainbodytextofthe tence is calculated on a word-by-word basis for the next six article(i.e.nottranscludedviaatemplatetoappearinaside- revisionstoeacharticlesothatusersgetpartialcreditforan barorfooter).Thiswasdonesothatthearticlesinthesample edit even when part of their edit is subsequently changed or hadamoresubstantialconnectiontoOkinawathanjustbeing removed. partofalargegroupofarticleslinkedtogetherbyacommon WikiTrust also computes a more detailed measure of edit portal or category such as “Regions and administrative divi- sizesthanisreporteddirectlyfromWikipedia. Theeditsizes sions of Japan” or “USAAF Eighth Air Force in World War reportedbyWikipediasimplymeasurethedifferenceinpage II.” 6http://www.wikidata.org/ 7Thestatusofuseraccountswascheckedoneyearafterdatacollec- 3https://www.mediawiki.org/wiki/Wikimedia_Labs tioninDecember2014toallowsufficienttimeformaliciouseditors 4Theboundingboxusedincludedarticlesbetween25.75and26.9E tobereportedandblocked.Thedataaswellasthecodeusedfordata and126and129N. collectionandanalysisareavailableathttp://www.scotthale. 5https://www.mediawiki.org/wiki/API:Main_page net/pubs/?chi2015. 3 Sample en-only ja-only Both each Japanese article in the sample as a node, and one network for the English edition with each English article as a Geotag 52 185 152 node. Edgesinbothnetworksweretheinter-articlelinksbe- Category 156 2,819 707 tweenarticlesinthesamelanguageedition. Nodeswerethen Articlelink 3,411 9,984 5,567 rankedusingthePageRankmethod[27].Thisalgorithm,also Table1. Thenumberofuniqueconceptsineachsample. Themajor- usedbyGoogle,ranksnodesbythenumberoflinkstothem ityofconceptshaveanarticleeitheronlyintheEnglisheditionoronly weightedbythePageRanksofthenodesfromwhichthelinks intheJapaneseedition(en-onlyorja-only),whileasmallernumberof originate. conceptshavearticlesinboththeEnglishandJapaneseeditions(Both). Within the geotag sample, the top-ranked English-only arti- clesweremainlyaboutUSmilitaryfacilities,equipment,and size(inbytes)beforeandafteranedit. Incontrast,thesizes historicbattles,whilethetop-rankedgeotaggedJapanesearti- computedbyWikiTrustandusedinthispaperaredetermined clesincludedavarietyofparks,touristareas,ports/terminals, bythenumberofwordsadded, deleted, changed, ormoved. andashrine,reflectingthatOkinawaisaJapanesetouristhot Newwordsanddeletedwordscontributeonepointeach,re- spot. placement words contribute 0.5 points each, and moving a wordafractionxofthenormalizedpagelength(0<x<1) The top-ranked articles within the category and article link contributesxofapoint[2]. sampleswereverysimilartoeachother. Thetop-rankedar- ticles appearing only in English in the category and article WikiTrustwasdevelopedandtestedonlanguageswithspaces link samples included many articles related to karate, which betweenwords; however,Japaneseiswrittenwithoutspaces startedinOkinawa.Thetop-rankedarticlesappearingonlyin between words. Therefore, the Japanese text was first pre- Japanese in these samples included historic articles (articles processedtoaddspacesbetweenwordswithmecab,anopen- onthereversionofOkinawafromtheUStoJapanin1972)as source library for text segmentation, part-of-speech tagging, wellasarticlesaboutOkinawa-basedmediaandtransitcom- and morphological analysis of Japanese text.8 The Wiki- panies. Trust algorithm was then run twice: once over the articles intheEnglisharticlelinksampleandonceoverthearticlesin Given the substantial overlaps between the samples, the re- theJapanesearticlelinksample. Thereputationscoreswere mainderofthispaperusesonlythearticlelinksampletoin- computedonlyonthebasisofusers’editsinthedatasamples, vestigatetherolesofmultilingualusersinthespreadofcon- and not in the full multi-terabyte dumps of all of Wikipedia tentbetweentheJapaneseandEnglishlanguageeditions.The articles. articlelinksamplehasthebenefitsthatanarticlelinksample canbeformedforanyseedsetofarticlesandthatarticlelinks havebeenbetterstudiedonWikipedia[23]thancategories.In RESULTS Thissectionbeginsbydescribingthethreearticlesamplesin particular,Baoetal.[5]examinedthefrequencywithwhich ordertounderstandthearticlelandscapewithinwhichWiki- differentlanguageeditionsweremissingarticlelinks(thatis pediauserswereediting. Itthenaddresseseachofthethree the frequency with which a concept was mentioned without mainresearchquestionsinturn: a link to the article about the concept) and found that the English and Japanese editions were missing possible article 1. What articles do multilinguals edit in their non-primary linksatsimilarfrequencies. languages? (RQ1) 2. Whattypesofeditsdomultilingualusersmakeintheirnon- Articleselection primarylanguages? (RQ2) This subsection examines what articles users edited, with a 3. How valuable are the contributions by multilingual users particularemphasisonwhatarticlesmultilingualusersedited intheirnon-primarylanguages? (RQ3) in their non-primary languages in order to answer the first research question. Most editors were anonymous, had local All three article samples (geotag, category, and article link) accounts,orhadglobalaccountsthatprimarilyeditedthelan- showdifferencesintheconceptsrelatedtoOkinawacovered guage edition in question (Table 2). The prevalence of edits intheJapaneseandEnglisheditionsofWikipedia.Consistent byanonymoususersisconsistentwithpreviousresearchan- withpreviousresearch[18],therearemoreconceptswithan alyzing Wikipedia and not a peculiarity of this sample [4]. articleinonlyonelanguage(eitherJapaneseorEnglish)than It is difficult to calculate per-user statistics for anonymous conceptswitharticlesinbothlanguages(Table1). Thethree userssinceIPaddresseschangeovertimeandmultipleusers sampleshavesubstantialoverlap—allbutasmallhandfulof mayeditfromthesameIPaddress,butrelevantstatisticsfor articlesinthegeotagsamplearepresentinthecategorysam- anonymoususersarepresentedwherepossiblegiventhesize ple,andthearticlelinksamplecontainsallthearticlesinboth ofthisgroup. thegeotagsampleandthecategorysample. Asmallnumberofregisteredusersprimarilyeditedeitherthe In order to investigate the differences between the editions JapaneseorEnglishedition,butalsoeditedtheoppositeedi- further, the inter-article links that connect articles together tion:558primaryeditorsoftheEnglisheditioneditedarticles within each language were used to construct two networks inthearticlelinksamplefromtheJapaneseedition,and466 for each sample. One network for the Japanese edition with primaryeditorsoftheJapaneseeditionalsoeditedarticlesin 8https://code.google.com/p/mecab/(inJapanese) thearticlelinksamplefromtheEnglishedition. 4 Totalusers Totalarticlesedited Articleseditedperuser Editsizeperuser(log) Count % Count % Median Mean SD Median Mean SD Englishedition Anonymous 192,839 73.4% 216,840 46.15% Localaccount 15,008 5.7% 58,689 12.49% 1 3.91 21.79 1.79 1.97 1.93 Pri. English 50,038 19.0% 179,951 38.30% 1 3.60 20.74 1.94 2.08 1.97 Pri. Japanese 466 0.2% 1,488 0.32% 1 3.19 7.32 1.16 1.38 1.67 Pri. Other 4,341 1.7% 12,911 2.75% 1 2.97 16.44 0.47 1.13 1.71 Totals 262,692 100.0% 469,879 100.0% 1 3.62 20.67 1.84 2.00 1.96 Japaneseedition Anonymous 372,852 88.4% 717,608 62.74% Localaccount 9,945 2.4% 109,765 9.60% 2 11.04 47.84 3.09 2.95 1.81 Pri. English 558 0.1% 5,531 0.48% 1 9.91 58.47 0.96 1.55 1.95 Pri. Japanese 37,191 8.8% 301,980 26.40% 1 8.12 43.97 3.00 2.91 1.83 Pri. Other 1,174 0.3% 8,954 0.78% 1 7.63 57.44 0.18 1.07 1.76 Totals 421,720 100.0% 1,143,838 100.0% 1 8.72 45.35 2.97 2.87 1.85 Table2. Usercounts,articlesedited,andeditsizes. Theprimary(pri.) languageofauserwithaglobalaccountisthelanguageofthemost-edited editionofWikipedia. The finding that multilingual users mostly edit articles with corresponding articles in their primary languages is con- 69% 31% firmedbyalinearregression(Table3),whichalsoshowsthe 89% 11% articlesuserseditedintheirnon-primarylanguagestendedto havemoreoveralledits/editors,moreimages,andhavehigher 64% 36% PageRankscorescomputedasdescribedearlier. Thearticles 75% 25% Japanese users edited in English also tended to have more linkstoexternalsources, butthenumberoflinkstoexternal sources was not significantly associated with the number of 64% 36% EnglishuserseditinganarticleinJapanese. These results give a more nuanced understanding of the ar- 58% 42% ticle selection behavior of multilingual users. Data on the 89% 11% articles that users viewed is not available, and thus it is not possible to say whether multilingual users read (but did not 55% 45% edit) articles in their primary languages before editing the corresponding articles in their non-primary languages. The editing data, however, clearly shows that while multilingual users in this dataset edit similar proportions as other user groups of one- and two-language concepts in their primary languages, they disproportionately edit a smaller amount of one-languageconceptsintheirnon-primarylanguages. Figure1. EnglishuserseditingtheJapaneseeditionarefarlesslikely thanotheruserstoeditarticlesthatonlyappearinJapanese.Similarly, Typesofcontributions JapaneseuserseditingtheEnglisheditionarefarlesslikelythanother userstoeditarticlesthatonlyappearinEnglish. Thissubsectionaddressesthesecondresearchquestiononthe sizeandtypeofeditsmultilingualusersmake. Settingaside anonymous users and local accounts to compare users who Thearticlesuserschoosetoeditreflecthowdifferentgroups have global accounts to one another, a finding in both edi- of users distribute their time and energy across articles. Al- tionsisthatthoseuserswhoeditedarticlesinbothJapanese though the previous section showed that most concepts had and English in the dataset were very active on their primary articlesinonlyonelanguage,themajorityofeditsbyallusers language editions. Considering users with global accounts were to concepts with articles in both languages (Figure 1). who primarily edited the English edition, about one percent Even so, the edits by multilingual users writing in a non- of these users also edited the Japanese edition. However, primary language were significantly more concentrated on this group of users was responsible for six percent of all conceptswithcorrespondingarticlesinbothlanguagescom- edits to the English edition made by English users. Simi- paredtotheeditsofotheruserswritingineachlanguage.9 larly,JapaneseuserswhoalsoeditedtheEnglisheditionwere also about one percent of all Japanese users with global ac- 9Thedifferenceinmeansbetweenanytwogroupswithineitheredi- counts, but they made 13% of all edits by Japanese users in tionissignificantwithp<0.001. theJapaneseedition. 5 #ofJapaneseuserseditingEnglish #ofEnglishuserseditingJapanese Estimate (Standarderror) Estimate (Standarderror) Existsinbothlanguages 0.641∗∗∗ (0.024) 3.285∗∗∗ (0.034) Totalnumberofeditors 0.001∗∗∗ (0.0001) 0.003∗∗∗ (0.0001) PageRank 0.014∗∗∗ (0.0005) 0.245∗∗∗ (0.006) Numberofimages 0.003∗∗∗ (0.001) 0.054∗∗∗ (0.002) Numberofexternallinks 0.001∗∗∗ (0.0003) −0.0003 (0.0004) Constant 0.008 (0.015) 0.029 (0.019) Observations 5,441 14,825 AdjustedR2 0.348 0.572 ResidualStd. Error 0.849(df=5435) 1.828(df=14819) ∗p<0.1;∗∗p<0.05;∗∗∗p<0.01 Table3. LinearregressionresultsfittingthenumberofprimaryJapaneseuserseditingeachEnglisharticleandthenumberofprimaryEnglishusers editingeachJapanesearticle. Figure2.Densityplotsfornon-anonymoususerseditingarticlesintheJapanese(left)andEnglish(right)editionsgroupedbytheirprimarylanguage editions.Verticallinesindicatedistributionmeans. While multilingual users were very active in their primary edittheEnglishedition(median3.9vs.3.0,mean3.8vs.2.9, languages,asmallerpercentageofeachuser’stotaleditswere sd both 1.8, p < 0.001). Likewise, English users who also toeachuser’snon-primarylanguages.Overall,14%ofalled- editedtheJapaneseeditionmadelargersizededitsinEnglish itsbymultilingualusersweretotheirnon-primarylanguages, comparedtoEnglishuserswhodidnotedittheJapaneseedi- whichishigherthanthepriorworkthatfoundonly2.6%of tion (median 2.4 vs. 1.9, mean 2.5 vs. 2.1, sd 1.9 and 2.0, edits across all language editions for one month were from p<0.001). userswritingintheirnon-primarylanguages[16]. Thiscould While multilingual users made larger sized edits in their be due to the focus on a geographic region with large num- primary languages, the analysis of their edits in their non- bersofEnglishandJapanesespeakersand/orthelongertime primary languages reveals a very different picture (Figure 2 frameoftheanalysis. and Table 2). The sizes of edits to the Japanese edition by Editsizes Englishusersweresignificantlysmallerthanthesizesofed- UserswhoeditedboththeJapaneseandEnglisheditionshave its to the Japanese edition by Japanese users (p < 0.001). twosetsofscoresfortheireditsizesandeditpersistence.One Similarly,thesizesofeditstotheEnglisheditionbyJapanese setfortheEnglisheditionandonesetfortheJapaneseedition. usersweresignificantlysmallerthanthesizesofeditstothe First looking at the scores for users’ primary language edi- EnglisheditionbyEnglishusers(p<0.001). tions,aconsistentfindinginbothJapaneseandEnglishisthat Contentchangesinprimaryandnon-primarylanguages thoseuserswhoeditedbotheditionsmadesignificantlylarger In order to better understand the gap between the edit sizes editsthantheuserswhoonlyeditedoneedition.10 Japanese ofusersintheirprimaryandnon-primarylanguages,asmall users who also edited the English edition made larger sized subset of edits was explored qualitatively. A random set of edits in Japanese compared to Japanese users who did not 70 users who had edited both editions was chosen: 35 users 10Giventheheavy-taileddistributionofaverageeditsizes,theresults who primarily edited the English edition and 35 users who arereportedafterbeingtransformedwiththelogarithm. primarily edited the Japanese edition. Up to five edits from 6 Editcategory Pri.lang. Non-pri.lang. p-val† worthy that the task of locating a related article and linking itacrosslanguageswasmotivatingenoughforsomeusersto Addition 97 31% 47 26% 0.25 editaforeignlanguageedition. Maintenance 103 33% 44 24% 0.04 Deletion/Reversion 37 12% 11 6% 0.03 Theproportionofadditioneditsandchangeeditsdidnotdif- Image-related 27 9% 32 18% 0.01 fersignificantlybetweenusers’primaryandnon-primarylan- Interlanguagelinks 8 3% 32 18% 0.00 guages. Overall, 15% of the edits made by Japanese users Change 65 21% 34 19% 0.62 to the English edition concerned fixing incorrect romaniza- tionsofJapanesewordsand/oraddingJapanesecharactersfor Totaledits‡ 315 181 terms.Thesetypesoflanguage-specificeditsthatareeasyfor Table4. Exploratory,qualitativecodingofeditsinusers’primarylan- native speakers but harder for non-native speakers illustrate guages(pri.lang.)andnon-primarylanguages(non-pri.lang.). boththevalueofcross-languagecollaborationandalsowhy †p-valuesarefortwo-tailedt-testsondifferenceofpercentagemeans. theseusersmayhavebeenmakingeditsofdifferenttypesin ‡Someeditsareassignedtomultiplecategoriesand,therefore,thecol- umnsumsaregreaterthanthetotalnumberofeditsreported. theirprimaryandnon-primarylanguages. Valueofedits eacheditionwererandomlychosenforatotalofupto10edits AsstatedintheDatasection, therearemanywaysinwhich fromeachuser. Despitethemeasurestoremove(ro)botsde- userscontributevaluetoWikipedia,butacommon,quantita- scribed in the data section, qualitative analysis revealed that tive measure on which to compare the many different types onerandomlychosenuserwasabot: thisuserwasreplaced ofcontributionsishowmuchofeacheditisretainedthrough with another randomly chosen user. Not all users had five subsequentrevisionstothearticle. UsingtheWikiTrustalgo- edits in each edition; so, 496 edits were reviewed in total. rithms, the next six revisions after each edit were examined ThissetincludededitsbyEnglishuserstotheEnglishedition to compute how much of the edit was retained (persisted) (145 edits) and the Japanese edition (96) as well as edits by through these revisions. Each edit was given a normalized Japanese users to the English edition (85) and the Japanese scorefrom-1(editcompletelyremoved)to1(editcompletely edition (170). The findings suggest that users made differ- retained). Comparingthemeaneditpersistentscoresshowed ent types of contributions in their primary and non-primary thatthetextfromeditsmadebynon-primaryeditorssurvived languages,whichmayaccountforthedifferencesinthecom- atasimilarratetothetextfromeditsmadebyuserswhopri- putedsizeoftheiredits. marilyeditedeachedition.11 Edits made to articles (but not to talk pages, etc.) were examined in order to understand the contributions users made DISCUSSION to articles in their primary and non-primary languages. Af- The large differences in content observed between different terconsultingpreviousliterature[22, 28], initialcodeswere editions of Wikipedia globally [18] also applied to articles created through an emergent coding of a subset of the data. relatedtoOkinawadespitethepresenceofJapaneseandEn- These were refined into six (non-exclusive) categories, and glish speakers living on the island. Similarly, the small per- the full sample was systematically coded using these cate- centage of users editing multiple editions of Wikipedia [16] gories.Eacheditwasclassifiedasmakinganaddition(adding applied to this dataset as well. In many ways, Okinawa is newtextorreferencestoanexistingarticleorcreatinganew ahardcase: JapaneseandEnglishuseverydifferentwriting article),asmaintenance(adding,removing,oradjustingtem- systems,andJapaneseusershaveconsistentlybeenobserved plates,categories,linksina“SeeAlso”section,orwhitespace toengagelesswithother-languagecontentnotonlyonWiki- changesthatdidnotaltertext),asdeletion/reversion(revert- pedia[16],butalsoonTwitter[15]. inganeditordeletingtextfromanarticle),asimage-related Nonetheless,thisworkhasshedgreaterlightontheselection (adding, altering, or removing an image), as altering inter- ofarticlesmultilingualusersinthesetwolanguagesedit,the languagelinks,and/oraschange(editsthatchangedexisting typesofcontributionstheymake,andthevalueoftheseedits. text such as correcting spelling errors or updating facts that If further research confirms the patterns found in this paper hadchangedlikethelatestwinnerofanannualsportstourna- applymorebroadly, thenonekeychallengefordesignersof ment). multilingualplatformsseekingtofacilitatecross-languagein- There was a significant difference between the types of ed- formation exchange is increasing multilingual users’ aware- its users made in their primary languages compared to the ness of and contributions to related other-language content. types they made in their non-primary languages (χ2 = Themultilingualusersinthisstudywerefarlesslikelytoedit 48,p<0.001,Table4). Usersmadesignificantlymoredele- articlesintheirnon-primarylanguagesthatdidnothavecor- tions/reversions and maintenance edits in their primary lan- responding articles in their primary languages despite these guages compared to in their non-primary languages. On the 11The average score for Japanese users editing the English edition other hand, users made significantly more image-related ed- (median0.44,mean0.38,sd0.58)washigherbutnotsignificantly itsandadded/removedsignificantlymoreinterlanguagelinks differentfromtheaveragescoreforEnglishuserseditingtheEnglish intheirnon-primarylanguagescomparedtointheirprimary edition(median0.47,mean0.35,sd0.64,p=0.46). Similarly,the average score for English users editing the Japanese edition (me- languages. Thefindingsrelatedtointerlanguagelinksareno dian0.45,mean0.44,sd0.53)wasmarginallyhigherbutnotsignif- longer applicable to Wikipedia as these links are now main- icantly different from the average score for Japanese users editing tained separately within WikiData. Nonetheless, it is note- theJapaneseedition(median0.37,mean0.43,sd0.50,p=0.72). 7 articles being more numerous. The large difference in con- multilingual users more frequently edited images and inter- tent between languages applies not only to the sample used language links in their non-primary languages. In contrast, for this study, but also overall on Wikipedia [18], on Twit- they made more maintenance and deletion/reversion edits in ter[19],andlikelytomostotheruser-generatedcontentplat- theirprimarylanguages. forms. Thechallengeofmakingusersawareofcontentavail- Eveniftheeditsinusers’non-primarylanguagesaresmaller ableintheirnon-primarylanguagesbutnotintheirprimarily and of a different type, they still have value. The qualita- languagesisthusachallengelikelytobefacedbydesigners tiveexplorationofeditsrevealedmanyexamplesofusersup- ofallmultilingualuser-generatedcontentplatforms. dating out-of-date information and correcting errors in their Furtherresearchwillbeneededtounderstandthepreciseim- non-primarylanguages. Japaneseusersalsofrequentlyadded plications of design on user content selection, but it seems or corrected relevant Japanese-language text in the English likelythattheprominenceofinterlanguagelinksconnecting edition. There were also edits of addition and, occasionally, relatedarticlesacrosslanguagesonWikipediaandthelackof translationintousers’non-primarylanguages. other-language discovery tools are partially responsible for The Language Engineering team of the Wikimedia Founda- the narrow scope of multilingual users’ edits in their non- tion12 has been actively developing an (open-source) con- primary languages. Currently, for example, there is no fa- tent translation tool to help users translate content between cility to search multiple language editions of Wikipedia si- different language editions of Wikipedia.13 While the in- multaneously. So, if a user generally searches only in his tegration of machine translation and bilingual dictionaries, primarylanguage,thatusermaynotdiscoverthecontentthat the automatic conversion of article links, and the stream- exists in another language if the content has no correspond- lined user-interface of the translation tool will no doubt as- ingarticleinhisprimarylanguageeveniftheuserreadsthe sist would-be translators, this research indicates that help- other language. Very often the full text of articles in the ing users find articles they want to translate will be a major Japanese edition includes an English translation of the arti- hurdle. Multilingual users in this study clearly made their cle’sconcept,andlikewiseJapanesetermsareoftenincluded largest-sized contributions in their primary languages, sug- in Japanese-themed articles in the English edition. If the gesting that platforms might be more successful in encour- search interface automatically checked a user’s non-primary agingtranslationfromusers’non-primarylanguagestotheir languageeditionswhennomatcheswerefoundintheuser’s primarylanguagesratherthanfromusers’primarylanguages primarylanguageedition,thatusermightdiscoverarticlesin totheirnon-primarylanguages. Thiswouldrequiresurfacing hisnon-primarylanguagesthathavenoarticleinhisprimary contentinusers’non-primarylanguagesthatdoesnotexistin language. theusers’primarylanguagesevenwhiletheusersareviewing Anotherpossibledesignchangewouldbetosuggestarticles contentintheirprimarylanguages. related to a specific theme that exist in users’ non-primary Beyond translation, there are a range of contributions users languages, but not in their primary languages. Such an ap- can make on multilingual user-generated content platforms proach could employ similar methods to those used here: that require a varying level of cross-language proficiency. gathering all articles linking to a given article and comput- Offlinedataonmultilingualismisimperfectandincomplete, ingthePageRankscoresorothermethodslikeLatentDirich- but best estimates suggest around half of all humans speak let Allocation (LDA) [17]. In practice, this might look very two or more languages [12]. Thus, it might be possible to similar to the valuable SuggestBot [9] tool, which can rec- encourage far more users to contribute content in multiple ommend articles for users to edit based on the articles the languagesonuser-generatedcontentplatforms. Surveywork usershavepreviouslyeditedinonelanguage. Currently,sep- suggeststhatInternetusersconsumecontentinmultiplelan- arate versions of SuggestBot run independently in multiple guagesmorefrequentlythantheycontributecontentinmulti- languages, and a potentially useful (although certainly non- ple languages (not only on Wikipedia,14 but also more gen- trivial) step would be to extend SuggestBot to offer sugges- erally online [11]). The prevalence of image-related edits tions across user-selected (or inferred) languages for users inthisstudyandtheapparentmotivationimagesprovidefor who desire such suggestions. That is, based on the articles cross-language linking in the blogosphere [14] suggest mul- a user has previously edited in one language, the user could timedia content is a low-barrier entry point for increasing ask for articles in another language that either need work or multilingual contributions from users. Designers of multi- thathavenoequivalentintheuser’sprimarylanguage. lingualuser-generatedcontentplatformswishingtoincrease Asmall,butdedicatedgroupofusers,whomadelarge-sized multilingualactivitycouldspecificallyconsiderandoptimize editsintheirprimarylanguages,alsoeditedarticlesinasec- cross-language multimedia content related tasks and other ondlanguage. Theeditsintheusers’non-primarylanguages low-barrierentrypointsformultilingualcontributionsasone were smaller in size, but were equally valued by the site’s userspersistingthroughsubsequentrevisionsatasimilarrate to the edits made by users editing only one language edi- 12https://wikimediafoundation.org/wiki/Language_ Engineering_team tion. Anexploratoryanalysisoftheeditsusersmadeintheir 13https://www.mediawiki.org/wiki/Content_ primaryandnon-primarylanguagesindicatedthatthediffer- translation encesineditsizeswerepartiallyduetousersmakingdifferent 14https://meta.wikimedia.org/w/index.php?title= typesofeditsintheirprimaryandnon-primarylanguages— Editor_Survey_2011/Location_%26_Language&oldid= 8409990 8 possiblewaytoincreasethenumberofuserscontributingin WWW’07,ACM(NewYork,NY,USA,2007), multiplelanguagesontheirsites. 261–270. Adler et al. [1, 2, 3] advocate combining together the mea- 3. Adler,B.T.,deAlfaro,L.,Pye,I.,andRaman,V. sures of edit size and edit persistence to form a reputation MeasuringauthorcontributionstotheWikipedia.In scoreforeachuser, whichtheirWikiTrustworkusestopre- Proceedingsofthe4thInternationalSymposiumon dict the quality or trustworthiness of users’ contributions to Wikis,WikiSym’08,ACM(NewYork,NY,USA, Wikipedia. If such a system evaluated the contributions of 2008),15:1–15:10. multilingual users separately for each language, most multilingual users would have low reputation scores in their non- 4. Anthony,D.,Smith,S.W.,andWilliamson,T. primarylanguagesduetothesmallersizesandsmallernum- Reputationandreliabilityincollectivegoods: Thecase ber of their edits while they would have large reputation oftheonlineencyclopediaWikipedia.Rationalityand scores in their primary languages. The analysis in this pa- Society21,3(2009),283–306. per, therefore, shows the importance of evaluating multilin- 5. Bao,P.,Hecht,B.,Carton,S.,Quaderi,M.,Horn,M., gual users holistically across their multiple languages to ac- andGergle,D.Omnipedia: BridgingtheWikipedia curatelymeasuretheircontributionstouser-generatedcontent languagegap.InProceedingsoftheSIGCHIConference sites. onHumanFactorsinComputingSystems,CHI’12, Successfully combating the risk of fragmenting users and ACM(NewYork,NY,USA,2012),1075–1084. content too thinly across multiple languages on user- generatedplatformsrequiresamoreadvancedunderstanding 6. Bilic,P.,andBulian,L.Lostintranslation: Contexts, of localization and internationalization than traditional prin- computing,disputingonWikipedia.IniConference cipalssuchastranslatinginterfacesto“speaktheuser’slan- 2014Proceedings,M.KindlingandE.Greifender,Eds. guage(s)” [25]. Multilingual users need to be specifically iSchools,2014,32–44. considered in site design. This involves not only making 7. Burt,R.S.Structuralholesandgoodideas.The existingcross-languageconnectionsvisible,butalsodesign- AmericanJournalofSociology110,2(2004),349–399. ingforthediscoveryofrelatedforeign-languagecontentnot available in users’ primary languages. The most active and 8. Callahan,E.S.,andHerring,S.C.Culturalbiasin dedicateduserswillreachacrosslanguageboundariesonline Wikipediacontentonfamouspersons.Journalofthe to contribute to other-language content, but the users in this AmericanSocietyforInformationScienceand study made their largest-sized contributions in their primary Technology62,10(2011),1899–1915. languages. Thus,successfulmultilingualuser-generatedcontent platforms need to nurture both active, dedicated mono- 9. Cosley,D.,Frankowski,D.,Terveen,L.,andRiedl,J. lingualcommunitiesandalsoencouragemultilingualusersto Suggestbot: Usingintelligenttaskroutingtohelppeople discoverother-languagecontentandserveasbridgesbetween findworkinwikipedia.InProceedingsofthe12th monolingual communities. Further research should analyze InternationalConferenceonIntelligentUserInterfaces, theextenttowhichmultilingualsearch, cross-languagecon- IUI’07,ACM(NewYork,NY,USA,2007),32–41. tentrecommendation,andoptimizedlow-barrierentrypoints 10. Eleta,I.,andGolbeck,J.Bridginglanguagesinsocial formultilingualcontributioncanhelpmultilingualusersbet- networks: HowmultilingualusersofTwitterconnect terunderstandthedifferencesincontentbetweenlanguages, languagecommunities.ProceedingsoftheAmerican encourage these users to transfer more information between SocietyforInformationScienceandTechnology49,1 different languages, and thereby enable wider access for all (2012),1–4. users to the most interesting and important material that is notyetintheirprimarylanguages. 11. Eurobarometer.Userlanguagepreferencesonline: Analyticalreport.Tech.rep.,2011.Surveyconductedby ACKNOWLEDGMENTS TheGallupOrganization,Hungaryupontherequestof IwouldliketothankEricT.Meyer,TahaYasseri,andAlolita Directorate-GeneralInformationSocietyandMedia, Sharma as well as the anonymous reviewers who provided EuropeanCommission. helpfulcommentsonpreviousversionsofthisarticle. 12. Grosjean,F.Bilingual: LifeandReality.Harvard UniversityPress,2010. REFERENCES 1. Adler,B.T.,Chatterjee,K.,deAlfaro,L.,Faella,M., 13. Hale,S.A.Impactofplatformdesignoncross-language Pye,I.,andRaman,V.AssigningtrusttoWikipedia informationexchange.InProceedingsofthe2012ACM content.InProceedingsofthe4thInternational AnnualConferenceonHumanFactorsinComputing SymposiumonWikis,WikiSym’08,ACM(NewYork, SystemsExtendedAbstracts,CHIEA’12,ACM(New NY,USA,2008),26:1–26:12. York,NY,USA,2012),1363–1368. 2. Adler,B.T.,anddeAlfaro,L.Acontent-driven 14. Hale,S.A.Netincrease? Cross-linguallinkinginthe reputationsystemfortheWikipedia.InProceedingsof blogosphere.JournalofComputer-Mediated the16thInternationalConferenceonWorldWideWeb, Communication17,2(2012),135–151. 9 15. Hale,S.A.Globalconnectivityandmultilingualsinthe 23. Milne,D.,andWitten,I.H.Aneffective,low-cost Twitternetwork.InProceedingsoftheSIGCHI measureofsemanticrelatednessobtainedfrom ConferenceonHumanFactorsinComputingSystems, Wikipedialinks.InInProceedingsoftheAAAI2008 CHI’14,ACM(NewYork,NY,USA,2014),833–842. WorkshoponWikipediaandArtificialIntelligence, WIKIAI2008,AAAI(Chicago,IL,USA,2008). 16. Hale,S.A.MultilingualsandWikipediaediting.In Proceedingsofthe6thAnnualACMWebScience 24. MinistryofForeignAffairsofJapan.米軍人等の施 Conference,WebSci’14,ACM(NewYork,NY,USA, 設・区域内外居住者の人数について(全国)(平 2014). 成20年3月31日時点)(Informationonthenumberof peoplelivingonandoffUSmilitaryrelatedfacilities 17. Haruechaiyasak,C.,andDamrongrat,C.Article [Nationwide][March31,2008]).http://www.mofa.go. recommendationbasedonatopicmodelforWikipedia jp/mofaj/press/release/h20/6/1181033_910.html, selectionforschools.InDigitalLibraries: Universal 2008. andUbiquitousAccesstoInformation,G.Buchanan, M.Masoodian,andS.Cunningham,Eds.,vol.5362of 25. Nielsen,J.UsabilityEngineering.AcademicPress, LectureNotesinComputerScience.SpringerBerlin 1993. Heidelberg,2008,339–342. 26. OkinawaPrefecture.沖縄の米軍基地及び自衛隊基地 18. Hecht,B.,andGergle,D.TheTowerofBabelmeets （統計資料集）(USmilitarybasesandJapan Web2.0: User-generatedcontentanditsapplicationsin Self-DefenseForcebasesinOkinawa[Compiled amultilingualcontext.InProceedingsofthe28th statistics]).http://www.pref.okinawa.jp/site/ InternationalConferenceonHumanFactorsin chijiko/kichitai/toukeisiryousyu2503.html,2013. ComputingSystems,CHI’10,ACM(NewYork,NY, 27. Page,L.,Brin,S.,Motwani,R.,andWinograd,T.The USA,2010),291–300. pagerankcitationranking: Bringingordertotheweb. 19. Hong,L.,Convertino,G.,andChi,E.Languagematters TechnicalReport1999-66,StanfordInfoLab,Nov.1999. inTwitter: Alargescalestudy.InInternationalAAAI 28. Pfeil,U.,Zaphiris,P.,andAng,C.S.Cultural ConferenceonWeblogsandSocialMedia(2011), differencesincollaborativeauthoringofWikipedia. 518–521. JournalofComputer-MediatedCommunication12,1 20. Kim,S.,Weber,I.,Wei,L.,andOh,A.Sociolinguistic (2006),88–113. analysisofTwitterinmultilingualsocieties.In 29. Priedhorsky,R.,Chen,J.,Lam,S.T.K.,Panciera,K., Proceedingsofthe25thACMConferenceonHypertext Terveen,L.,andRiedl,J.Creating,destroying,and andSocialMedia,HT’14,ACM(NewYork,NY,USA, restoringvalueinWikipedia.InProceedingsofthe2007 2014),243–248. InternationalACMConferenceonSupportingGroup 21. Kittur,A.,Chi,E.,Pendleton,B.A.,Suh,B.,and Work,GROUP’07,ACM(NewYork,NY,USA,2007), Mytkowicz,T.Powerofthefewvs.wisdomofthe 259–268. crowd: Wikipediaandtheriseofthebourgeoisie. 30. Welser,H.T.,Cosley,D.,Kossinets,G.,Lin,A., Presentedatalt.CHI,ACMSIGCHIConference,San Dokshin,F.,Gay,G.,andSmith,M.Findingsocialroles Jose,CA,USA,2007. inWikipedia.InProceedingsofthe2011iConference, 22. Kriplean,T.,Beschastnikh,I.,andMcDonald,D.W. iConference’11,ACM(NewYork,NY,USA,2011), Articulationsofwikiwork: Uncoveringvaluedworkin 122–129. Wikipediathroughbarnstars.InProceedingsofthe2008 31. Zuckerman,E.Rewire: DigitalCosmopolitansinthe ACMConferenceonComputerSupportedCooperative AgeofConnection.W.W.Norton&Company,London, Work,CSCW’08,ACM(NewYork,NY,USA,2008), 2013. 47–56. 10