Automated Data Collection with R A Practical Guide to Web Scraping and Text Mining | | | Simon Munzert Christian Rubba Peter Meißner Dominic Nyhuis Automated Data Collection with R Automated Data Collection with R A Practical Guide to Web Scraping and Text Mining Simon Munzert DepartmentofPoliticsandPublicAdministration,UniversityofKonstanz, Germany Christian Rubba DepartmentofPoliticalScience,UniversityofZurichandNationalCenterof CompetenceinResearch,Switzerland Peter Meißner DepartmentofPoliticsandPublicAdministration,UniversityofKonstanz, Germany Dominic Nyhuis DepartmentofPoliticalScience,UniversityofMannheim,Germany Thiseditionfirstpublished2015 ©2015JohnWiley&Sons,Ltd Registeredoffice JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,WestSussex,PO198SQ,UnitedKingdom Fordetailsofourglobaleditorialoffices,forcustomerservicesandforinformationabouthowtoapplyfor permissiontoreusethecopyrightmaterialinthisbookpleaseseeourwebsiteatwww.wiley.com. Therightoftheauthortobeidentifiedastheauthorofthisworkhasbeenassertedinaccordancewiththe Copyright,DesignsandPatentsAct1988. Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted,in anyformorbyanymeans,electronic,mechanical,photocopying,recordingorotherwise,exceptaspermittedby theUKCopyright,DesignsandPatentsAct1988,withoutthepriorpermissionofthepublisher. Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbe availableinelectronicbooks. Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.Allbrandnamesand productnamesusedinthisbookaretradenames,servicemarks,trademarksorregisteredtrademarksoftheir respectiveowners.Thepublisherisnotassociatedwithanyproductorvendormentionedinthisbook. LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbesteffortsinpreparing thisbook,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyorcompletenessofthecontents ofthisbookandspecificallydisclaimanyimpliedwarrantiesofmerchantabilityorfitnessforaparticularpurpose. Itissoldontheunderstandingthatthepublisherisnotengagedinrenderingprofessionalservicesandneitherthe publishernortheauthorshallbeliablefordamagesarisingherefrom.Ifprofessionaladviceorotherexpert assistanceisrequired,theservicesofacompetentprofessionalshouldbesought. LibraryofCongressCataloging-in-PublicationData Munzert,Simon. AutomateddatacollectionwithR:apracticalguidetowebscrapingandtextmining/SimonMunzert,Christian Rubba,PeterMeißner,DominicNyhuis. pagescm Summary:“Thisbookprovidesaunifiedframeworkofwebscrapingandinformationextractionfromtextdata withRforthesocialsciences”–Providedbypublisher. Includesbibliographicalreferencesandindex. ISBN978-1-118-83481-7(hardback) 1.Datamining. 2.Automaticdatacollectionsystems. 3.Socialsciences–Research–Dataprocessing. 4.R(Computerprogramlanguage) I.Title. QA76.9.D343M8652014 006.3′12–dc23 2014032266 AcataloguerecordforthisbookisavailablefromtheBritishLibrary. ISBN:9781118834817 Setin10/12ptTimesbyAptaraInc.,NewDelhi,India. 1 2015 Tomyparents,fortheirunendingsupport.Also,toStefanie. —Simon Tomyparents,fortheirloveandencouragement. —Christian ToKristin, Buddy,andPaulforlove, regularwalks,andafinaldeadline. —Peter MeinerFamilie. —Dominic Contents Preface xv 1 Introduction 1 1.1 Casestudy:WorldHeritageSitesinDanger 1 1.2 Someremarksonwebdataquality 7 1.3 Technologiesfordisseminating,extracting,andstoringwebdata 9 1.3.1 TechnologiesfordisseminatingcontentontheWeb 9 1.3.2 Technologiesforinformationextractionfromwebdocuments 11 1.3.3 Technologiesfordatastorage 12 1.4 Structureofthebook 13 PartOne APrimeronWebandDataTechnologies 15 2 HTML 17 2.1 Browserpresentationandsourcecode 18 2.2 Syntaxrules 19 2.2.1 Tags,elements,andattributes 20 2.2.2 Treestructure 21 2.2.3 Comments 22 2.2.4 Reservedandspecialcharacters 22 2.2.5 Documenttypedefinition 23 2.2.6 Spacesandlinebreaks 23 2.3 Tagsandattributes 24 2.3.1 Theanchortag 24 2.3.2 Themetadatatag 25 2.3.3 Theexternalreferencetag 26 2.3.4 Emphasizingtags,, 26 2.3.5 Theparagraphstag
27 2.3.6 Headingtags
,
,
,… 27 2.3.7 Listingcontentwith
,
,and
27 2.3.8 Theorganizationaltags
and 27 viii CONTENTS 2.3.9 The
taganditscompanions 29 2.3.10 Theforeignscripttag

Automated Data Collection with R

Similar Automated Data Collection with R