Machine Learners: Archaeology of a Data Practice

267 Pages·2017·3.814 MB·English

by  Adrian Mackenzie

MachinLee arners
Archaeoloofga y D ataP ractice
AdriaMna ckenzie
TheM ITP ress
CambridMgaes,s achusetts
LondoEnn,g land
© 201M7a ssachuIsnestttisot fuT teec hnology
Alrli ghrtess ervNeopd a.r otft hibso omk ayb er eproduicnae ndy f orbmy a ny
electroornm iecc hanimceaaln (si nclupdhiontgo copyriencgo,r doirni gn,f ormation
storaagnedr etriewviatlh)op uetr missiinwo rni tifnrgo tmh ep ublisher.
Thibso owka ss eitn S tonSee rbiyfW estchePsutbelri sSheirnvgi acneds
wasp rintaenddb ounidn t heU nitSetd atoefAs m erica.
LibraorfCy o ngreCsast aloging-in-PDuabtlai cation
NamesM:a ckenzAider,i a1n9,6 2a-uthor.
TitlMea:c hilneea rn:ae rrcsh aeoolfoa gd ya tpar actIAi dcrei Maanc kenzie.
Description: CMaAm:Tb hreiM dIgTPe r,e s[s2,0 1I7I 1n clubdiebsl iographical
refereanncdei snd ex.
IdentifiLeCrCsN2: 0 17005I3I 4S3B 9N7 8026203(6h8a2r5d c:oa vlekpr.a per)
SubjecLtCsS:HI :n formatthieoonrI My a.c hilneea rning-PhiIl osophy.
Electrdoantipacr ocessing-Philosophy.
ClassificLaCtCiQ o3n6:0. M313240 171DD0C03/.54-dc23
LCr ecoarvda ilaabth ltet ps://lccn.loc.gov/2017005343
10 9 8 7 6 5 4 3 2 1
Contents
LiosftF igurveisi
LiosfTt a bleisx
Prefaxcie
Acknowledgxmve nts
1 IntroductIinotnto:h eD ata 1
2 DiagrammiMnagc hines2 1
3 VectorizaatnidIo ntC so nsequence5s1
4 MachineFsi ndiFnugn ction7s5
5 N= VX:P robabilizaantdit ohneT aminogf M achines1 03
6 PatterannsdD ifference1s2 5
7 RegulariaznidnM ga terialiOzbijnegc ts1 51
8 PropagatSiunbgj ePcots itio1n7s9
9 ConclusiOounto: f t heD ata 209
Glossa2r19y
Bibliogr22a3p hy
Inde2x4 3

LisotfF igures
1.1G oogTlree nsdesa rvcohl umfeom ra chilneea rning 3
1.2C ata sh istogorfga rma dients 4
1.3M achilneea rniensr csi enltiitfiecr ature 16
2.1E lemeonftl se arntiomn agc hilneea rn 28
2.2P agecsi tferdo Eml emenotfSs t atisLteiacmailn bgy a cadempiucb lications
ina lfile lds 33
2.3P ublicactiitoienndEs l emenotfSs t atisLteiacmailn g 35
2.4T hel ineraerg resmsoidoenl 41
2.5C lass note5s,S tlaencftCouSrr2de2 9, 2007 45
2.61 95p8e rcepatnrdo2 n0 0n1e urnaelct o mpared 47
3.1S catptleomrta troifpx r ostatdea ta 60
3.2V ectsopra cceo mpritsreasn sformations 66
4.1S eikti- learmna po fm achilneea rners 76
4.2L ogisotrsi icg mofiudn ction 87
4.3S outAhf ricHaena rDti seraesgeu laripzlaotti on 93
4.4S outAhf ricHaena rDti sedaescei spiloann e 94
4.5G radiaesncte fnotlr o girsetgirce ssion 98
4.6S tochagsrtaidcid eenstc epnatt h 99
6.1R ecurspiavret itiooftn hifene ga tsupraec e 128
6.2A ID classifier 130
6.3D ecistiroeonen i risd ataset 135
6.4S uppovretc tmoarc hionnei risd ataset 140
6.5M NISpTo stdailg istasm:p olfe" 6's" 143
6.6M argiinnas s uppovretc tmoarc hine 144
7.1A humagne nomdei agramumseidnC gi rcos 159
7.2H ierarcchliucsatleo rfti hneSgB RCTg eneex pression data 165
viii LisotfF igures
7. 3S hrinkpaagteh 170
7.4F ormulaotfki -onnmn o del 172
7.5k -nmno deflossr i muladtaetidan t woc lasses 175
8.1T echniqaunedcs o ncempotssft r equemnetnltyi oned 183
8.2T heb ack-propaaglagtoiroint hm 185
8.3N eurnaelt wotrokp olfoog3ry- hidudnein"t t itadnaitca" 196
8.4K aggdlaet sac iecnocmep etitions 202
LisotfT ables
1.1A smalsla mpolfet itolfes sc ienatritfiictc hlaeutss mea chilneea rning
inr elattio"o dni fference" 15
2.1T het ruttahb floetr h eB oolefaunn ctNiOoTn- ANtDr uth 24
2.2I teracthiavneig new eighatsas p erceptront hlNeeA aNrDfn usn ction 26
2.3S ubjceactte goorfri eesse aprucbhl icactiitoiEnlnseg m enotsfS tatistical
Leamin2g0 01-2015 32
2.4R p ackasguegsg esbtyte hdEe l emStatpLaecakrang e 39
2.5R p ackadgeepse ndoenbd y t he" ElemStatpLaecakrang"e 40
3.1D atasienEt lse menotsfS tatisLteiacmailn g 54
3.2F irrsotw osf t he"p rostatdea" taset 60
3.3F ittail nign emaord etlot hper ostatdea taset 71
4.1S ampolfeh ighcliyt meadc hilneea rnpiunbgl icarteifoenrstr oi ng
"functiinot ni"to lrke e yword 78
4.2S ampolfeh ighcliyt secdi enptuibfilci carteifoenrstr oi" nlgo gistic
regresisnit ointo"lr ke e yword 89
5.1S omes tructduirfifnegr iennm caecsh ilneea rning 107
5.2M oscti tNeadi vBea yepsu blications 1945-2015 117
6.1R eferetnoMc oersg aann dS onquiAsutt'osm aItnitce raDcettieocnt or 129
6.2M oscti tpeadp eornss uppovretc tmoarc hines 139
7.1T het op2 0d iscipolfti hnteeo sp5 00c0i treeds eaprucbhl ications
inm achilneea rn1i9n9g0,- 2015 157
7.2F ir5sr to wosf F ishe"irr'issd "a taset 162
7.3S malrlo und bltuuem-ocdrea ltslaa mp(lKeh a2n0,0 1) 164
8.1T heh ighepsrtim zoen emya chilneea rncionmgp etiotniK oangsg le 204

Preface
Althotuhgihs ibsno ootak n e thongrhaypi,th a asn e thnoghrisacip tuoanIt.ifi t h aas
fielsdi te,i tlti hceelsno tsote h pel acwehse re wtrhietw iansdg o ne-uinni versoint ies,
campusiencs l,a ssraonodom nsl itnrea incionugr (siensc lumdaissnigvo ep eonn line
caus[eMsO OC])s,a ndt heanm itdh e bodookcsu,m enwtesb,s istoefst,w maarneu als
andd ocumentaatnidao r na,t hvearsa tc cumulaotfsi coine nptuibfilcii ocanItst.a' s
casoef" diwgh eryeo us tnad"o r" uato-arcohgay".e ol
Readefmraisl iwairtt he xtboionck osm putsecri eanncdes taticsatni cdset theec t
tracoefts h ose ifine lvdasrt iyopuosg racpohninvcte iodnrsa wn frofime latdbhso eu t
whicIhw rite. Imcpoonervnttainiotnn csl ude:
1.Ty pesettthinena gm eo fa nyc odoer d evitcheast madcoh ilneea rnainnddg a ta­
setosnw hicmha chilneea rnoepresr aitnae m onospaceo rt ermifnoanlmt a:c hine
learnero ri ris;
2.P resenftoirnmgu lfuanec,t ioannsde, q uatiuosnisnt gh eb ristilnidnegxi itcoyaf l
fi
mathemattyipcoaglr aphy:
Ie mulattheae p paraotfsu csi eanncdee ngineepruibnlgi caasta ineo xnp eriment
ini n-shiybtur idinz.aS toicoisacli eanncdeh umanirteiseesa rcehveewrnhs e,nt hey
aroeb servpaanrtt iciipnta hnetfiiser l sdi treasr,ee lxyp eriaec nocien cidbeentcwee en
theoiwrn w ritpirnagc taincdte hsa otft hpea rticiipnta hnrete ss easrictthhe e y study.
Theo bjeocfst t udiynt hibso okh,o weviesra, k nowlepdrgaec ttihcadeto cuments
itsienlc fo dee,q uations, adnidsa tgartaemmsce,in rtcsu lianat retdi cbloeoska,sn ,d
varioonulsi fnoer ma(tbsi owgisk,is so,f twraerpeo siie.tsI o)itra sl spoo ssifbolrse o ac ial
rescehaertroa dopsto me tohfe se conveevnetinitf oh nessyo, m etitmaekuses t ot he
limiotfcs o hereanncde e,oa fs uinlyd erstbalenexd pearssion.
Iv'eb eewnr itciondgfe o r y(eMaarcsk e2n0z0i.6eW ) riticnogdw ea sn earallyw ays
somethdiitnsagnf tr owmr tiinagb oucto dbee cause wcaosad bionusgto ftwaorejec tpsr
xii Preface
andw ritwiansga boutth inkainndkg n olwed.g weIa ss lotwor ealtihzeeay r meu ch
entaendgR.le cednetv elopmienwn atysos f a nalyzainndpg u blisshciinegn dtaitfiac
bricnogd ianngdw ritcilnogs teorg etIhmeprl.ee nmticnogd cea nb ed onael moisnt
thes amsep acet,h esi anm sec reoerpn a nea,sw ritaibnogu cto d.Te hem ingloifn g
codianngdw ritaibnoguc to dber inagbso usto metigmeense rastoimveet,i fmreuss trat­
inge ncounwtietrvhsa riosucsi entific k(nmoawtlheedmgasett iactsi,cs otmipcusrt, e
scienwciet)ih,n frastrauncddt euvrieocsne m sa nys cal(ersa ngaicnrgo nsest works,
texetd itdoartsa,b haesreaesn dt herhea,r dwaanrdpe l atfoofrv masr ikoiunsd assw, e ll
asi nterfaancdme asn)yd omains.
Atm anyp oinitnrs e seartchhbeio nogkI d, i greasl soendg way intteoc hrneim­ote
cadlo maionfss t atisticaplr oibnafbetirhleoeirntycly,ei ,n aelagre bdryan,a mic models,
asw elals d atabdaessei agnndd atsat anrddasI.nt hei nteroefsc tosn necttihneg
manyp roopsitifoonrsm,u latdiioangsr,a ms, ecqiutaattiaionondnis sm,,a gienst his
bookm,u cho f tchoed Iev' ew rittienin m plenmteinmga chilneea rnmiondge olrs
reconstrcuecrttidanaigtn pa r acticenso tad popeeisa nrt hitse xjtsu,t a sn ota lolf
thew ordIs' vwer ittientn r yitnocg onstruacrtg umeonrtt sh ink adbaotpuart a c­
ticheasv bee einn clu.Md uecdhh as bceueatnw aayn dl eoftn t hger ound (although
thegi tr eposittohreby o oopkfr esemravneyst racoefts h ew ritianngdc odes;e e
htt:p/gsi/th.ucbo/mdatapraaccthiicnee/lme.aA rsin nit nhmgea) nym acihnlee arning
textboroekcsib,po eo kcso,o kbohookws-,t toust,ao lrasin,dm anuaIlh sa vree acdo,d e,
grahpicasn,dp rohsaev bee etni diheedr.M e anye xplorfaotroarayyrls e o satn da lmost
fortgoe.tn
Thes everal hyaevsaepr esn tI daonidwn rgi ting abporuatc thdiaacvtfeeae slut b­
stantdiiaflfleytr heanantn yo thperorej cbty v irtoufte h hey bridibzeattwieocenon d e
int exatn dt exitnc odeP.r actliytc,h aiilssm adep ossibbywl oer kionngc odaen d
texwti thtihnes amfiel ei,nt hes amtee xetd itSowri.t chbientgw ewerni tiR nagn d
Pythcoond (ea bout Iws haimyco hr bee lotwo)r etrideavtteao ,t ransfiottr,omp ro­
ducgera phitcocs o,n stmruocdte olrss o mkei nodfg rapihmiacg aen,dw ithtihnse a me
filbee w ritaicnagd epmriocs e, moingewh aty t bowe r iatbeo umta chilneea rnian g as
datpar accet.i
Thec apactiomt iyn gle ctoedxeat,n, di magdeesp enodnsa ne nsemobflo ep en
sourocfet,ce onm mand-sloifntewt aoroetl hsad ti fsfoemre whfartot mh tey piscoacli al
scienotrhi usmta nirteiseesa rcsohfetrwsta'or oeil tok fw orpdr ocesbsiobrl,i ographic
softwiamraeg,ee d itaonrdw, e bb rowsIenpr a.r ticIhu alvarere ,l iones do ftwpaarcek ­
ageisnt hRe p rogramming sluacnhgt uhaae"sgk n ei t r"( Xi2e01 3;X iea ndA llaire
2021)a ndi np ython,t hiep ythonn otebeonovki ron(mPeenraten zdG rang2e0r0 .7 )
Bothha vbee edne velobpyse cdi enatnidss ttast isitintc hiea nnsoa fm" er eproducible