Statistical Rethinking: A Bayesian Course with Examples in R and STAN (Chapman & Hall/CRC Texts in Statistical Science)

612 Pages·2020·24.663 MB·English

by  Richard Mcelreath

Statistical Rethinking
CHAPMAN & HALL/CRC
Texts in Statistical Science Series
Joseph K. Blitzstein, Harvard University, USA
Julian J. Faraway, University of Bath, UK
Martin Tanner, Northwestern University, USA
Jim Zidek, University of British Columbia, Canada
Recently Published Titles
Theory of Spatial Statistics
A Concise Introduction
M.N.M van Lieshout
Bayesian Statistical Methods
Brian J. Reich and Sujit K. Ghosh
Sampling
Design and Analysis, Second Edition
Sharon L. Lohr
The Analysis of Time Series
An Introduction with R, Seventh Edition
Chris Chatfield and Haipeng Xing
Time Series
A Data Analysis Approach Using R
Robert H. Shumway and David S. Stoffer
Practical Multivariate Analysis, Sixth Edition
Abdelmonem Afifi, Susanne May, Robin A. Donatello, and Virginia A. Clark
Time Series: A First Course with Bootstrap Starter
Tucker S. McElroy and Dimitris N. Politis
Probability and Bayesian Modeling
Jim Albert and Jingchen Hu
Surrogates
Gaussian Process Modeling, Design, and Optimization for the Applied Sciences
Robert B. Gramacy
Statistical Analysis of Financial Data
With Examples in R
James Gentle
Statistical Rethinking
A Bayesian Course with Examples in R and Stan, Second Edition
Richard McElreath
For more information about this series, please visit: https://www.crcpress.com/Chapman–
HallCRC-Texts-in-Statistical-Science/book-series/CHTEXSTASCI
Statistical Rethinking
A Bayesian Course with Examples
in R and Stan
Second Edition
Richard McElreath
Second edition published 2020
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
© 2020 Taylor & Francis Group, LLC
First edition published by CRC Press 2015
CRC Press is an imprint of Taylor & Francis Group, LLC
Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have
attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders
if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please
write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized
in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying,
microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the
Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are
not available on CCC please contact [email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Library of Congress Cataloging‑in‑Publication Data
Library of Congress Control Number:2019957006
ISBN: 978-0-367-13991-9 (hbk)
ISBN: 978-0-429-02960-8 (ebk)
Contents
PrefacetotheSecondEdition ix
Preface xi
Audience xi
Teachingstrategy xii
Howtousethisbook xii
InstallingtherethinkingRpackage xvi
Acknowledgments xvi
Chapter1. TheGolemofPrague 1
1.1. Statisticalgolems 1
1.2. Statisticalrethinking 4
1.3. Toolsforgolemengineering 10
1.4. Summary 17
Chapter2. SmallWorldsandLargeWorlds 19
2.1. Thegardenofforkingdata 20
2.2. Buildingamodel 28
2.3. Componentsofthemodel 32
2.4. Makingthemodelgo 36
2.5. Summary 46
2.6. Practice 46
Chapter3. SamplingtheImaginary 49
3.1. Samplingfromagrid-approximateposterior 52
3.2. Samplingtosummarize 53
3.3. Samplingtosimulateprediction 61
3.4. Summary 68
3.5. Practice 68
Chapter4. GeocentricModels 71
4.1. Whynormaldistributionsarenormal 72
4.2. Alanguagefordescribingmodels 77
4.3. Gaussianmodelofheight 78
4.4. Linearprediction 91
4.5. Curvesfromlines 110
4.6. Summary 120
4.7. Practice 120
Chapter5. TheManyVariables&TheSpuriousWaffles 123
5.1. Spuriousassociation 125
5.2. Maskedrelationship 144
v
vi CONTENTS
5.3. Categoricalvariables 153
5.4. Summary 158
5.5. Practice 159
Chapter6. TheHauntedDAG&TheCausalTerror 161
6.1. Multicollinearity 163
6.2. Post-treatmentbias 170
6.3. Colliderbias 176
6.4. Confrontingconfounding 183
6.5. Summary 189
6.6. Practice 189
Chapter7. Ulysses’Compass 191
7.1. Theproblemwithparameters 193
7.2. Entropyandaccuracy 202
7.3. Golemtaming: regularization 214
7.4. Predictingpredictiveaccuracy 217
7.5. Modelcomparison 225
7.6. Summary 235
7.7. Practice 235
Chapter8. ConditionalManatees 237
8.1. Buildinganinteraction 239
8.2. Symmetryofinteractions 250
8.3. Continuousinteractions 252
8.4. Summary 260
8.5. Practice 260
Chapter9. MarkovChainMonteCarlo 263
9.1. GoodKingMarkovandhisislandkingdom 264
9.2. Metropolisalgorithms 267
9.3. HamiltonianMonteCarlo 270
9.4. EasyHMC:ulam 279
9.5. CareandfeedingofyourMarkovchain 287
9.6. Summary 296
9.7. Practice 296
Chapter10. BigEntropyandtheGeneralizedLinearModel 299
10.1. Maximumentropy 300
10.2. Generalizedlinearmodels 312
10.3. Maximumentropypriors 321
10.4. Summary 321
Chapter11. GodSpikedtheIntegers 323
11.1. Binomialregression 324
11.2. Poissonregression 345
11.3. Multinomialandcategoricalmodels 359
11.4. Summary 365
11.5. Practice 366
Chapter12. MonstersandMixtures 369
12.1. Over-dispersedcounts 369
12.2. Zero-inflatedoutcomes 376
CONTENTS vii
12.3. Orderedcategoricaloutcomes 380
12.4. Orderedcategoricalpredictors 391
12.5. Summary 397
12.6. Practice 397
Chapter13. ModelsWithMemory 399
13.1. Example: Multileveltadpoles 401
13.2. Varyingeffectsandtheunderfitting/overfittingtrade-off 408
13.3. Morethanonetypeofcluster 415
13.4. Divergenttransitionsandnon-centeredpriors 420
13.5. Multilevelposteriorpredictions 426
13.6. Summary 431
13.7. Practice 431
Chapter14. AdventuresinCovariance 435
14.1. Varyingslopesbyconstruction 437
14.2. Advancedvaryingslopes 447
14.3. Instrumentsandcausaldesigns 455
14.4. Socialrelationsascorrelatedvaryingeffects 462
14.5. ContinuouscategoriesandtheGaussianprocess 467
14.6. Summary 485
14.7. Practice 485
Chapter15. MissingDataandOtherOpportunities 489
15.1. Measurementerror 491
15.2. Missingdata 499
15.3. Categoricalerrorsanddiscreteabsences 516
15.4. Summary 521
15.5. Practice 521
Chapter16. GeneralizedLinearMadness 525
16.1. Geometricpeople 526
16.2. Hiddenmindsandobservedbehavior 531
16.3. Ordinarydifferentialnutcracking 536
16.4. Populationdynamics 541
16.5. Summary 550
16.6. Practice 550
Chapter17. Horoscopes 553
Endnotes 557
Bibliography 573
Citationindex 585
Topicindex 589

Preface to the Second Edition
ItcameasacompletesurprisetomethatIwroteastatisticsbook. Itisevenmoresur-
prisinghowpopularthebookhasbecome. ButIhadsetouttowritethestatisticsbookthat
IwishIcouldhavehadingraduateschool. NooneshouldhavetolearnthisstuffthewayI
did. Iamgladthereisanaudiencetobenefitfromthebook.
Itconsumedfiveyearstowriteit. Therewasaninitialsetofcoursenotes,melteddown
and hammered into a first 200-page manuscript. I discarded that first manuscript. But it
taughtmetheoutlineofthebookIreallywantedtowrite. Then, severalyearsofteaching
withthemanuscriptfurtherrefinedit.
Really,Icouldhavecontinuedrefiningiteveryyear. Goingtopresscarriesthepenaltyof
freezingadynamicprocessofbothlearninghowtoteachthematerialandkeepingupwith
changesinthematerial. Astimegoeson,IseemoreelementsofthebookthatIwishIhad
donedifferently. Iʼvealsoreceivedalotoffeedbackonthebook,andthatfeedbackhasgiven
meideasforimprovingit.
Sointhesecondedition,Iputthoseideasintoaction. Themajorchangesare:
The R package has some new tools. The map tool from the first edition is still here, but
nowitisnamedquap. Thisrenamingistoavoidmisunderstanding. Wejustusedittoget
aquadraticapproximationtotheposterior. Sonowitisnamedassuch. Abiggerchangeis
thatmap2stanhasbeenreplacedbyulam. Thenewulamisverysimilartomap2stan,and
in manycases canbe used identically. Butitisalso muchmoreflexible, mainlybecause it
doesnotmakeanyassumptionsaboutGLMstructureandallowsexplicitvariabletypes. All
themap2stancodeisstillinthepackageandwillcontinuetowork. Butnowulamallowsfor
muchmore,especiallyinlaterchapters. Bothofthesetoolsallowsamplingfromtheprior
distribution,usingextract.prior,aswellastheposterior. Thishelpswiththenextchange.
Much more prior predictive simulation. A prior predictive simulation means simulating
predictions from a model, using only the prior distribution instead of the posterior distri-
bution. Thisisveryusefulforunderstandingtheimplicationsofaprior. Therewasonlya
vestigialamountofthisinthefirstedition. Nowmanymodelingexampleshavesomeprior
predictivesimulation. Ithinkthisisoneofthemostusefuladditionstothesecondedition,
sinceithelpssomuchwithunderstandingnotonlypriorsbutalsothemodelitself.
Moreemphasisonthedistinctionbetweenpredictionandinference. Chapter5,thechap-
ter on multiple regression, has been split into two chapters. The first chapter focuses on
helpfulaspectsofregression;thesecondfocusesonwaysthatitcanmislead. Thisallowsas
wellamoredirectdiscussionofcausalinference. ThismeansthatDAGs—directedacyclic
ix