Statistical rethinking: a Bayesian course with examples in R and Stan

484 Pages·2016·11.089 MB·English

by  McElreath, Richard

Statistical
Rethinking
A Bayesian Course with
Examples in R and Stan
CHAPMAN & HALL/CRC
Texts in Statistical Science Series
Series Editors
Francesca Dominici, Harvard School of Public Health, USA
Julian J. Faraway, University of Bath, UK
Martin Tanner, Northwestern University, USA
Jim Zidek, University of British Columbia, Canada
Statistical Theory: A Concise Introduction Statistics for Technology: A Course in Applied
F. Abramovich and Y. Ritov Statistics, Third Edition
C. Chatfield
Practical Multivariate Analysis, Fifth Edition
A. Afifi, S. May, and V.A. Clark Bayesian Ideas and Data Analysis: An
Introduction for Scientists and Statisticians
Practical Statistics for Medical Research
D.G. Altman R. Christensen, W. Johnson, A. Branscum,
and T.E. Hanson
Interpreting Data: A First Course
in Statistics Modelling Binary Data, Second Edition
A.J.B. Anderson D. Collett
Introduction to Probability with R Modelling Survival Data in Medical Research,
K. Baclawski Third Edition
D. Collett
Linear Algebra and Matrix Analysis for
Statistics Introduction to Statistical Methods for
S. Banerjee and A. Roy Clinical Trials
T.D. Cook and D.L. DeMets
Mathematical Statistics: Basic Ideas and
Selected Topics, Volume I, Second Edition Applied Statistics: Principles and Examples
P. J. Bickel and K. A. Doksum D.R. Cox and E.J. Snell
Mathematical Statistics: Basic Ideas and Multivariate Survival Analysis and Competing
Selected Topics, Volume II Risks
P. J. Bickel and K. A. Doksum M. Crowder
Analysis of Categorical Data with R Statistical Analysis of Reliability Data
C. R. Bilder and T. M. Loughin M.J. Crowder, A.C. Kimber,
T.J. Sweeting, and R.L. Smith
Statistical Methods for SPC and TQM
D. Bissell An Introduction to Generalized
Linear Models, Third Edition
Introduction to Probability
A.J. Dobson and A.G. Barnett
J. K. Blitzstein and J. Hwang
Nonlinear Time Series: Theory, Methods, and
Bayesian Methods for Data Analysis,
Applications with R Examples
Third Edition
R. Douc, E. Moulines, and D.S. Stoffer
B.P. Carlin and T.A. Louis
Introduction to Optimization Methods and
Second Edition
Their Applications in Statistics
R. Caulcutt
B.S. Everitt
The Analysis of Time Series: An Introduction,
Extending the Linear Model with R:
Sixth Edition
Generalized Linear, Mixed Effects and
C. Chatfield
Nonparametric Regression Models
Introduction to Multivariate Analysis
J.J. Faraway
C. Chatfield and A.J. Collins
Linear Models with R, Second Edition
Problem Solving: A Statistician’s Guide,
J.J. Faraway
Second Edition
A Course in Large Sample Theory
C. Chatfield
T.S. Ferguson
Multivariate Statistics: A Practical Exercises and Solutions in Statistical Theory
Approach L.L. Kupper, B.H. Neelon, and S.M. O’Brien
B. Flury and H. Riedwyl
Design and Analysis of Experiments with R
Readings in Decision Analysis J. Lawson
S. French
Design and Analysis of Experiments with SAS
Markov Chain Monte Carlo: J. Lawson
Stochastic Simulation for Bayesian Inference,
A Course in Categorical Data Analysis
Second Edition
T. Leonard
D. Gamerman and H.F. Lopes
Statistics for Accountants
Bayesian Data Analysis, Third Edition
S. Letchford
A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson,
Introduction to the Theory of Statistical
A. Vehtari, and D.B. Rubin
Inference
Multivariate Analysis of Variance and
H. Liero and S. Zwanzig
Repeated Measures: A Practical Approach for
Statistical Theory, Fourth Edition
Behavioural Scientists
B.W. Lindgren
D.J. Hand and C.C. Taylor
Stationary Stochastic Processes: Theory and
Practical Longitudinal Data Analysis
Applications
D.J. Hand and M. Crowder
G. Lindgren
Logistic Regression Models
Statistics for Finance
J.M. Hilbe
E. Lindström, H. Madsen, and J. N. Nielsen
Richly Parameterized Linear Models:
The BUGS Book: A Practical Introduction to
Additive, Time Series, and Spatial Models
Bayesian Analysis
Using Random Effects
D. Lunn, C. Jackson, N. Best, A. Thomas, and
J.S. Hodges
D. Spiegelhalter
Statistics for Epidemiology
Introduction to General and Generalized
N.P. Jewell
Linear Models
Stochastic Processes: An Introduction,
H. Madsen and P. Thyregod
Second Edition
Time Series Analysis
P.W. Jones and P. Smith
H. Madsen
The Theory of Linear Models
Pólya Urn Models
B. Jørgensen
H. Mahmoud
Principles of Uncertainty
Randomization, Bootstrap and Monte Carlo
J.B. Kadane
Methods in Biology, Third Edition
Graphics for Statistics and Data Analysis with R
B.F.J. Manly
K.J. Keen
Introduction to Randomized Controlled
Mathematical Statistics
Clinical Trials, Second Edition
K. Knight
J.N.S. Matthews
Introduction to Multivariate Analysis:
Statistical Rethinking: A Bayesian Course with
Linear and Nonlinear Modeling
Examples in R and Stan
S. Konishi
R. McElreath
Nonparametric Methods in Statistics with SAS
Statistical Methods in Agriculture and
Applications
Experimental Biology, Second Edition
O. Korosteleva
R. Mead, R.N. Curnow, and A.M. Hasted
Modeling and Analysis of Stochastic Systems,
Statistics in Engineering: A Practical Approach
Second Edition
A.V. Metcalfe
V.G. Kulkarni
Statistical Inference: An Integrated Approach,
Exercises and Solutions in Biostatistical Theory
Second Edition
L.L. Kupper, B.H. Neelon, and S.M. O’Brien
H. S. Migon, D. Gamerman, and
F. Louzada
Beyond ANOVA: Basics of Applied Statistics Decision Analysis: A Bayesian Approach
R.G. Miller, Jr. J.Q. Smith
A Primer on Linear Models Analysis of Failure and Survival Data
J.F. Monahan P. J. Smith
Applied Stochastic Modelling, Second Edition Applied Statistics: Handbook of GENSTAT
B.J.T. Morgan Analyses
E.J. Snell and H. Simpson
Elements of Simulation
B.J.T. Morgan Applied Nonparametric Statistical Methods,
Fourth Edition
Probability: Methods and Measurement
P. Sprent and N.C. Smeeton
A. O’Hagan
Data Driven Statistical Methods
Introduction to Statistical Limit Theory
P. Sprent
A.M. Polansky
Generalized Linear Mixed Models:
Applied Bayesian Forecasting and Time Series
Modern Concepts, Methods and Applications
Analysis
W. W. Stroup
A. Pole, M. West, and J. Harrison
Survival Analysis Using S: Analysis of
Statistics in Research and Development,
Time-to-Event Data
Time Series: Modeling, Computation, and
M. Tableman and J.S. Kim
Inference
R. Prado and M. West Applied Categorical and Count Data Analysis
W. Tang, H. He, and X.M. Tu
Introduction to Statistical Process Control
P. Qiu Elementary Applications of Probability Theory,
Second Edition
Sampling Methodologies with Applications
H.C. Tuckwell
P.S.R.S. Rao
Introduction to Statistical Inference and Its
A First Course in Linear Model Theory
Applications with R
N. Ravishanker and D.K. Dey
M.W. Trosset
Essential Statistics, Fourth Edition
Understanding Advanced Statistical Methods
D.A.G. Rees
P.H. Westfall and K.S.S. Henning
Stochastic Modeling and Mathematical
Statistical Process Control: Theory and
Statistics: A Text for Statisticians and
Practice, Third Edition
Quantitative Scientists
G.B. Wetherill and D.W. Brown
F.J. Samaniego
Generalized Additive Models:
Statistical Methods for Spatial Data Analysis
An Introduction with R
O. Schabenberger and C.A. Gotway
S. Wood
Bayesian Networks: With Examples in R
Epidemiology: Study Design and
M. Scutari and J.-B. Denis
Data Analysis, Third Edition
Large Sample Methods in Statistics
M. Woodward
P.K. Sen and J. da Motta Singer
Practical Data Analysis for Designed
Spatio-Temporal Methods in Environmental
Experiments
Epidemiology
B.S. Yandell
G. Shaddick and J.V. Zidek
Texts in Statistical Science
Statistical
Rethinking
A Bayesian Course with
Examples in R and Stan
Richard McElreath
Max Planck Institute for Evolutionary Anthropology
Leipzig, Germany
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2016 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20150910
International Standard Book Number-13: 978-1-4822-5346-7 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Preface xi
Audience xi
Teachingstrategy xii
Howtousethisbook xii
InstallingtherethinkingRpackage xvi
Acknowledgments xvi
Chapter1. TheGolemofPrague 1
1.1. Statisticalgolems 1
1.2. Statisticalrethinking 4
1.3. Threetoolsforgolemengineering 10
1.4. Summary 16
Chapter2. SmallWorldsandLargeWorlds 19
2.1. Thegardenofforkingdata 20
2.2. Buildingamodel 28
2.3. Componentsofthemodel 32
2.4. Makingthemodelgo 37
2.5. Summary 45
2.6. Practice 45
Chapter3. SamplingtheImaginary 49
3.1. Samplingfromagrid-approximateposterior 52
3.2. Samplingtosummarize 53
3.3. Samplingtosimulateprediction 61
3.4. Summary 68
3.5. Practice 69
Chapter4. LinearModels 71
4.1. Whynormaldistributionsarenormal 72
4.2. Alanguagefordescribingmodels 77
4.3. AGaussianmodelofheight 78
4.4. Addingapredictor 92
4.5. Polynomialregression 110
4.6. Summary 115
4.7. Practice 115
Chapter5. MultivariateLinearModels 119
5.1. Spuriousassociation 121
5.2. Maskedrelationship 135
5.3. Whenaddingvariableshurts 141
vii
viii CONTENTS
5.4. Categoricalvariables 152
5.5. Ordinaryleastsquaresandlm 159
5.6. Summary 162
5.7. Practice 162
Chapter6. Overfitting,Regularization,andInformationCriteria 165
6.1. Theproblemwithparameters 167
6.2. Informationtheoryandmodelperformance 174
6.3. Regularization 186
6.4. Informationcriteria 188
6.5. Usinginformationcriteria 195
6.6. Summary 205
6.7. Practice 205
Chapter7. Interactions 209
7.1. Buildinganinteraction 211
7.2. Symmetryofthelinearinteraction 223
7.3. Continuousinteractions 225
7.4. Interactionsindesignformulas 235
7.5. Summary 236
7.6. Practice 236
Chapter8. MarkovChainMonteCarlo 241
8.1. GoodKingMarkovandHisislandkingdom 242
8.2. MarkovchainMonteCarlo 245
8.3. EasyHMC:map2stan 247
8.4. CareandfeedingofyourMarkovchain 255
8.5. Summary 263
8.6. Practice 263
Chapter9. BigEntropyandtheGeneralizedLinearModel 267
9.1. Maximumentropy 268
9.2. Generalizedlinearmodels 280
9.3. Maximumentropypriors 288
9.4. Summary 289
Chapter10. CountingandClassification 291
10.1. Binomialregression 292
10.2. Poissonregression 311
10.3. Othercountregressions 322
10.4. Summary 328
10.5. Practice 329
Chapter11. MonstersandMixtures 331
11.1. Orderedcategoricaloutcomes 331
11.2. Zero-inflatedoutcomes 342
11.3. Over-dispersedoutcomes 346
11.4. Summary 351
11.5. Practice 352
Chapter12. MultilevelModels 355
12.1. Example: Multileveltadpoles 357
12.2. Varyingeffectsandtheunderfitting/overfittingtrade-off 364
CONTENTS ix
12.3. Morethanonetypeofcluster 370
12.4. Multilevelposteriorpredictions 376
12.5. Summary 384
12.6. Practice 384
Chapter13. AdventuresinCovariance 387
13.1. Varyingslopesbyconstruction 389
13.2. Example: Admissiondecisionsandgender 398
13.3. Example: Cross-classifiedchimpanzeeswithvaryingslopes 403
13.4. ContinuouscategoriesandtheGaussianprocess 410
13.5. Summary 419
13.6. Practice 419
Chapter14. MissingDataandOtherOpportunities 423
14.1. Measurementerror 424
14.2. Missingdata 431
14.3. Summary 439
14.4. Practice 439
Chapter15. Horoscopes 441
Endnotes 445
Bibliography 457
Citationindex 465
Topicindex 467