Bayesian Computation with R

2007·8.84 MB·English

by  Jim Albert

Use R!
Series Editors:
Robert Gentleman Kurt Hornik Giovanni Parmigiani
Use R!
Albert: Bayesian Computation with R
Cook/Swayne:Interactive andDynamic Graphics for Data Analysis: With R
and GGobi
Paradis: Analysis of Phylogenetics and Evolution with R
Pfaff: Analysis ofIntegrated and Cointegrated Time Series with R
Jim Albert
Bayesian Computation
with R
ABC
Professor Jim Albert
Department of Mathematics and Statistics
Bowling Green State University
Bowling Green, OH 43403-0221
USA
[email protected]
Series Editors:
Robert Gentleman Kurt Hornik
Program in Computational Biology Departmentfu¨rStatistik und Mathematik
Division ofPublic Health Sciences Wirtschaftsuniversität Wien Augasse 2-6
Fred Hutchinson Cancer Research Center A-1090 Wien
1100 Fairview Ave.N, M2-B876 Austria
Seattle,Washington981029-1024
USA
Giovanni Parmigiani
The Sidney Kimmel Comprehensive
Cancer Center at Johns Hopkins University
550 North Broadway
Baltimore,MD21205-2011
USA
ISBN 978-0-387-71384-7 e-ISBN 978-0-387-71385-4
Library of Congress Control Number: 2007929182
©2007 Springer Science+Business Media, LLC
All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY
10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection
with any form of information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are
not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject
to proprietary rights.
Printed on acid-free paper
9 8 7 6 5 4 3 (corrected 3rd printing, 2008)
springer.com
Preface
There has been dramatic growth in the development and application of
Bayesian inference in statistics. Berger (2000) documents the increase in
Bayesianactivitybythenumberof publishedresearcharticles,thenumberof
books,andtheextensivenumberofapplicationsofBayesianarticlesinapplied
disciplines such as science and engineering.
One reason for the dramatic growth in Bayesian modeling is the availabil-
ity of computational algorithms to compute the range of integrals that are
necessary in a Bayesian posterior analysis. Due to the speed of modern com-
puters, it is now possible to use the Bayesian paradigm to fit very complex
models that cannot be fit by alternative frequentist methods.
To fit Bayesian models, one needs a statistical computing environment.
This environment should be such that one can
• write short scripts to define a Bayesian model
• use or write functions to summarize a posterior distribution
• use functions to simulate from the posterior distribution
• construct graphs to illustrate the posterior inference
An environment that meets these requirements is the R system. R provides a
wide range of functions for data manipulation, calculation, and graphical dis-
plays. Moreover, it includes a well-developed, simple programming language
that users can extend by adding new functions. Many such extensions of the
language in the form of packages are easily downloadable from the Compre-
hensive R Archive Network (CRAN).
The purpose of this book is to introduce Bayesian modeling by the use
of computation using the R language. At Bowling Green State University,
I have taught an introductory Bayesian inference class to students in masters
anddoctoralprogramsinstatisticsforwhichthisbookwouldbeappropriate.
This book would serve as a useful companion to the introductory Bayesian
textsGelman et al (2003), Carlin and Louis (2000), Press(2003), Gill(2002),
or Lee (2004). Also the book would be valuable to the statistical practitioner
who wishes to learn more about the R language and Bayesian methodology.
vi Preface
Chapters 2, 3, and 4 illustrate the use of R for Bayesian inference for
standardone-andtwo-parameterproblems.Thesechaptersdiscusstheuseof
different types of priors, the use of the posterior distribution to perform dif-
ferent types of inference, and the use of the predictive distribution. The base
packageofRprovidesfunctionstosimulatefromallofthestandardprobabil-
ity distributions, and these functions can be used to simulate from a variety
ofposteriordistributions.ModernBayesiancomputingisintroducedinChap-
ters 5 and 6. Chapter 5 discusses the summarization of posterior distribution
by posterior modes and introduces rejection sampling and the Monte Carlo
approachforcomputingintegrals.Chapter6introducesthefundamentalideas
ofMarkovchainMonteCarlo(MCMC)methodsandtheuseofMCMCoutput
analysis to decide if the batch of simulated draws provides a reasonable app-
roximation to the posterior distribution of interest. The remaining chapters
illustrate the use of these computational algorithms for a variety of Bayesian
applications. Chapter 7 introduces the use of exchangeable models in the
simultaneousestimationofasetofPoissonrates.Chapter8describesBayesian
tests of simple hypotheses and the use of Bayes factors in comparing models.
Chapter 9 describes Bayesian regression models, and Chapter 10 describes
several applications, such as robust modeling, binary regression with a probit
link,andorder-restrictedinferencethatarewell-suitedfortheGibbssampling
algorithm. Chapter 11 describes the use of R to interface with WinBUGS,
a popular program for implementing MCMC algorithms.
AnRpackage,LearnBayes,availablefromtheCRANsite,hasbeenwritten
toaccompanythistext.ThispackagecontainsalloftheBayesianRfunctions
and datasets described in the book. One goal in writing LearnBayes is to
provide guidance for the student and applied statistician in writing short R
functions for implementing Bayesian calculations for their specific problems.
Also the LearnBayes package will make it easier for users to use the growing
number of R packages for fitting a variety of Bayesian models.
I would like to expressmy appreciation for the people who provided assis-
tance in preparing this book. John Kimmel, my editor, was most helpful in
encouraging me to write this book and provide helpful feedback. I am appre-
ciative of Patricia Williamson and Sherwin Toribio for providing useful sug-
gestions. I am appreciative to all of the students at Bowling Green who have
enrolled in my Bayesian statistics class over the years. Finally, but certainly
notleast,IwishtothankmywifeAnneandmychildrenLynne,Bethanyand
Steven for encouragement and inspiration.
Bowling Green, Ohio, Jim Albert
January 2007
Contents
1 An Introduction to R...................................... 1
1.1 Overview............................................... 1
1.2 Exploring a Student Dataset.............................. 1
1.2.1 Introduction to the Dataset......................... 1
1.2.2 Reading the Data into R ........................... 2
1.2.3 R Commands to Summarize and Graph a Single Batch. 2
1.2.4 R Commands to Compare Batches .................. 4
1.2.5 R Commands for Studying Relationships ............. 6
1.3 Exploring the Robustness of the t Statistic ................. 8
1.3.1 Introduction ...................................... 8
1.3.2 Writing a Function to Compute the t Statistic ........ 9
1.3.3 Programming a Monte Carlo Simulation.............. 11
1.3.4 The Behavior of the True Significance Level Under
Different Assumptions ............................. 12
1.4 Further Reading......................................... 13
1.5 Summary of R Functions ................................. 14
1.6 Exercises ............................................... 15
2 Introduction to Bayesian Thinking......................... 19
2.1 Introduction ............................................ 19
2.2 Learning About the Proportion of Heavy Sleepers ........... 19
2.3 Using a Discrete Prior ................................... 20
2.4 Using a Beta Prior ...................................... 22
2.5 Using a Histogram Prior ................................. 26
2.6 Prediction .............................................. 29
2.7 Further Reading......................................... 34
2.8 Summary of R Functions ................................. 34
2.9 Exercises ............................................... 35
viii Contents
3 Single-Parameter Models .................................. 39
3.1 Introduction ............................................ 39
3.2 Normal Distribution with Known Mean
but Unknown Variance................................... 39
3.3 Estimating a Heart Transplant Mortality Rate .............. 41
3.4 An Illustration of Bayesian Robustness..................... 44
3.5 A Bayesian Test of the Fairness of a Coin................... 50
3.6 Further Reading......................................... 53
3.7 Summary of R Functions ................................. 53
3.8 Exercises ............................................... 54
4 Multiparameter Models ................................... 57
4.1 Introduction ............................................ 57
4.2 Normal Data with Both Parameters Unknown .............. 57
4.3 A Multinomial Model .................................... 60
4.4 A Bioassay Experiment .................................. 60
4.5 Comparing Two Proportions.............................. 65
4.6 Further Reading......................................... 70
4.7 Summary of R Functions ................................. 70
4.8 Exercises ............................................... 71
5 Introduction to Bayesian Computation .................... 75
5.1 Introduction ............................................ 75
5.2 Computing Integrals ..................................... 76
5.3 Setting Up a Problem on R............................... 77
5.4 A Beta-Binomial Model for Overdispersion ................. 78
5.5 Approximations Based on Posterior Modes ................. 80
5.6 The Example ........................................... 82
5.7 Monte Carlo Method for Computing Integrals............... 84
5.8 Rejection Sampling ...................................... 85
5.9 Importance Sampling .................................... 88
5.10 Sampling Importance Resampling ......................... 91
5.11 Further Reading......................................... 94
5.12 Summary of R Functions ................................. 94
5.13 Exercises ............................................... 96
6 Markov Chain Monte Carlo Methods ......................101
6.1 Introduction ............................................101
6.2 Introduction to Discrete Markov Chains....................101
6.3 Metropolis-Hasting Algorithms............................104
6.4 Gibbs Sampling .........................................106
6.5 MCMC Output Analysis .................................106
6.6 A Strategy in Bayesian Computing ........................108
6.7 Learning About a Normal Population from Grouped Data ....108
6.8 Example of Output Analysis ..............................113
6.9 Modeling Data with Cauchy Errors ........................116
Contents ix
6.10 Analysis of the Stanford Heart Transplant Data .............124
6.11 Further Reading.........................................129
6.12 Summary of R Functions .................................130
6.13 Exercises ...............................................131
7 Hierarchical Modeling .....................................137
7.1 Introduction ............................................137
7.2 Introduction to Hierarchical Modeling......................137
7.3 Individual and Combined Estimates .......................139
7.4 Equal Mortality Rates? ..................................141
7.5 Modeling a Prior Belief of Exchangeability..................145
7.6 Posterior Distribution....................................147
7.7 Simulating from the Posterior.............................147
7.8 Posterior Inferences......................................151
7.8.1 Shrinkage ........................................152
7.8.2 Comparing Hospitals ..............................153
7.9 Posterior Predictive Model Checking.......................155
7.10 Further Reading.........................................157
7.11 Summary of R Functions .................................158
7.12 Exercises ...............................................158
8 Model Comparison ........................................163
8.1 Introduction ............................................163
8.2 Comparison of Hypotheses................................163
8.3 A One-Sided Test of a Normal Mean.......................164
8.4 A Two-Sided Test of a Normal Mean ......................167
8.5 Comparing Two Models ..................................168
8.6 Models for Soccer Goals..................................169
8.7 Is a Baseball Hitter Really Streaky? .......................172
8.8 A Test of Independence in a Two-Way Contingency Table ....176
8.9 Further Reading.........................................180
8.10 Summary of R Functions .................................181
8.11 Exercises ...............................................183
9 Regression Models.........................................187
9.1 Introduction ............................................187
9.2 Normal Linear Regression ................................187
9.2.1 The Model .......................................187
9.2.2 The Posterior Distribution..........................188
9.2.3 Prediction of Future Observations ...................188
9.2.4 Computation .....................................189
9.2.5 Model Checking...................................189
9.2.6 An Example ......................................190
9.3 Survival Modeling .......................................199
9.4 Further Reading.........................................204
x Contents
9.5 Summary of R Functions .................................205
9.6 Exercises ...............................................206
10 Gibbs Sampling............................................211
10.1 Introduction ............................................211
10.2 Robust Modeling ........................................212
10.3 Binary Response Regression with a Probit Link .............216
10.4 Estimating a Table of Means..............................219
10.4.1 Introduction ......................................219
10.4.2 A Flat Prior Over the Restricted Space ..............223
10.4.3 A Hierarchical Regression Prior .....................227
10.4.4 Predicting the Success of Future Students ............232
10.5 Further Reading.........................................233
10.6 Summary of R Functions .................................233
10.7 Exercises ...............................................234
11 Using R to Interface with WinBUGS ......................237
11.1 Introduction to WinBUGS................................237
11.2 An R Interface to WinBUGS..............................238
11.3 MCMC Diagnostics Using the boa Package .................239
11.4 A Change-Point Model...................................240
11.5 A Robust Regression Model ..............................243
11.6 Estimating Career Trajectories ............................247
11.7 Further Reading.........................................253
11.8 Exercises ...............................................254
References.....................................................259
Index..........................................................263