Applied Bayesian Statistics: With R and OpenBUGS Examples

236 Pages·2013·4.427 MB·English

by  Mary Kathryn Cowles

Springer Texts in Statistics
SeriesEditors:
G.Casella
S.E.Fienberg
I.Olkin
Forfurthervolumes:
http://www.springer.com/series/417

Mary Kathryn Cowles
Applied Bayesian Statistics
With R and OpenBUGS Examples
123
MaryKathrynCowles
DepartmentofStatistics
andActuarialScience
UniversityofIowa
IowaCity,Iowa,USA
ISSN1431-875X
ISBN978-1-4614-5695-7 ISBN978-1-4614-5696-4(eBook)
DOI10.1007/978-1-4614-5696-4
SpringerNewYorkHeidelbergDordrechtLondon
LibraryofCongressControlNumber:2012951150
©SpringerScience+BusinessMediaNewYork2013
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
To Brendan,Lucy, andDonald.

Preface
Ihavetaughta coursecalled“BayesianStatistics” atthe UniversityofIowaevery
academicyearsince1998–1999.Thisbookisintendedtofitthegoalsandaudience
addressedbymycourse.The“CourseObjectives”sectionofmysyllabusreads:
Through hands-on experience withreal datafrom avarietyofapplications, studentswill
learn the basics of designing and carrying out Bayesian analyses, and interpreting and
communicatingtheresults.Studentswilllearntousesoftwarepackages includingRand
OpenBUGStofitBayesianmodels.
The course is intended to be intensely practical, focussing on building under-
standingof the conceptsand proceduresrequiredto performBayesian analysisof
realdatatoanswerrealquestions.Emphasisisgiventosuchissuesasdetermining
what data is needed to address a particular question; choosing an appropriate
probabilitydistributionforsampledata;quantifyingalready-existingknowledgein
the form of a prior distribution on model parameters; verifying that the posterior
distribution will be proper if improper prior distributions are used; and when and
howtospecifyhierarchicalmodels.Interpretationandcommunicationofresultsare
stressed,includingdifferencesfrom,andsimilaritiesto,classicalapproachestothe
sameproblems.
WinBUGS and OpenBUGS currently are the dominant software in applied
use of Bayesian methods. I have chosen to introduce OpenBUGS as the primary
data analysis software in this textbook because, unlike WinBUGS, OpenBUGS is
undergoingcontinuingdevelopmentandhasversionsthatrunnativelyunderLinux
andMacintoshoperatingsystemsaswellasWindows.Althoughsomebackground
is provided on the Markov chain Monte Carlo sampling procedures employed by
WinBUGS andOpenBUGS, the emphasisis on those tasks thatauser mustcarry
outcorrectlyforreasonablytrustworthyinference.Theseincludeusingappropriate
toolstoassesswhetherandwhenasamplerhasconvergedtothetargetdistribution,
decidinghowmanyiterationsareneededforacceptableaccuracyinestimation,and
how to reportresults of a Bayesian analysis conductedwith OpenBUGS. Caveats
aboutthefallibilityofconvergencediagnosticsareemphasized.
vii
viii Preface
Studentsofdifferentlevelsanddisciplinestakethecourse,including:undergrad-
uatemathematicsandstatisticsmajors;master’sstudentsinstatistics, biostatistics,
statistical genetics, educational testing and measurement, and engineering; and
PhD students in economics, marketing,psychology,and geographyas well as the
previouslylistedfields.Inaddition,severalpracticingstatisticiansemployedbythe
UniversityofIowaandAmericanCollegeTesting(ACT)havetakenthecourse.
Thegoalofthecourse,andofthisbook,istoprovideanintroductiontoBayesian
principlesandpracticethatisclear,useful,andunintimidatingtomotivatedstudents
evenif theydo nothavean advancedbackgroundin mathematicsand probability.
I emphasizeintuitiveinsightwithoutsacrificingmathematicalcorrectness.Prereq-
uisites are one or two semesters of calculus-based probability and mathematical
statistics(atleastattheHoggandTannislevel)andoneortwosemestersofclassical
statistical methods, including linear regression (David Moore’s Basic Practice of
Statistics level). Elementaryintegraland differentialcalculusis occasionallyused
inlecturesandhomework.Linearalgebraisnotrequired.
Coralville,Iowa MaryKathrynCowles
Contents
1 WhatIsBayesianStatistics? .............................................. 1
1.1 TheScientificMethod(ButItIsNotJustforScience...).......... 1
1.2 ABitofHistory ...................................................... 2
1.3 ExampleoftheBayesianMethod:DoesMyFriend
HaveBreastCancer?................................................. 3
1.3.1 QuantifyingUncertaintyUsingProbabilities.............. 3
1.3.2 ModelsandPriorProbabilities............................. 5
1.3.3 Data ......................................................... 6
1.3.4 LikelihoodsandPosteriorProbabilities ................... 7
1.3.5 BayesianSequentialAnalysis.............................. 8
1.4 CalibrationExperimentsforAssessingSubjectiveProbabilities... 8
1.5 WhatIstoCome? .................................................... 10
Problems ..................................................................... 11
2 ReviewofProbability ...................................................... 13
2.1 ReviewofProbability................................................ 13
2.1.1 EventsandSampleSpaces................................. 13
2.1.2 Unions,Intersections,Complements ...................... 14
2.1.3 TheAdditionRule.......................................... 15
2.1.4 MarginalandConditionalProbabilities ................... 15
2.1.5 TheMultiplicationRule.................................... 17
2.2 PuttingItAllTogether:DidBrendanMailtheBillPayment?..... 17
2.2.1 TheLawofTotalProbability .............................. 17
2.2.2 Bayes’RuleintheDiscreteCase.......................... 19
2.3 RandomVariablesandProbabilityDistributions................... 20
Problems ..................................................................... 21
3 IntroductiontoOne-ParameterModels:Estimating
aPopulationProportion ................................................... 25
3.1 WhatProportionofStudentsWouldQuitSchoolIfTuition
WereRaised19%:EstimatingaPopulationProportion............ 25
ix
x Contents
3.2 TheFirstStageofaBayesianModel................................ 25
3.2.1 TheBinomialDistributionforOurSurvey................ 26
3.2.2 KernelsandNormalizingConstants....................... 27
3.2.3 TheLikelihoodFunction................................... 27
3.3 TheSecondStageoftheBayesianModel:ThePrior............... 28
3.3.1 OtherPossiblePriorDistributions......................... 29
3.3.2 PriorProbabilityIntervals.................................. 31
3.4 UsingtheDatatoUpdatethePrior:ThePosteriorDistribution ... 32
3.5 ConjugatePriors...................................................... 34
3.5.1 ComputingthePosteriorDistribution
withaConjugatePrior...................................... 34
3.5.2 ChoosingtheParametersofaBetaDistribution
toMatchPriorBeliefs...................................... 35
3.5.3 ComputingandGraphingthePosteriorDistribution...... 38
3.5.4 PlottingthePriorDensity,theLikelihood,
andthePosteriorDensity................................... 38
3.6 IntroductiontoRforBayesianAnalysis............................ 38
3.6.1 FunctionsandObjectsinR ................................ 39
3.6.2 Summarizingand GraphingProbability
DistributionsinR........................................... 42
3.6.3 PrintingandSavingRGraphics ........................... 44
3.6.4 RPackagesUsefulinBayesianAnalysis.................. 44
3.6.5 EndingaSession............................................ 46
Problems ..................................................................... 46
4 InferenceforaPopulationProportion ................................... 49
4.1 EstimationandTesting:FrequentistApproach ..................... 49
4.1.1 MaximumLikelihoodEstimation.......................... 49
4.1.2 FrequentistConfidenceIntervals........................... 51
4.1.3 FrequentistHypothesisTesting ............................ 52
4.2 BayesianInference:SummarizingthePosteriorDistribution...... 54
4.2.1 ThePosteriorMean......................................... 54
4.2.2 OtherBayesianPointEstimates............................ 55
4.2.3 BayesianPosteriorIntervals ............................... 57
4.3 UsingthePosteriorDistributiontoTestHypotheses............... 59
4.4 PosteriorPredictiveDistributions ................................... 61
Problems ..................................................................... 63
5 SpecialConsiderationsinBayesianInference........................... 67
5.1 RobustnesstoPriorSpecifications .................................. 67
5.2 InferenceUsingNonconjugatePriors............................... 69
5.2.1 DiscretePriors.............................................. 69
5.2.2 AHistogramPrior.......................................... 71
5.3 NoninformativePriors ............................................... 72
5.3.1 ReviewofProperandImproperDistributions ............ 72
5.3.2 ANoninformativePriorfortheBinomialLikelihood .... 73