Table Of Content

J0470018755pre October17,2006 20:27 CharCount=0 Bayesian Statistical Modelling SecondEdition PETERCONGDON QueenMary,UniversityofLondon,UK iii J0470018755pre October17,2006 20:27 CharCount=0 iii J0470018755pre October17,2006 20:27 CharCount=0 Bayesian Statistical Modelling i J0470018755pre October17,2006 20:27 CharCount=0 WILEYSERIESINPROBABILITYANDSTATISTICS establishedbyWalterA.ShewhartandSamuelS.Wilks Editors DavidJ.Balding,PeterBloomfield,NoelA.C.Cressie,NicholasI.Fisher, IainM.Johnstone,J.B.Kadane,GeertMolenberghs,LouiseM.Ryan, DavidW.Scott,AdrianF.M.Smith,JozefL.Teugels EditorsEmeriti VicBarnett,J.StuartHunter,DavidG.Kendall Acompletelistofthetitlesinthisseriesappearsattheendofthisvolume. ii J0470018755pre October17,2006 20:27 CharCount=0 Bayesian Statistical Modelling SecondEdition PETERCONGDON QueenMary,UniversityofLondon,UK iii J0470018755pre October17,2006 20:27 CharCount=0 Copyright(cid:2)C 2006 JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester, WestSussexPO198SQ,England Telephone (+44)1243779777 Email(forordersandcustomerserviceenquiries):[email protected] Visit our Home Page on www.wiley.com AllRightsReserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystemortransmittedin anyformorbyanymeans,electronic,mechanical,photocopying,recording,scanningorotherwise,exceptunder thetermsoftheCopyright,DesignsandPatentsAct1988orunderthetermsofalicenceissuedbytheCopyright LicensingAgencyLtd,90TottenhamCourtRoad,LondonW1T4LP,UK,withoutthepermissioninwritingofthe Publisher.RequeststothePublishershouldbeaddressedtothePermissionsDepartment,JohnWiley&SonsLtd, TheAtrium,SouthernGate,Chichester,WestSussexPO198SQ,England,[email protected],or faxedto(+44)1243770620. Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.Allbrandnamesand productnamesusedinthisbookaretradenames,servicemarks,trademarksorregisteredtrademarksoftheir respectiveowners.ThePublisherisnotassociatedwithanyproductorvendormentionedinthisbook. Thispublicationisdesignedtoprovideaccurateandauthoritativeinformationinregardtothesubjectmatter covered.ItissoldontheunderstandingthatthePublisherisnotengagedinrenderingprofessionalservices.If professionaladviceorotherexpertassistanceisrequired,theservicesofacompetentprofessionalshouldbesought. OtherWileyEditorialOffices JohnWiley&SonsInc.,111RiverStreet,Hoboken,NJ07030,USA Jossey-Bass,989MarketStreet,SanFrancisco,CA94103-1741,USA Wiley-VCHVerlagGmbH,Boschstr.12,D-69469Weinheim,Germany JohnWiley&SonsAustraliaLtd,42McDougallStreet,Milton,Queensland4064,Australia JohnWiley&Sons(Asia)PteLtd,2ClementiLoop#02-01,JinXingDistripark,Singapore129809 JohnWiley&SonsCanadaLtd,6045FreemontBlvd,Mississauga,Ontario,L5R4J3,Canada Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbe availableinelectronicbooks. BritishLibraryCataloguinginPublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary ISBN-13 978-0-470-01875-0(HB) ISBN-10 0-470-01875-5(HB) Typesetin10/12ptTimesbyTechBooks,NewDelhi,India PrintedandboundinGreatBritainbyAntonyRoweLtd,Chippenham,Wiltshire Thisbookisprintedonacid-freepaperresponsiblymanufacturedfromsustainableforestry inwhichatleasttwotreesareplantedforeachoneusedforpaperproduction. iv J0470018755pre October17,2006 20:27 CharCount=0 Contents Preface xiii Chapter1 Introduction:TheBayesianMethod,itsBenefitsandImplementation 1 1.1 TheBayesapproachanditspotentialadvantages 1 1.2 ExpressingprioruncertaintyaboutparametersandBayesianupdating 2 1.3 MCMCsamplingandinferencesfromposteriordensities 5 1.4 ThemainMCMCsamplingalgorithms 9 1.4.1 Gibbssampling 12 1.5 ConvergenceofMCMCsamples 14 1.6 Predictionsfromsampling:usingtheposteriorpredictivedensity 18 1.7 Thepresentbook 18 References 19 Chapter2 BayesianModelChoice,ComparisonandChecking 25 2.1 Introduction:theformalapproachtoBayesmodelchoiceand averaging 25 2.2 AnalyticmarginallikelihoodapproximationsandtheBayes informationcriterion 28 2.3 MarginallikelihoodapproximationsfromtheMCMCoutput 30 2.4 ApproximatingBayesfactorsormodelprobabilities 36 2.5 Jointspacesearchmethods 38 2.6 Directmodelaveragingbybinaryandcontinuousselectionindicators 41 2.7 Predictivemodelcomparisonviacross-validation 43 2.8 Predictivefitcriteriaandposteriorpredictivemodelchecks 46 2.9 TheDICcriterion 48 2.10 Posterioranditeration-specificcomparisonsoflikelihoodsand penalisedlikelihoods 50 2.11 Montecarloestimatesofmodelprobabilities 52 References 57 Chapter3 TheMajorDensitiesandtheirApplication 63 3.1 Introduction 63 3.2 Univariatenormalwithknownvariance 64 3.2.1 Testinghypothesesonnormalparameters 66 J0470018755pre October17,2006 20:27 CharCount=0 vi CONTENTS 3.3 Inferenceonunivariatenormalparameters,meanandvariance unknown 69 3.4 Heavytailedandskewdensityalternativestothenormal 71 3.5 Categoricaldistributions:binomialandbinarydata 74 3.5.1 Simulatingcontrolsthroughhistoricalexposure 76 3.6 Poissondistributionforeventcounts 79 3.7 Themultinomialanddirichletdensitiesforcategoricaland proportionaldata 82 3.8 Multivariatecontinuousdata:multivariatenormalandtdensities 85 3.8.1 Partitioningmultivariatepriors 87 3.8.2 Themultivariatetdensity 88 3.9 Applicationsofstandarddensities:classificationrules 91 3.10 Applicationsofstandarddensities:multivariatediscrimination 98 Exercises 100 References 102 Chapter4 NormalLinearRegression,GeneralLinearModels andLog-LinearModels 109 4.1 ThecontextforBayesianregressionmethods 109 4.2 Thenormallinearregressionmodel 111 4.2.1 Unknownregressionvariance 112 4.3 Normallinearregression:variableandmodelselection,outlier detectionanderrorform 116 4.3.1 Otherpredictorandmodelsearchmethods 118 4.4 Bayesianridgepriorsformulticollinearity 121 4.5 Generallinearmodels 123 4.6 Binaryandbinomialregression 123 4.6.1 Priorsonregressioncoefficients 124 4.6.2 Modelchecks 126 4.7 Latentdatasamplingforbinaryregression 129 4.8 Poissonregression 132 4.8.1 Poissonregressionforcontingencytables 134 4.8.2 Log-linearmodelselection 139 4.9 Multivariateresponses 140 Exercises 143 References 146 Chapter5 HierarchicalPriorsforPoolingStrengthandOverdispersed RegressionModelling 151 5.1 Hierarchicalpriorsforpoolingstrengthandingenerallinear modelregression 151 5.2 Hierarchicalpriors:conjugateandnon-conjugatemixing 152 5.3 Hierarchicalpriorsfornormaldatawithapplicationsin meta-analysis 153 5.3.1 Priorforsecond-stagevariance 155 J0470018755pre October17,2006 20:27 CharCount=0 CONTENTS vii 5.4 Poolingstrengthunderexchangeablemodelsforpoissonoutcomes 157 5.4.1 Hierarchicalpriorchoices 158 5.4.2 Parametersampling 159 5.5 Combininginformationforbinomialoutcomes 162 5.6 Randomeffectsregressionforoverdispersedcountand binomialdata 165 5.7 Overdispersednormalregression:thescale-mixturestudentt model 169 5.8 Thenormalmeta-analysismodelallowingforheterogeneityin studydesignorpatientrisk 173 5.9 Hierarchicalpriorsformultinomialdata 176 5.9.1 Histogramsmoothing 177 Exercises 179 References 183 Chapter6 DiscreteMixturePriors 187 6.1 Introduction:therelevanceandapplicabilityofdiscretemixtures 187 6.2 Discretemixturesofparametricdensities 188 6.2.1 Modelchoice 190 6.3 Identifiabilityconstraints 191 6.4 Hurdleandzero-inflatedmodelsfordiscretedata 195 6.5 Regressionmixturesforheterogeneoussubpopulations 197 6.6 Discretemixturescombinedwithparametricrandomeffects 200 6.7 Non-parametricmixturemodellingviadirichletprocesspriors 201 6.8 Othernon-parametricpriors 207 Exercises 212 References 216 Chapter7 MultinomialandOrdinalRegressionModels 219 7.1 Introduction:applicationswithcategoricandordinaldata 219 7.2 Multinomiallogitchoicemodels 221 7.3 Themultinomialprobitrepresentationofinterdependentchoices 224 7.4 Mixedmultinomiallogitmodels 228 7.5 Individuallevelordinalregression 230 7.6 Scoresfororderedfactorsincontingencytables 235 Exercises 237 References 238 Chapter8 TimeSeriesModels 241 8.1 Introduction:alternativeapproachestotimeseriesmodels 241 8.2 Autoregressivemodelsintheobservations 242 8.2.1 Priorsonautoregressivecoefficients 244 8.2.2 Initialconditionsaslatentdata 246 8.3 TrendstationarityintheAR1model 248 8.4 Autoregressivemovingaveragemodels 250