Flexible Bayesian Regression Modelling

292 Pages·2019·13.542 MB·English

by  Yanan Fan (editor), David Nott (editor), Mike S. Smith (editor), Jean-Luc Dortet-Bernadet (editor)

AcademicPressisanimprintofElsevier
125LondonWall,LondonEC2Y5AS,UnitedKingdom
525BStreet,Suite1650,SanDiego,CA92101,UnitedStates
50HampshireStreet,5thFloor,Cambridge,MA02139,UnitedStates
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UnitedKingdom
Copyright©2020ElsevierInc.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,
electronicormechanical,includingphotocopying,recording,oranyinformationstorageand
retrievalsystem,withoutpermissioninwritingfromthepublisher.Detailsonhowtoseek
permission,furtherinformationaboutthePublisher’spermissionspoliciesandourarrangements
withorganizationssuchastheCopyrightClearanceCenterandtheCopyrightLicensingAgency,
canbefoundatourwebsite:www.elsevier.com/permissions.
Thisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythe
Publisher(otherthanasmaybenotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperience
broadenourunderstanding,changesinresearchmethods,professionalpractices,ormedical
treatmentmaybecomenecessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgein
evaluatingandusinganyinformation,methods,compounds,orexperimentsdescribedherein.In
usingsuchinformationormethodstheyshouldbemindfuloftheirownsafetyandthesafetyof
others,includingpartiesforwhomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,
assumeanyliabilityforanyinjuryand/ordamagetopersonsorpropertyasamatterofproducts
liability,negligenceorotherwise,orfromanyuseoroperationofanymethods,products,
instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-12-815862-3
ForinformationonallAcademicPresspublications
visitourwebsiteathttps://www.elsevier.com/books-and-journals
Publisher:CandiceJanco
AcquisitionEditor:CandiceJanco
EditorialProjectManager:SusanIkeda
ProductionProjectManager:OmerMukthar
Designer:ChristianBilbow
TypesetbyVTeX
Contents
Contributors ix
Preface xiii
1. BayesianquantileregressionwiththeasymmetricLaplace
distribution 1
J.-L.Dortet-Bernadet,Y.Fan,T.Rodrigues
1.1. Introduction 1
1.2. TheasymmetricLaplacedistributionforquantileregression 3
1.3. Oncoverageprobabilities 16
1.4. Postprocessingformultiplefittings 18
1.5. Finalremarksandconclusion 22
References 23
2. Avignetteonmodel-basedquantileregression:analysingexcess
zeroresponse 27
ErikaCunningham,SuryaT.Tokdar,JamesS.Clark
2.1. Introduction 28
2.2. Excesszeroregressionanalysis 30
2.3. Casestudydataandobjective 31
2.4. Fittingsinglecovariatebasalareamodels 32
2.5. Interpretingquantileregressions 37
2.6. Assessingmodelassumptionsandmakingimprovements 41
2.7. Predictionandinterpretingpredictedresponses 47
2.8. Fittingmultipleregressionbasalareamodels 50
2.9. Conclusionsandfinalremarks 62
Acknowledgement 63
References 63
3. Bayesiannonparametricdensityregressionforordinalresponses 65
MariaDeYoreo,AthanasiosKottas
3.1. Introduction 65
3.2. Bayesiannonparametricdensityregression 68
3.3. Mixturemodellingforordinalresponses 73
3.4. Summary 86
Acknowledgements 87
References 87
v
vi Contents
4. Bayesian nonparametric methods for financial and
macroeconomictimeseriesanalysis 91
MariaKalli
4.1. Introduction 91
4.2. Bayesiannonparametricmethodsfortheinnovationdistributionin
volatilitymodels 93
4.3. Bayesiannonparametricmethodsforlong-rangedependenceinSV
models 98
4.4. Bayesiannonparametricmethodsfortheanalysisofmacroeconomic
timeseries 105
4.5. Conclusion 114
References 115
5. Bayesianmixedbinary-continuouscopularegressionwithan
applicationtochildhoodundernutrition 121
NadjaKlein,ThomasKneib,GiampieroMarra,RosalbaRadice
5.1. Introduction 121
5.2. Bivariatecopulamodelswithmixedbinary-continuousmarginals 125
5.3. Bayesianinference 132
5.4. Modelselectionandmodelevaluation 138
5.5. Results 142
5.6. Summaryanddiscussion 147
Acknowledgements 149
References 149
6. NonstandardflexibleregressionviavariationalBayes 153
JohnT.Ormerod
6.1. Introduction 154
6.2. Preparatorymodellingcomponents 156
6.3. Astandardsemiparametricregressionmodel 163
6.4. Robustnonparametricregression 165
6.5. Generalisedadditivemodelwithheteroscedasticvariance 170
6.6. Generalisedadditivenegativebinomialmodel 173
6.7. Logisticregressionwithmissingcovariates 177
6.8. Conclusion 182
Acknowledgements 183
References 183
7. ScalableBayesianvariableselectionregressionmodelsforcount
data 187
YinsenMiao,JeongHwanKook,YadongLu,MicheleGuindani,
MarinaVannucci
Contents vii
7.1. Introduction 188
7.2. Bayesianvariableselectionviaspike-and-slabpriors 189
7.3. Negativebinomialregressionmodels 190
7.4. Dirichlet-multinomialregressionmodels 200
7.5. Simulationstudy 206
7.6. Benchmarkapplications 212
7.7. Conclusion 215
References 216
8. Bayesianspectralanalysisregression 221
TaeryonChoi,PeterJ.Lenk
8.1. Introduction 221
8.2. Smoothoperators 223
8.3. Bayesianspectralanalysisregression 227
8.4. Shapeconstraints 234
8.5. Nonnormaldistributions 237
8.6. RlibrarybsamGP 238
8.7. Conclusion 245
Acknowledgements 246
References 246
9. Flexibleregressionmodellingundershapeconstraints 251
AndrewA.Manderson,KevinMurray,BerwinA.Turlach
9.1. Introduction 252
9.2. Orthonormaldesignmatrices 253
9.3. Monotonicpolynomialmodel 254
9.4. Covariateselection 261
9.5. Conclusion 278
References 278
Index 281
Contributors
TaeryonChoi
KoreaUniversity,DepartmentofStatistics,Seoul,RepublicofKorea
JamesS.Clark
DukeUniversity,Durham,NC,UnitedStates
ErikaCunningham
DukeUniversity,Durham,NC,UnitedStates
MariaDeYoreo
TheRANDCorporation,1776MainSt.,SantaMonica,CA90401,UnitedStates
J.-L.Dortet-Bernadet
InstitutdeRechercheMathématiqueAvancée,UMR7501CNRS,Universitéde
Strasbourg,Strasbourg,France
Y.Fan
SchoolofMathematicsandStatistics,UniversityofNewSouthWales,Sydney,
NSW,Australia
MicheleGuindani
UniversityofCalifornia,Irvine,DepartmentofStatistics,BrentHall2241,Irvine,CA
92697,UnitedStates
MariaKalli
UniversityofKent,SchoolofMathematics,Statistics&ActuarialScience,Sibson
Building,ParkwoodRoad,Canterbury,CT27FS,UnitedKingdom
NadjaKlein
HumboldtUniversitätzuBerlin,SchoolofBusinessandEconomics,Unterden
Linden6,10099Berlin,Germany
ThomasKneib
Georg-August-UniversitätGöttingen,FacultyofBusinessandEconomicSciences,
Humboldtallee3,37073Göttingen,Germany
JeongHwanKook
RiceUniversity,DepartmentofStatistics,6100MainSt,Houston,TX77005,
UnitedStates
ix
x Contributors
AthanasiosKottas
UniversityofCalifornia,SantaCruz,DepartmentofStatistics,1156HighStreet,
SantaCruz,CA95064,UnitedStates
PeterJ.Lenk
UniversityofMichigan,StephenM.RossSchoolofBusiness,AnnArbor,MI,
UnitedStates
YadongLu
UniversityofCalifornia,Irvine,DepartmentofStatistics,BrentHall2241,Irvine,CA
92697,UnitedStates
AndrewA.Manderson
TheUniversityofWesternAustralia,DepartmentofMathematicsandStatistics,
35StirlingHighway,Crawley,WA6009,Australia
GiampieroMarra
UniversityCollegeLondon,DepartmentofStatisticalScience,GowerStreet,
LondonWC1E6BT,UnitedKingdom
YinsenMiao
RiceUniversity,DepartmentofStatistics,6100MainSt,Houston,TX77005,
UnitedStates
KevinMurray
TheUniversityofWesternAustralia,SchoolofPopulationandGlobalHealth,35
StirlingHighway,Crawley,WA6009,Australia
JohnT.Ormerod
SchoolofMathematicsandStatistics,UniversityofSydney,Sydney,NSW2006,
Australia
ARCCentreofExcellenceforMathematical&StatisticalFrontiers,Universityof
Melbourne,Parkville,VIC3010,Australia
RosalbaRadice
CassBusinessSchool,CityUniversityofLondon,FacultyofActuarialScienceand
Insurance,106BunhillRow,LondonEC1Y8TZ,UnitedKingdom
T.Rodrigues
DepartamentodeEstatistica,UniversidadedeBrasilia,Brasília,Brazil
Contributors xi
SuryaT.Tokdar
DukeUniversity,Durham,NC,UnitedStates
BerwinA.Turlach
TheUniversityofWesternAustralia,DepartmentofMathematicsandStatistics,
35StirlingHighway,Crawley,WA6009,Australia
MarinaVannucci
RiceUniversity,DepartmentofStatistics,6100MainSt,Houston,TX77005,
UnitedStates
Preface
The2000sand2010shaveseenhugegrowthinBayesianmodelling,which
nowfindsapplicationinfieldsasdiverseasengineering,law,medicine,psy-
chology, astronomy, climate science and philosophy. Much of the increase
in popularity is due to advances in Bayesian computation, most notably
Markov chain Monte Carlo methods. The availability of general and easily
applicable simulation-based computational algorithms has made it easier to
build more realistic models, involving greater complexity and high dimen-
sionality.
Introductory textbook accounts of Bayesian regression inference often
focus on rather inflexible parametric models. When planning this book,
we wanted to bring together, in a single volume, a discussion of Bayesian
regression methods allowing three types of flexibility: flexibility in the re-
sponselocation,flexibilityintheresponse-covariaterelationship,andflexi-
bilityintheerrordistributions.Theaimistoproduceacollectionofworks
accessibletopractitioners,whileatthesametimedetailedenoughforinter-
estedresearchersinBayesianmethods.Softwareimplementingthemethods
in the book is also available.
Chapters 1 and 2 cover quantile regression. These are methods where
inferential interest may lie away from the mean, in noncentral parts of
the distribution. Quantile methods do not specify an error model, and
are therefore challenging to implement in the Bayesian setting. Chapters 3
and 4 cover regression using Dirichlet process (DP) mixtures to flexibly
capture the unknown error distribution. In Chapter 3, DP mixtures are
consideredin anordinal regressionsetting,wheretherelationship between
the covariates and response is modelled flexibly via density regression. In
Chapter 4, DP mixtures are used for time series. Chapter 5 extends to
regression with multivariate response, using the copula approach to han-
dle mixed binary-continuous responses. Chapters 6 and 7 cover scalable
BayesianmodellingusingvariationalBayesianinference:inChapter6,vari-
ational inference is described in detail for various spline-based models to
flexibly model the covariate-response relationship in the mean. Chapter 7
developsavariationalalgorithmforcountresponsedata,inthepresenceof
variableselection.Finally,Chapters8and9showcasesomeoftheflexibility
of the Bayesian methods when models incorporate shape constraints. The
chapters of the book often deal with quite specialised and complex models
xiii
xiv Preface
and data types, but some general themes emerge from the discussion. The
reader will obtain an understanding of the basic modelling and computa-
tionalbuildingblockswhicharefundamentaltosuccessfulnewapplications
of modern and flexible Bayesian regression methods.
Each of the chapters is written in an easy to follow, tutorial style, with
the aim to encourage practitioners to take advantage of powerful Bayesian
regression methodology. Computer codes are available for each chapter at
the website
https://www.elsevier.com/books/
flexible-bayesian-regression-modelling/fan/978-0-12-815862-3
Wherever appropriate, the chapters contain instructions on how to use
the codes.
We are proud to be able to bring together a book containing the lat-
est developments in flexible Bayesian methods. We warmly thank all the
contributors to this project.
Yanan Fan
David Nott
Michael S. Smith
Jean-Luc Dortet-Bernadet