Bayesian Methods for the Physical Sciences: Learning from Examples in Astronomy and Physics

245 Pages·2015·6.542 MB·English

by  Stefano Andreon, Brian Weaver (auth.)

Springer Series in Astrostatistics
Stefano Andreon
Brian Weaver
Bayesian
Methods for
the Physical
Sciences
Learning from Examples in Astronomy
and Physics
Springer Series in Astrostatistics
Editor-in-chief:JosephM.Hilbe,JetPropulsionLaboratory,and
ArizonaStateUniversity,USA
JogeshBabu,ThePennsylvaniaStateUniversity,USA
BruceBassett,UniversityofCapeTown,SouthAfrica
SteffenLauritzen,OxfordUniversity,UK
ThomasLoredo,CornellUniversity,USA
OlegMalkov,MoscowStateUniversity,Russia
Jean-LucStarck,CEA/Saclay,France
DavidvanDyk,ImperialCollege,London,UK
SpringerSeriesinAstrostatistics,
Moreinformationaboutthisseriesathttp://www.springer.com/series/1432
Springer Series in Astrostatistics
AstrostatisticalChallengesfortheNewAstronomy:ed.JosephM.Hilbe
AstrostatisticsandDataMining:ed.LuisManuelSarro,LaurentEyer,William
O’Mullane,JorisDeRidder
StatisticalMethodsforAstronomicalDataAnalysis:byAsisKumarChattopadhyay
&TanukaChattopadhyay
Stefano Andreon • Brian Weaver
Bayesian Methods
for the Physical Sciences
Learning from Examples in Astronomy
and Physics
123
StefanoAndreon BrianWeaver
INAF StatisticalSciences
OsservatorioAstronomicodiBrera LosAlamosNationalLaboratory
Milano,Italy LosAlamos,NM,USA
ISSN2199-1030 ISSN2199-1049 (electronic)
SpringerSeriesinAstrostatistics
ISBN978-3-319-15286-8 ISBN978-3-319-15287-5 (eBook)
DOI10.1007/978-3-319-15287-5
LibraryofCongressControlNumber:2015932855
SpringerChamHeidelbergNewYorkDordrechtLondon
©SpringerInternationalPublishingSwitzerland2015
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
SpringerInternationalPublishingAGSwitzerlandispartofSpringerScience+BusinessMedia(www.
springer.com)
Preface
This book is a consultant’s guide for the researcher in astronomy or physics who
is willing to analyze his (or her) own data by offering him (or her) a statistical
background, some numerical advice, and a large number of examples of statisti-
calanalysesofreal-worldproblems,manyfromtheexperiencesofthefirstauthor.
Whilewritingthisbook,weplacedourselvesintheroleoftheresearcherwillingto
analyzehis/herowndatabutlackingapracticalwaytodoit(infactoneofuswas
thisonce).Forthisreason,wechoosetheJAGSsymboliclanguagetoperformthe
Bayesianfitting,allowingus(theresearchers)tofocusontheresearchapplications,
not on programming (coding) issues. By using JAGS, it is easy for the researcher
totakeoneofthemanyapplicationspresentedinthisbookandmorphthemintoa
formthatisrelevanttohisneeds.Morethan50examplesarelistedandareintended
to be the starting points for the researchers, so they can develop the confidence to
solvetheirownproblem.Allexamplesareillustratedwith(morethan100)figures
showing the data, the fitted model, and its uncertainty in a way that researchers
in astronomy and physics are used to seeing them. All models and datasets used
in this book are made available at the site http://www.brera.mi.astro.it/∼andreon/
BayesianMethodsForThePhysicalSciences/.
Thetopicsinthisbookincludeanalyzingmeasurementssubjecttoerrorsofdif-
ferentamplitudesandwithalarger-than-expectedscatter,dealingwithupperlimits
and with selection effects, modeling of two populations (one we are interested in
andanuisancepopulation),regression(fitting)modelswithandwithouttheabove
“complications” and population gradients, and how to predict the value of an un-
available quantity by exploiting the existence of a trend with another, available,
quantity. The book also provides some advice to answer the following questions:
Is the considered model at odds with the fitted data? Furthermore, are the results
constrained by the data or unduly affected by assumptions? These are the “bread
andbutter”activitiesofresearchersinphysicsandastronomyandalsotoallthose
analyzing and interpreting datasets in other branches as far as they are confronted
withsimilarproblems(measurementerrors,varietyinthestudiedpopulation,etc.).
Thebook alsoserves asaguide for(andhas been shaped thanks to)Ph.D.stu-
dentsinthephysicalandspacesciences:bydedicating12hofstudyingthecontent
v
vi Preface
ofthisbook,theaveragestudentisabletowritealoneandunsupervisedthecomplex
modelsinChap.8(andindeedtheexampleinSect.8.2waswrittenbyastudent).
Content
Thebookconsistsoftenchapters.Afterashortdescriptiononhowtousethebook
(Chap. 1), Chaps. 2 and 3 are a gentle introduction to the theory behind Bayesian
methods and how to practically compute the target numbers (e.g., parameter esti-
matesofthemodel).Inlaterchapters,wepresentproblemsofincreasingdifficulty
with an emphasis on astronomy examples. Each application in the book has the
same structure: it presents the problem, provides the data (listed in the text, or in
the case of large datasets, a URL is provided) or shows how to generate the data,
hasadetailedanalysisincludingquantitativesummariesandfigures(morethan100
figures), and provides the code used for the analysis (using again the open-source
JAGSprogram).Theideaisthatthereaderhasthecapabilitytofullyreproduceour
analysis. The provided JAGS code (of which the book contain over 50 examples)
can be the starting point for a researcher applying Bayesian methods to different
problems.Thekeysubjectofmodelcheckingandsensitivityanalysisisaddressed
inChap.9.Lastly,Chap.10providessomecomparisonsoftheBayesianapproachto
olderwaysofdoingstatistics(i.e.,frequentistmethods)inastronomyandphysics.
Mostchaptersconcludewithasetofexercises.Anexercisesolutionbookisavail-
ableatthesameURLgivenabove.
ContactDetails
[e-mail]@
lanl.gov.Updatedprogramsandanerratumarekeptathttp://www.brera.mi.astro.it/
∼andreon/BayesianMethodsForThePhysicalSciences/.
Acknowledgments
SAthanksMerrileeHurnforhercollaborationsthatinspiredmuchofthecontentof
thisbook,SteveGullforenjoyablediscussionsonthisbook’scontentandfortheuse
ofhisofficeinCambridge(UK),JoeHilbeforencouragementstocontinuetheeffort
of writing this book, Andrew Gelman for his reactiveness in answering questions,
AndreaBenagliafortheexampleinSect.8.2,andGiulioD’Agostiniforcomputing
theanalyticalexpressionoftheposteriorprobabilitydistributionforafewproblems
of astronomical relevance. The practical way to summarize our knowledge about
the quantity under study, more than philosophical debates, is maximally useful to
researchers.
BWthanksWilliamMeekerandMaxMorrisforteachinghimhowtobeaprac-
ticalstatistician.HealsothanksMichaelHamada,AlysonWilson,andHarryMartz
fortakingayoungfrequentistandopeninghiseyestothebeautyofBayesianstatis-
tics,andDavidHigdonforteachinghimhowtohavefunbeingastatistician.
Lastly,wethankourbeautifulwivesfortheirencouragementandsupport.
Finally,awarmthanksgoestoMartinPlummerforJAGS,i.e.,thesoftwareused
toperformthenumericalcomputationsofthisbook.TheeaseofuseofJAGSmade
ourdailyresearchworkeasierandallowsustoeasilyexploreotherpossiblemodels
andthesensitivityofourconclusionsto(unavoidable)assumptions.
Contents
1 RecipesforaGoodStatisticalAnalysis ........................... 1
2 ABitofTheory ................................................ 3
2.1 Axiom1:ProbabilitiesAreintheRangeZerotoOne............. 3
2.2 Axiom2:WhenaProbabilityIsEitherZeroorOne.............. 3
2.3 Axiom3:TheSum,orMarginalization,Axiom ................. 4
2.4 ProductRule .............................................. 5
2.5 BayesTheorem ............................................ 5
2.6 ErrorPropagation .......................................... 7
2.7 BringingItAllHome ....................................... 8
2.8 ProfilingIsNotMarginalization .............................. 8
2.9 Exercises ................................................. 10
References..................................................... 13
3 ABitofNumericalComputation................................. 15
3.1 SomeTechnicalities ........................................ 17
3.2 HowtoSamplefromaGenericFunction ....................... 18
References..................................................... 20
4 SingleParameterModels ....................................... 21
4.1 Step-by-StepGuideforBuildingaBasicModel ................. 21
4.1.1 ALittleBitof(Science)Background.................... 21
4.1.2 BayesianModelSpecification.......................... 22
4.1.3 ObtainingthePosteriorDistribution..................... 22
4.1.4 BayesianPointandIntervalEstimation .................. 23
4.1.5 CheckingChainConvergence.......................... 24
4.1.6 ModelCheckingandSensitivityAnalysis................ 25
4.1.7 ComparisonwithOlderAnalyses....................... 25
4.2 OtherUsefulDistributionswithOneParameter.................. 26
4.2.1 MeasuringaRate:Poisson ............................ 26
vii
viii Contents
4.2.2 CombiningTwoorMore(Poisson)Measurements ........ 28
4.2.3 MeasuringaFraction:Binomial ........................ 28
4.3 Exercises ................................................. 32
References..................................................... 34
5 ThePrior ..................................................... 35
5.1 ConclusionsDependonthePrior.............................. 35
5.1.1 ...SometimesaLot:TheMalmquist-EddingtonBias ...... 35
5.1.2 ...byLowerAmountswithIncreasingDataQuality....... 37
5.1.3 ...butEventuallyBecomesNegligible .................. 39
5.1.4 ...andthePreciseShapeofthePriorOften
DoesNotMatter..................................... 40
5.2 WheretoFindPriors........................................ 41
5.3 WhyThereAreSoManyUniformPriorsinthisBook?........... 42
5.4 OtherExamplesontheInfluenceofPriorsonConclusions ........ 42
5.4.1 TheImportantRoleofthePriorintheDeterminationof
theMassoftheMostDistantKnownGalaxyCluster....... 42
5.4.2 TheImportanceofPopulationGradientsforPhotometric
Redshifts ........................................... 44
5.5 Exercises ................................................. 47
References..................................................... 49
6 Multi-parametersModels ....................................... 51
6.1 CommonSimpleProblems................................... 51
6.1.1 LocationandSpread.................................. 51
6.1.2 TheSourceIntensityinthePresenceofaBackground ..... 54
6.1.3 EstimatingaFractioninthePresenceofaBackground..... 59
6.1.4 SpectralSlope:HardnessRatio......................... 62
6.1.5 SpectralShape ...................................... 64
6.2 Mixtures.................................................. 67
6.2.1 ModelingaBimodalDistribution:TheCaseofGlobular
ClusterMetallicity ................................... 67
6.2.2 AverageofIncompatibleMeasurements ................. 73
6.3 AdvancedAnalysis ......................................... 75
6.3.1 SourceIntensitywithOver-PoissonBackground
Fluctuations......................................... 75
6.3.2 The Cosmological Mass Fraction Derived from the
Cluster’sBaryonFraction ............................. 77
6.3.3 LightConcentrationinthePresenceofaBackground ...... 80
6.3.4 AComplexBackgroundModelingforGeo-Neutrinos ..... 82
6.3.5 UpperLimitsfromCountingExperiments ............... 89
6.4 Exercises ................................................. 92
References..................................................... 96
Contents ix
7 Non-randomDataCollection .................................... 99
7.1 TheGeneralCase ..........................................100
7.2 SharpSelectionontheValue .................................102
7.3 SharpSelectionontheValue,MixtureofGaussians:Measuring
theGravitationalRedshift....................................103
7.4 SharpSelectionontheTrueValue.............................106
7.5 ProbabilisticSelectionontheTrueValue.......................109
7.6 SharpSelectionontheObservedValue,MixtureofGaussians .....111
7.7 NumericalImplementationoftheModels ......................113
7.7.1 SharpSelectionontheValue...........................113
7.7.2 SharpSelectionontheTrueValue ......................114
7.7.3 ProbabilisticSelectionontheTrueValue ................116
7.7.4 Sharp Selection on the Observed Value, Mixture of
Gaussians ..........................................117
7.8 FinalRemarks .............................................118
Reference......................................................119
8 FittingRegressionModels.......................................121
8.1 ClearingUpSomeMisconceptions............................121
8.1.1 PayAttentiontoSelectionEffects ......................121
8.1.2 AvoidFishingExpeditions ............................123
8.1.3 DoNotConfusePredictionwithParameterEstimation.....124
8.2 Non-linear Fit with No Error on Predictor and No Spread:
EfficiencyandCompleteness.................................128
8.3 FitwithSpreadandNoErrorsonPredictor:VaryingPhysical
Constants? ................................................131
8.4 FitwithErrorsandSpread:TheMagorrianRelation .............134
8.5 FitwithMoreThanOnePredictorandaComplexLink:Star
FormationQuenching.......................................137
8.6 FitwithUpperandLowerLimits:TheOptical-to-XFluxRatio ....141
8.7 FitwithAnImportantDataStructure:The
Mass-RichnessScaling......................................146
8.8 FitwithaNon-ignorableDataCollection.......................149
8.9 FitWithoutAnxietyAboutNon-randomDataCollection .........154
8.10 Prediction.................................................159
8.11 AMeta-Analysis:CombinedFitofRegressionswithDifferent
IntrinsicScatter ............................................165
8.12 AdvancedAnalysis .........................................168
8.12.1 CosmologicalParametersfromSNIa....................168
8.12.2 TheEnrichmentHistoryoftheICM.....................175
8.12.3 TheEnrichmentHistoryAfterBinningbyRedshift........181
8.12.4 WithAnOver-PoissonsSpread.........................182
8.13 Exercises .................................................186
References.....................................................189