Statistical Modelling by Exponential Families

297 Pages·2019·2.55 MB·english

by  Rolf Sundberg

StatisticalModellingbyExponentialFamilies
Thisbookisareadable,digestibleintroductiontoexponentialfamilies,encompassing
statisticalmodelsbasedonthemostusefuldistributionsinstatisticaltheory,suchas
thenormal,gamma,binomial,Poisson,andnegativebinomial.Stronglymotivatedby
applications,itpresentstheessentialtheoryandthendemonstratesthetheory’s
practicalpotentialbyconnectingitwithdevelopmentsinareassuchasitemresponse
analysis,socialnetworkmodels,conditionalindependenceandlatentvariable
structures,andpointprocessmodels.Extensionstoincompletedatamodelsand
generalizedlinearmodelsarealsoincluded.Inaddition,theauthorgivesaconcise
accountofthephilosophyofPerMartin-Lo¨finordertoconnectstatisticalmodelling
withideasinstatisticalphysics,suchasBoltzmann’slaw.Writtenforgraduate
studentsandresearcherswithabackgroundinbasicstatisticalinference,thebook
includesavastsetofexamplesdemonstratingmodelsforapplicationsandnumerous
exercisesembeddedwithinthetextaswellasattheendsofchapters.
ROLF SUNDBERG isProfessorEmeritusofStatisticalScienceatStockholm
University.Hisworkembracesboththeoreticalandappliedstatistics,including
principlesofstatistics,exponentialfamilies,regression,chemometrics,stereology,
surveysamplinginference,molecularbiology,andpaleoclimatology.In2003,hewon
(withM.Linder)theawardforbesttheoreticalpaperintheJournalofChemometrics
fortheirworkonmultivariatecalibration,andin2017hewasnamedStatisticianofthe
YearbytheSwedishStatisticalSociety.
INSTITUTE OF MATHEMATICAL STATISTICS
TEXTBOOKS
EditorialBoard
NancyReid(UniversityofToronto)
RamonvanHandel(PrincetonUniversity)
XumingHe(UniversityofMichigan)
SusanHolmes(StanfordUniversity)
ISBAEditorialRepresentative
PeterMu¨ller(UniversityofTexasatAustin)
IMSTextbooksgiveintroductoryaccountsoftopicsofcurrentconcernsuitablefor
advancedcoursesatmaster’slevel,fordoctoralstudentsandforindividualstudy.They
aretypicallyshorterthanafullydevelopedtextbook,oftenarisingfrommaterial
createdforatopicalcourse.Lengthsof100–290pagesareenvisaged.Thebooks
typicallycontainexercises.
IncollaborationwiththeInternationalSocietyforBayesianAnalysis(ISBA),
selectedvolumesintheIMSTextbooksseriescarrythe“withISBA”designationatthe
recommendationoftheISBAeditorialrepresentative.
OtherBooksintheSeries(*withISBA)
1. ProbabilityonGraphs,byGeoffreyGrimmett
2. StochasticNetworks,byFrankKellyandElenaYudovina
3. BayesianFilteringandSmoothing,bySimoSa¨rkka¨
4. TheSurprisingMathematicsofLongestIncreasingSubsequences,byDanRomik
5. NoiseSensitivityofBooleanFunctionsandPercolation,byChristopheGarban
andJeffreyE.Steif
6. CoreStatistics,bySimonN.Wood
7. LecturesonthePoissonProcess,byGu¨nterLastandMathewPenrose
8. ProbabilityonGraphs(SecondEdition),byGeoffreyGrimmett
9. IntroductiontoMalliavinCalculus,byDavidNualartandEula`liaNualart
10. AppliedStochasticDifferentialEquations,bySimoSa¨rkka¨andArnoSolin
11. *ComputationalBayesianStatistics,byM.Anto´niaAmaralTurkman,Carlos
DanielPaulino,andPeterMu¨ller
12. StatisticalModellingbyExponentialFamilies,byRolfSundberg
Statistical Modelling by
Exponential Families
ROLF SUNDBERG
StockholmUniversity
UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
314–321,3rdFloor,Plot3,SplendorForum,JasolaDistrictCentre,
NewDelhi–110025,India
79AnsonRoad,#06–04/06,Singapore079906
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781108476591
DOI:10.1017/9781108604574
©RolfSundberg2019
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2019
PrintedandboundinGreatBritainbyClaysLtd,ElcografS.p.A.
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
LibraryofCongressCataloguing-in-Publicationdata
Names:Sundberg,Rolf,1942–author.
Title:Statisticalmodellingbyexponentialfamilies/RolfSundberg(StockholmUniversity).
Description:Cambridge;NewYork,NY:CambridgeUniversityPress,2019.|
Series:InstituteofMathematicalStatisticstextbooks;12|
Includesbibliographicalreferencesandindex.
Identifiers:LCCN2019009281|ISBN9781108476591(hardback:alk.paper)|
ISBN9781108701112(pbk.:alk.paper)
Subjects:LCSH:Exponentialfamilies(Statistics)–Problems,exercises,etc.|
Distribution(Probabilitytheory)–Problems,exercises,etc.
Classification:LCCQA276.7.S862019|DDC519.5–dc23
LCrecordavailableathttps://lccn.loc.gov/2019009281
ISBN978-1-108-47659-1Hardback
ISBN978-1-108-70111-2Paperback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracyof
URLsforexternalorthird-partyinternetwebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
v
ToMargareta,
andtoPer;
celebrating50years
Contents
Examples ix
Preface xii
1 WhatIsanExponentialFamily? 1
2 ExamplesofExponentialFamilies 6
2.1 ExamplesImportantfortheSequel 6
2.2 ExamplesLessImportantfortheSequel 15
2.3 Exercises 21
3 RegularityConditionsandBasicProperties 24
3.1 RegularityandAnalyticalProperties 24
3.2 LikelihoodandMaximumLikelihood 31
3.3 AlternativeParameterizations 36
3.4 SolvingLikelihoodEquationsNumerically 45
3.5 ConditionalInferenceforCanonicalParameter 46
3.6 CommonModelsasExamples 50
3.7 CompletenessandBasu’sTheorem 59
3.8 MeanValueParameterandCrame´r–Rao(In)equality 61
4 AsymptoticPropertiesoftheMLE 64
4.1 LargeSampleAsymptotics 64
4.2 SmallSampleRefinement:SaddlepointApproximations 70
5 TestingModel-ReducingHypotheses 75
5.1 ExactTests 76
5.2 Fisher’sExactTestforIndependence,Homogeneity,Etc. 80
5.3 FurtherRemarksonStatisticalTests 84
5.4 LargeSampleApproximationoftheExactTest 86
5.5 AsymptoticallyEquivalentLargeSampleTests 90
5.6 APoissonTrickforDerivingTestStatistics 94
vi
Contents vii
6 Boltzmann’sLawinStatistics 100
6.1 MicrocanonicalDistributions 100
6.2 Boltzmann’sLaw 102
6.3 HypothesisTestsinaMicrocanonicalSetting 108
6.4 StatisticalReduncancy 109
6.5 AModellingExerciseintheLightofBoltzmann’sLaw 114
7 CurvedExponentialFamilies 118
7.1 IntroductoryExamples 118
7.2 BasicTheoryforMLEstimationandHypothesisTesting 124
7.3 StatisticalCurvature 129
7.4 MoreonMultipleRoots 131
7.5 ConditionalInferenceinCurvedFamilies 136
8 ExtensiontoIncompleteData 143
8.1 Examples 143
8.2 BasicProperties 147
8.3 TheEMAlgorithm 150
8.4 Large-SampleTests 155
8.5 IncompleteDatafromCurvedFamilies 155
8.6 BloodGroupsunderHardy–WeinbergEquilibrium 156
8.7 HiddenMarkovModels 159
8.8 GaussianFactorAnalysisModels 161
9 GeneralizedLinearModels 164
9.1 BasicExamplesandBasicDefinition 164
9.2 ModelswithoutDispersionParameter 169
9.3 ModelswithDispersionParameter 175
9.4 ExponentialDispersionModels 181
9.5 Quasi-Likelihoods 183
9.6 GLMsversusBox–CoxMethodology 184
9.7 MoreApplicationAreas 186
10 GraphicalModelsforConditionalIndependenceStructures 191
10.1 GraphsforConditionalIndependence 192
10.2 GraphicalGaussianModels 195
10.3 GraphicalModelsforContingencyTables 201
10.4 ModelsforMixedDiscreteandContinuousVariates 205
11 ExponentialFamilyModelsforSocialNetworks 210
11.1 SocialNetworks 210
11.2 TheFirstModelStage:BernoulliGraphs 211
11.3 MarkovRandomGraphs 212
viii Contents
11.4 IllustrativeToyExample,n=5 218
11.5 BeyondMarkovModels:GeneralERGMType 225
12 RaschModelsforItemResponseandRelatedModels 228
12.1 TheJointModel 229
12.2 TheConditionalModel 231
12.3 TestingtheConditionalRaschModelFit 234
12.4 RaschModelConditionalAnalysisbyLog-LinearModels 239
12.5 RaschModelsforPolytomousResponse 240
12.6 FactorAnalysisModelsforBinaryData 241
12.7 ModelsforRankData 243
13 ModelsforProcessesinSpaceorTime 246
13.1 ModelsforSpatialPointProcesses 246
13.2 TimeSeriesModels 254
14 MoreModellingExercises 258
14.1 GenotypesunderHardy–WeinbergEquilibrium 258
14.2 ModelforControlledMultivariateCalibration 259
14.3 RefindingsofRingedBirds 259
14.4 StatisticalBasisforPositronEmissionTomography 262
AppendixA StatisticalConceptsandPrinciples 265
AppendixB UsefulMathematics 268
B.1 SomeUsefulMatrixResults 268
B.2 SomeUsefulCalculusResults 269
Bibliography 271
Index 278
Examples
1.1 ThestructurefunctionforrepeatedBernoullitrials 4
2.1 Bernoullitrialsandthebinomial 6
2.2 Logisticregressionmodels 7
2.3 Poissondistributionfamilyforcounts 7
2.4 Themultinomialdistribution 8
2.5 MultiplicativePoissonandotherlog-linearmodels 9
2.6 ‘Timetofirstsuccess’(geometricdistribution) 10
2.7 Exponentialandgammadistributionmodels 10
2.8 Asamplefromthenormaldistribution 11
2.9 Gaussianlinearmodels 11
2.10 Themultivariatenormaldistribution 12
2.11 Covarianceselectionmodels 13
2.12 Exponentialtiltingfamilies 14
2.13 FiniteMarkovchains 15
2.14 VonMisesandFisherdistributionsfordirectionaldata 16
2.15 Maxwell–Boltzmannmodelinstatisticalphysics 17
2.16 TheIsingmodel 19
3.1 MarginalityandconditionalityforGaussiansample 41
3.2 TwoPoissonvariates 47
3.3 ConditionalinferenceforGaussiansample,cont’d 48
3.4 Bernoullitrials,continuedfromExample2.1 50
3.5 Logisticregression,continuedfromExample2.2 50
3.6 Conditionalinferenceinlogisticregression 51
3.7 Poissonsample,continuedfromExample2.3 52
3.8 MultiplicativePoissonmodel,continuedfromExample2.5 52
3.9 Exponentialandgamma,continuedfromExample2.7 53
3.10 AGaussiansample,continuedfromExample2.8 54
3.11 Themultivariatenormal,continuedfromExample2.10 55
3.12 Covarianceselectionmodels,continued 55
ix