Essentials of Statistical Inference

238 Pages·2005·2.929 MB·English

by  G. A. Young, R. L. Smith

This page intentionally left blank
Essentials of Statistical Inference
EssentialsofStatisticalInferenceisamodernandaccessibletreatmentoftheprocedures
usedtodrawformalinferencesfromdata.Aimedatadvancedundergraduateandgraduate
studentsinmathematicsandrelateddisciplines,aswellasthoseinotherfieldsseekinga
concisetreatmentofthekeyideasofstatisticalinference,itpresentstheconceptsandresults
underlyingtheBayesian,frequentistandFisherianapproaches,withparticularemphasison
the contrasts between them. Contemporary computational ideas are explained, as well as
basicmathematicaltheory.
Writteninalucidandinformalstyle,thisconcisetextprovidesbothbasicmaterialonthe
mainapproachestoinference,aswellasmoreadvancedmaterialonmoderndevelopments
in statistical theory, including: contemporary material on Bayesian computation, such as
MCMC,higher-orderlikelihoodtheory,predictiveinference,bootstrapmethodsandcon-
ditional inference. It contains numerous extended examples of the application of formal
inferencetechniquestorealdata,aswellashistoricalcommentaryonthedevelopmentof
thesubject.Throughout,thetextconcentratesonconcepts,ratherthanmathematicaldetail,
whilemaintainingappropriatelevelsofformality.Eachchapterendswithasetofaccessible
problems.
BasedtoalargeextentonlecturesgivenattheUniversityofCambridgeoveranumberof
years,thematerialhasbeenpolishedbystudentfeedback.Somepriorknowledgeofprob-
abilityisassumed,whilesomepreviousknowledgeoftheobjectivesandmainapproaches
tostatisticalinferencewouldbehelpfulbutisnotessential.
G.A.YoungisProfessorofStatisticsatImperialCollegeLondon.
R.L.SmithisMarkL.ReedDistinguishedProfessorofStatisticsatUniversityofNorth
Carolina,ChapelHill.
CAMBRIDGE SERIES IN STATISTICAL AND
PROBABILISTIC MATHEMATICS
EditorialBoard
R.Gill(DepartmentofMathematics,UtrechtUniversity)
B.D.Ripley(DepartmentofStatistics,UniversityofOxford)
S.Ross(DepartmentofIndustrialEngineering,UniversityofCalifornia,Berkeley)
B.W.Silverman(St.Peter’sCollege,Oxford)
M.Stein(DepartmentofStatistics,UniversityofChicago)
This series of high-quality upper-division textbooks and expository monographs covers
all aspects of stochastic applicable mathematics. The topics range from pure and applied
statistics to probability theory, operations research, optimization, and mathematical pro-
gramming. The books contain clear presentations of new developments in the field and
alsoofthestateoftheartinclassicalmethods.Whileemphasizingrigoroustreatmentof
theoreticalmethods,thebooksalsocontainapplicationsanddiscussionsofnewtechniques
madepossiblebyadvancesincomputationalpractice.
Alreadypublished
1. BootstrapMethodsandTheirApplication,byA.C.DavisonandD.V.Hinkley
2. MarkovChains,byJ.Norris
3. AsymptoticStatistics,byA.W.vanderVaart
4. WaveletMethodsforTimeSeriesAnalysis,byDonaldB.PercivalandAndrewT.Walden
5. BayesianMethods,byThomasLeonardandJohnS.J.Hsu
6. EmpiricalProcessesinM-Estimation,bySaravandeGeer
7. NumericalMethodsofStatistics,byJohnF.Monahan
8. AUser’sGuidetoMeasureTheoreticProbability,byDavidPollard
9. TheEstimationandTrackingofFrequency,byB.G.QuinnandE.J.Hannan
10. DataAnalysisandGraphicsusingR,byJohnMaindonaldandJohnBraun
11. StatisticalModels,byA.C.Davison
12. SemiparametricRegression,byD.Ruppert,M.P.Wand,R.J.Carroll
13. ExercisesinProbability,byLoicChaumontandMarcYor
14. StatisticalAnalysisofStochasticProcessesinTime,byJ.K.Lindsey
15. MeasureTheoryandFiltering,byLakhdarAggounandRobertElliott
Essentials of Statistical Inference
G. A. Young
DepartmentofMathematics,ImperialCollegeLondon
R. L. Smith
DepartmentofStatisticsandOperationsResearch,
UniversityofNorthCarolina,ChapelHill
cambridge university press
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo
Cambridge University Press
The Edinburgh Building, Cambridge cb2 2ru, UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridge.org/9780521839716
© Cambridge University Press 2005
This publication is in copyright. Subject to statutory exception and to the provision of
relevant collective licensing agreements, no reproduction of any part may take place
without the written permission of Cambridge University Press.
First published in print format 2005
isbn-13 978-0-511-12616-1 eBook (NetLibrary)
isbn-10 0-511-12616-6 eBook (NetLibrary)
isbn-13 978-0-521-83971-6 hardback
isbn-10 0-521-83971-8 hardback
isbn-13 978-0-521-54866-3 paperback
isbn-10 0-521-54866-7 paperback
Cambridge University Press has no responsibility for the persistence or accuracy of urls
for external or third-party internet websites referred to in this publication, and does not
guaranteethatanycontentonsuchwebsitesis,orwillremain,accurateorappropriate.
Contents
Preface page ix
1 Introduction 1
2 Decision theory 4
2.1 Formulatihon 4
2.2 The risk function 5
2.3 Criteria for a good decision rule 7
2.4 Randomiseddecisionrules 11
2.5 Finitedecisionproblems 11
2.6 Findingminimaxrulesingeneral 18
2.7 AdmissibilityofBayesrules 19
2.8 Problems 19
3 Bayesianmethods 22
3.1 Fundamentalelements 22
3.2 ThegeneralformofBayesrules 28
3.3 Back to minimax. . . 32
3.4 Shrinkage and the James–Stein estimator 33
3.5 EmpiricalBayes 38
3.6 Choice of prior distributions 39
3.7 Computationaltechniques 42
3.8 Hierarchicalmodelling 48
3.9 Predictivedistributions 52
3.10 Dataexample:Coal-miningdisasters 55
3.11 Dataexample:Geneexpressiondata 57
3.12 Problems 60
4 Hypothesistesting 65
4.1 Formulationofthehypothesistestingproblem 65
4.2 TheNeyman–PearsonTheorem 68
4.3 Uniformlymostpowerfultests 69
4.4 Bayesfactors 73
4.5 Problems 78
vi Contents
5 Specialmodels 81
5.1 Exponentialfamilies 81
5.2 Transformationfamilies 86
5.3 Problems 88
6 Sufficiencyandcompleteness 90
6.1 Definitionsandelementaryproperties 90
6.2 Completeness 94
6.3 TheLehmann–Scheffe´ Theorem 95
6.4 Estimationwithconvexlossfunctions 95
6.5 Problems 96
7 Two-sidedtestsandconditionalinference 98
7.1 Two-sidedhypothesesandtwo-sidedtests 99
7.2 Conditionalinference,ancillarityandsimilartests 105
7.3 Confidencesets 114
7.4 Problems 117
8 Likelihoodtheory 120
8.1 Definitionsandbasicproperties 120
8.2 TheCrame´r–RaoLowerBound 125
8.3 Convergenceofsequencesofrandomvariables 127
8.4 Asymptoticpropertiesofmaximumlikelihoodestimators 128
8.5 LikelihoodratiotestsandWilks’Theorem 132
8.6 Moreonmultiparameterproblems 134
8.7 Problems 137
9 Higher-ordertheory 140
9.1 Preliminaries 141
9.2 Parameterorthogonality 143
9.3 Pseudo-likelihoods 145
9.4 Parametrisationinvariance 146
9.5 Edgeworthexpansion 148
9.6 Saddlepointexpansion 149
9.7 Laplaceapproximationofintegrals 152
9.8 Thep∗formula 153
9.9 Conditionalinferenceinexponentialfamilies 159
9.10 Bartlettcorrection 160
9.11 Modifiedprofilelikelihood 161
9.12 Bayesianasymptotics 163
9.13 Problems 164
10 Predictiveinference 169
10.1 Exactmethods 169
10.2 Decisiontheoryapproaches 172
10.3 Methodsbasedonpredictivelikelihood 175
10.4 Asymptoticmethods 179
Contents vii
10.5 Bootstrapmethods 183
10.6 Conclusionsandrecommendations 185
10.7 Problems 186
11 Bootstrapmethods 190
11.1 Aninferenceproblem 191
11.2 Theprepivotingperspective 194
11.3 Dataexample:Bioequivalence 201
11.4 Furthernumericalillustrations 203
11.5 Conditionalinferenceandthebootstrap 208
11.6 Problems 214
Bibliography 218
Index 223