Mathematical Statistics: A Unified Introduction PDF
Preview Mathematical Statistics: A Unified Introduction
Mathematical Statistics: A Unified Introduction George R. Terrell Springer Teacher’s Preface Why another textbook? The statistical community generally agrees that at the upper undergraduate level, or the beginning master’s level, students of statistics shouldbegintostudythemathematicalmethodsofthefield.Weassumethatby thentheywillhavestudiedtheusualtwo-yearcollegesequence,includingcalculus through multiple integrals and the basics of matrix algebra. Therefore, they are readytolearnthefoundationsoftheirsubject,inmuchmoredepththanisusual inanapplied,“cookbook,”introductiontostatisticalmethodology. There are a number of well-written, widely used textbooks for such a course. Theseseemtoreflectaconsensusforwhatneedstobetaughtandhowitshould betaught.So,whydoweneedyetanotherbookforthisspotinthecurriculum? Ilearnedmathematicalstatisticswiththehelpofthestandardtexts.Sincethen, Ihavetaughtthiscourseandsimilaronesmanytimes,atseveraldifferentuniversi- ties,usingwell-thought-oftextbooks.Butfromthebeginning,Ifeltthatsomething waswrong.Ittookmeseveralyearstoarticulatetheproblem,andmanymoreto assemblemysolutionintothebookyouhaveinyourhand. You see, I spend the rest of my day in statistical consulting and statistical re- search.Ishouldhavebeenpreparingmymathematicalstatisticsstudentstojoin meinthisexcitingwork.Butfromseeingwhatthebettergraduatingseniorsand beginninggraduatestudentsusuallyknew,Iconcludedthatthestandardcurricu- lumwasnotteachingthemtobesophisticatedcitizensofthestatisticalcommunity. Theseablestudentsseemedtobewellinformedaboutasetofnarrow,technical issuesandatthesametimeembarrassinglylackinginanyunderstandingofmore fundamental matters. For example, many of them could discourse learnedly on which sources of variation were testable in complicated linear models. But they becametongue-tiedwhenaskedtoexplain,inEnglish,whatthepresenceofsome interactionmeantforthereal-worldexperimentunderdiscussion! vi Teacher’sPreface Whatwentwrong?Ihavecometobelievethattheproblemliesinourhistory. The first modern textbooks were written in the 1950s. This was at the end of the Heroic Age of statistics, roughly, the first half of the twentieth century. Two bodiesofmagnificentachievementsmarkthatera.Thefirst,identifiedwithStudent, Fisher,Neyman,Pearson,andmanyothers,developedthephilosophyandformal methodology of what we now call classical inference. The analysis of scientific experimentsbecamesostraightforwardthatthesetechniquesswepttheworldof applications. Many of our clients today seem to believe that these methods are statistics. Thesecond,associatedwithLiapunov,Kolmogorov,andmanyothers,wasthe formalmathematicizationofprobabilityandstatistics.Theseresearchersproved precisecentrallimittheorems,stronglawsoflargenumbers,andlawsoftheiterated logarithm(letmecalltheseadvancedasymptotics).Theyaxiomatizedprobability theory and placed distribution theory on a rigorous foundation, using Lebesgue integrationandmeasuretheory. By the 1950s, statisticians were dazzled by these achievements, and to some extent we still are. The standard textbooks of mathematical statistics show it. Unfortunately, this causes problems for teachers. Measure theory and advanced asymptoticsarestillwellbeyondthesophisticationofmostundergraduates,sowe cannotreallyteachthematthislevel.Furthermore,toomuchclassicalinference leadsustoneglecttheprecedingtwocenturiesofpowerfulbutlessformalmeth- ods,nottomentionthebroadadvancesofthelast50years:Bayesianinference, conditionalinference,likelihood-basedinference,andsoforth. Sothestandardtextbooksstartwithlong,dry,introductionstoabstractprobabil- ityanddistributiontheory,almostdevoidofstatisticalmotivationsandexamples (pokerproblems?!).Thenthereisafranticrush,againlargelyunmotivated,tointro- duceexactlythosedistributionsthatwillbeneededforclassicalinference.Finally, two-thirdsofthewaythrough,thefirstrealstatisticalapplicationsappear—means tests,one-wayANOVA,etc.—butrigidlyconfinedwithintheclassicalinferential framework.(Anearlyreaderofthemanuscriptcalledthis“thecultofthet-test.”) Finally,inperhapsChapter14,thebooksgettolinearregression.Now,regression is200yearsold,easy,intuitive,andincrediblyuseful.Unfortunately,ithasbeen madeverydifficult:“conditioningofmultivariateGaussiandistributions”asone cultistputit.Fortunately,itappearssolateinthetermthatitgetsomittedanyway. We distort the details of teaching, too, by our obsession with graduate-level rigor.Large-sampletheoryisattheheartofstatisticalthinking,butweareafraid totouchit.“Asymptoticsconsistsofcorollariestothecentrallimittheorem,”as another cultist puts it. We seem to have forgotten that 200 years of what I shall callelementaryasymptoticsprecededLiapunov’swork.Furthermore,thefearof sayinganythingthatwillhavetobemodifiedlater(ingraduateclassesthatassume measuretheory)forcesundergraduatemathematicalstatisticstextstoincludevery littlerealmathematics. Asaresult,mostofthesestandardtextsarehardlydifferentfromthecookbooks, withafewintegralstossedinforflavor,likejalapen˜obitsincornbread.Othersare spicedwithdefinitionsandtheoremshedgedaboutwithverytechnicalconditions, Teacher’sPreface vii which are never motivated, explained, or applied (remember “regularity condi- tions”?).Mathematicalproofs,surelyabasictoolforunderstanding,areconfined to a scattering of places, chosen apparently because the arguments are easy and “elegant.”Elsewhere,thedemoralizingrefrainbecomes“theproofisbeyondthe scopeofthiscourse.” Howisthisbookdifferent?Inshort,thisbookisintendedtoteachstudentsto domathematicalstatistics,notjusttoappreciateit.Therefore,Ihaveredesignedthe coursefromfirstprinciples.Ifyouarefamiliarwithastandardtextbookonthesub- jectandyouopenthisoneatrandom,youareverylikelytofindeitherasurprising topicoranunexpectedtreatmentorplacementofastandardtopic.Buteverything ishereforareason,anditsorderofappearancehasbeencarefullychosen. First, as the subtitle implies, the treatment in unified. You will find here no artificial separation of probability from statistics, distribution theory from infer- ence,orestimationfromhypothesistesting.Itreatprobabilityasamathematical handmaiden of statistics. It is developed, carefully, as it is needed. A statistical motivationforeachaspectofprobabilitytheoryisthereforeprovided. Second, I have updated the range of subjects covered. You will encounter in- troductionstosuchimportantmoderntopicsasloglinearmodelsforcontingency tables and logistic regression models (very early in the book!), finite population sampling,branchingprocesses,andsmall-sampleasymptotics. More important are the matters I emphasize systematically. Asymptotics is a major theme of this book. Many large-sample results are not difficult and quite appropriatetoanundergraduatecourse.Forexample,Ihadalwaystaughtthatwith “largen,smallp”onemayusethePoissonapproximationtobinomialprobabil- ities. Then I would be embarrassed when a student asked me exactly when this worked.Sowederivehereasimple,usefulerrorboundthatanswersthisquestion. Naturally,afullmoderncentrallimittheoremismathematicallyabovethelevelof thiscourse.Butagreatnumberofusefulyetmoreelementarynormallimitresults exist,andmanyarederivedhere. I emphasize those methods and concepts that are most useful in statistics in the broad sense. For example, distribution theory is motivated by detailed study ofthemostwidelyusefulfamiliesofrandomvariables.Classicalestimationand hypothesis testing are still dealt with, but as applications of these general tools. Simultaneously,Bayesian,conditional,andotherstylesofinferenceareintroduced aswell. Thestandardtextbooks,unfortunately,tendtointroduceveryobscureandab- stractsubjects“cold”(wheredidahorribleexpressionlike√1 e−x2/2comefrom?), 2π thenonlybelatedlygetaroundtomotivatingthemandgivingexamples.Herewe insistonconcreteness.Thebookprecedeseachnewtopicwitharelevantstatistical problem. We introduce abstract concepts gradually, working from the special to thegeneral.Atthesametime,eachnewtechniqueisappliedaswidelyaspossible. Thus, every chapter is quite broad, touching on many connections with its main topics. Thebook’sattitudetowardmathematicsmaysurpriseyou:Wetakeitseriously. Ourstudentsmaynotknowmeasuretheory,buttheydoknowanenormousamount viii Teacher’sPreface ofusefulmathematics.Thistextuseswhattheydoknowandteachesthemmore. We aim for reasonable completeness: Every formula is derived, every property is proved (often, students are asked to complete the arguments themselves as exercises).Thelevelofmathematicalprecisionandgeneralityisappropriatetoa seriousupper-levelundergraduatecourse. Atthesametime,studentsarenotexpectedtomemorizeexotictechnicalities, relevantonlyingraduateschool.Forexample,thebookdoesnotburdenthemwith theinfamous“triple”definitionofarandomvariable;alessobscuredefinitionis adequateforourworkhere.(Thosestudentswhogoontograduatemathematical statistics courses will be just the ones who will have no trouble switching to themoreabstractpointofviewlater.)Furthermore,weemphasizemathematical directness: Those short, elegant proofs so prized by professors are often here replacedbyslightlylongerbutmoreconstructivedemonstrations.Ourgoalisto stimulateunderstanding,nottodazzlewithourbrilliance. Whatisinthebook?Thesepedagogicalprinciplesimposeanunconventional orderoftopics.Letmetakeyouonabrieftourofthebook: The “Getting Started” chapter motivates the study of statistics, then prepares thestudentforhands-oninvolvement:completingproofsandderivationsaswell asworkingproblems. Chapter 1 adopts an attitude right away: Statistics precedes probability. That is, models for important phenomena are more important than models for mea- surement and sampling error. The first two chapters do not mention probability. We start with the linear data-summary models that make up so much of statisti- calpractice:one-waylayoutsandfactorialmodels.Fundamentalconceptssuchas additivityandinteractionappearnaturally.Thesimplestlinearregressionmodels followbyinterpolation.Thenweconstructsimplecontingency-tablemodelsfor counting experiments and thereby discover independence and association. Then wetakelogarithms,toderiveloglinearmodelsforcontingencytables(whichare strikingly parallel to our linear models). Again, logistic regression models arise byinterpolation.Inthischapter,ofcourse,werestrictourselvestocasesforwhich reasonableparameterestimatesareobvious. Chapter2showshowtoestimateANOVAandregressionmodelsbytheancient, intuitivemethodofleastsquares.Weemphasizegeometricalinterpolationofthe method—shortestEuclideandistance.Thismotivatessamplevariance,covariance, andcorrelation.DecompositionofthesumofsquaresinANOVAandinsightinto degreesoffreedomfollownaturally. That is as far as we can go without models for errors, so Chapter 3 begins withaconventionalintroductiontocombinatorialprobability.Itis,however,very concrete: We draw marbles from urns. Rather than treat conditional probability as a later, artificially difficult topic, we start with the obvious: All probabilities are conditional. It is just that a few of them are conditional on a whole sample space.Thenthefirstasymptoticresultisobtained,toaidintheunderstandingof thefamous“birthdayproblem.”Thisleadstoinsightintothedifferencebetween finitepopulationandinfinitepopulationsampling. Teacher’sPreface ix Chapter4usesgeometricalexamplestointroducecontinuousprobabilitymod- els.Thenwegeneralizetoabstractprobability.Theaxiomsweusecorrespondto howoneactuallycalculatesprobability.Wegoontogeneraldiscreteprobability, andBayes’stheorem.ThechapterendswithanelementaryintroductiontoBorel algebraasabasisforcontinuousprobabilities. Chapter 5 introduces discrete random variables. We start with finite popula- tion sampling, in particular, the negative hypergeometric family. You may not be familiar with this family, but the reasons to be interested are numerous: (1) Manycommonrandomvariables(binomial,negativebinomial,Poisson,uniform, gamma, beta, and normal) are asymptotic limits of this family; (2) it possesses intransparentwaysthesymmetriesanddualitiesofthosefamilies;and(3)itbe- comesparticularlyeasyforthestudenttocarryouthisownsimulations,viaurn models.ThentheFisherexacttestgivesusthefirstexampleofanhypothesistest, for independence in the 2×2 tables we studied in Chapter 1. We introduce the expectation of discrete random variables as a generalization of the average of a finitepopulation.Finally,wegivethefirstestimatesforunknownparametersand confidenceboundsforthem. Chapter6introducesthegeometric,negativebinomial,binomial,andPoisson families.Wediscoverthatthefirstthreeariseasasymptoticlimitsinthenegative hypergeometric family and also as sequences of Bernoulli experiments. Thus, wehaverelatedfiniteandinfinitepopulationsampling.Weinvestigatejustwhen thePoissonfamilymaybeusedasanasymptoticapproximationinthebinomial andnegativebinomialfamilies.Generaldiscreteexpectationsandthepopulation variancearethenintroduced.Confidenceintervalsandtwo-sidedhypothesistests providenaturalapplications. Chapter 7 introduces random vectors and random samples. Here is where marginalandconditionaldistributionsappear,andfromthese,populationcovari- anceandcorrelation.Thistellsussomethingsaboutthedistributionofthesample meanandvariance,andleadstothefirstlawsoflargenumbers.Thestudyofcon- ditionaldistributionspermitsthefirstexamplesofparametricBayesianinference. Chapter8investigatesparameterestimationandevaluationoffitincomplicated discretemodels.Weintroducethediscretelikelihoodandthelog-likelihoodratio statistic. This turns out often to be asymptotically equivalent to Pearson’s chi- squaredstatistic,butitismuchmoregenerallyuseful.Thenweintroducemaximum likelihoodestimationandapplyittologlinearcontingencytablemodels;estimates arecomputedbyiterativeproportionalfitting.Weestimatelinearlogisticmodels bymaximumlikelihood,evaluatedbyNewton’smethod. Chapter 9 constructs the Poisson process, from which we obtain the gamma family.ThenaDirichletprocessisconstructed,fromwhichwegetthebetafamily. Connectionsbetweenthesetwofamiliesareexplored.Thecontinuousversionof thelikelihoodratioisintroduced,andweuseittoestablishtheNeyman–Pearson lemma. Chapter10definesthegeneralquantilefunctionofarandomvariable,byasking how we might simulate it. Then we may define the expectation of any random
The list of books you might like
