Table Of Content

de Gruyter Textbook Wiebe R.Pestman Mathematical Statistics At a young age, the author’s inability to convert statistical data into round figures caused much distress Wiebe R. Pestman Mathematical Statistics Second revised edition ≥ Walter de Gruyter Berlin · New York Author Wiebe R.Pestman Centre for Biostatistics Utrecht University Padualaan 14 3584 CH Utrecht, The Netherlands Mathematics Subject Classification 2000: 62-01 Keywords:Estimationtheory,hypothesistesting,regressionanalysis,non-parametrics, stochastic analysis, vectorial (multivariate) statistics (cid:2)(cid:2)Printedonacid-freepaperwhichfallswithintheguidelines oftheANSItoensurepermanenceanddurability. BibliographicinformationpublishedbytheDeutscheNationalbibliothek TheDeutscheNationalbibliothekliststhispublicationintheDeutscheNationalbibliografie; detailedbibliographicdataareavailableintheInternetathttp://dnb.d-nb.de. ISBN 978-3-11-020852-8 (cid:2) Copyright2009byWalterdeGruyterGmbH&Co.KG,10785Berlin,Germany. All rights reserved, including those of translation into foreign languages. No part of this book maybereproducedinanyformorbyanymeans,electronicormechanical,includingphotocopy, recording, orany information storage andretrieval system, without permissionin writing from thepublisher. PrintedinGermany. Coverdesign:MartinZech,Bremen. Typesetusingtheauthor’sLATXfiles:KayDimler,Müncheberg. E Printingandbinding:Hubert&Co.GmbH&Co.KG,Göttingen. Preface ThisbookarosefromaseriesoflecturesIgaveattheUniversityofNijmegen(Hol- land). It presents an introduction to mathematical statistics and it is intended for studentsalreadyhavingsomeelementarymathematicalbackground. Inthetexttheo- retical results are presented as theorems, propositions or lemmas, of which as a rule rigorous proofs are given. In the few exceptions to this rule references are given to indicatewherethemissingdetailscanbefound. Inordertounderstandtheproofs,anelementarycourseincalculusandlinearalgebra isaprerequisite. However, ifthereaderisnotinterestedinstudyingproofs, thiscan simply be skipped. Instead one can study the many examples, in which applications ofthetheoreticalresultsareillustrated. Inthiswaythebookisalsousefultostudents intheappliedsciences. Tohavethestartingstatisticianwellpreparedinthefieldofprobabilitytheory,Chap- terIiswhollydevotedtothissubject. Nowadaysmanyscientificarticles(instatistics and probability theory) are presented in terms of measure theoretical notions. For this reason, Chapter I also contains a short introduction to measure theory (without getting submerged by it). However, the subsequent chapters can very well be read whenskippingthissectiononmeasuretheoryandmyadvicetothestudentreadingit isthereforenottogetboggeddownbyit. The aim of the remainder of the book is to give the reader a broad and solid base in mathematical statistics. Chapter II is devoted to estimation theory. The probability distributions of the usual elementary estimators, when dealing with normally distri- butedpopulations,aretreatedindetail. Furthermore,thereisinthischapterasection about Bayesian estimation. In the final sections of Chapter II, estimation theory is putintoageneralframework. Inthesesections,forexample,theinformationinequal- ity isproved. Moreover, theconcepts ofmaximumlikelihood estimation andthat of sufficiencyarediscussed. InChaptersIII,IVandVtheclassicalsubjectsinmathematicalstatisticsaretreated: hypothesis testing, normal regression analysis and normal analysis of variance. In thesechaptersthereisno“cheating”concerningthenotionofstatisticalindependence. ChapterVIpresentsanintroductiontonon-parametricstatistics. InChapterVIIitis illustratedhowstochasticanalysiscanbeappliedinmodernstatistics. Heresubjects like the Kolmogorov–Smirnov test, smoothing techniques, robustness, density estimation and bootstrap methods are treated. Finally, Chapter VIII is about vectorial vi Preface statistics. Isometimesfinditannoying,whenteachingthissubject,thatmanyunder- graduatestudentsdonotseem(orindeednotanymore)tobeawareofthefundamental theoremsinlinearalgebra. TomeetthisneedIhavegivenasummaryofthemainre- sultsofelementarylinearalgebrain§VIII.1. The book is written in quite simple English; this might be an advantage to students who do not have English as their native language (like myself). The text contains some 250 exercises, which vary in difficulty. Ivo Alberink, one of my students, has expoundeddetailedsolutionsofalltheseexercisesinabook,titled MathematicalStatistics. ProblemsandDetailedSolutions. Ivohasalsoreadthisbook(whichyounowhaveinfrontofyou)inacriticalway. The many discussions which arose from this have been very useful. I wish to thank him forthis. IherebyalsowishtothankIngeStappers,FlipKlijnandmanyotherstudents attheUniversityofNijmegen. Theytooreadthetextcriticallyandgaveconstructive comments. Furthermore I would like to thank Alessandro di Bucchianico and Mark vandeWielfromtheEindhovenUniversityofTechnologyforprovidingmostofthe statisticaltables. Finally,IamveryindebtedtothesecretariatofthedepartmentofmathematicsinNij- megen,inparticulartoHannyHeitink. Shedidalmostallthetypewriting,acolossal amountofwork. Nijmegen,November1998 WiebeR.Pestman Preface to the second edition The second edition of this book very much resembles the first. However, there have been some additions, modifications and corrections. As to the latter, I wish to thank readers for drawing my attention to a couple of errors in the first edition. Moreover I would like to mention here the very pleasant collaboration with Walter de Gruyter publishing. In particular I wish to thank Dr. Robert Plato, Kay Dimler and Simon Albroscheitfortheformidablejobtheyperformedinrealizingthissecondedition. Utrecht,March2009 WiebeR.Pestman Contents Preface v I Probabilitytheory 1 I.1 Probabilityspaces . . . . . . . . . . . . . . . . . . . . . . . . . 1 I.2 Stochasticvariables . . . . . . . . . . . . . . . . . . . . . . . . 9 I.3 Productmeasuresandstatisticalindependence . . . . . . . . . . 13 I.4 Functionsofstochasticvectors . . . . . . . . . . . . . . . . . . . 18 I.5 Expectation,varianceandcovarianceofstochasticvariables . . . 21 I.6 Statisticalindependenceofnormallydistributedvariables . . . . 30 I.7 Distributionfunctionsandprobabilitydistributions . . . . . . . . 38 I.8 Moments,momentgeneratingandcharacteristicfunctions . . . . 42 I.9 Thecentrallimittheorem . . . . . . . . . . . . . . . . . . . . . 48 I.10 Transformationofprobabilitydensities . . . . . . . . . . . . . . 53 I.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 II Statisticsandtheirprobabilitydistributions,estimationtheory 65 II.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 II.2 Thegammadistributionandthe 2-distribution . . . . . . . . . . 69 II.3 The t-distribution . . . . . . . . . . . . . . . . . . . . . . . . . 79 II.4 Statisticstomeasuredifferencesinmean . . . . . . . . . . . . . 83 II.5 The F-distribution . . . . . . . . . . . . . . . . . . . . . . . . . 89 II.6 Thebetadistribution . . . . . . . . . . . . . . . . . . . . . . . . 94 II.7 Populationsthatarenotnormallydistributed . . . . . . . . . . . 99 II.8 Bayesianestimation . . . . . . . . . . . . . . . . . . . . . . . . 102 II.9 Estimationtheoryinaframework . . . . . . . . . . . . . . . . . 111 II.10 Maximumlikelihoodestimation,sufficiency . . . . . . . . . . . 129 II.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 III Hypothesistests 155 III.1 TheNeyman–Pearsontheory. . . . . . . . . . . . . . . . . . . . 155 III.2 Hypothesistestsconcerningnormallydistributedpopulations . . 168 III.3 The 2-testongoodnessoffit . . . . . . . . . . . . . . . . . . . 182 III.4 The 2-testonstatisticalindependence . . . . . . . . . . . . . . 188 III.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 viii Contents IV Simpleregressionanalysis 200 IV.1 Theleastsquaresmethod. . . . . . . . . . . . . . . . . . . . . . 200 IV.2 Constructionofanunbiasedestimatorof 2 . . . . . . . . . . . 210 IV.3 Normalregressionanalysis. . . . . . . . . . . . . . . . . . . . . 213 IV.4 Pearson’sproduct-momentcorrelationcoefficient . . . . . . . . . 219 IV.5 Thesumofsquaresoferrorsasameasureoflinearstructure . . . 222 IV.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 V Normalanalysisofvariance 229 V.1 One-wayanalysisofvariance . . . . . . . . . . . . . . . . . . . 229 V.2 Two-wayanalysisofvariance . . . . . . . . . . . . . . . . . . . 236 V.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 VI Non-parametricmethods 250 VI.1 Thesigntest,Wilcoxon’ssigned-ranktest . . . . . . . . . . . . . 250 VI.2 Wilcoxon’srank-sumtest . . . . . . . . . . . . . . . . . . . . . 256 VI.3 Therunstest . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 VI.4 Rankcorrelationtests . . . . . . . . . . . . . . . . . . . . . . . 265 VI.5 TheKruskal–Wallistest . . . . . . . . . . . . . . . . . . . . . . 270 VI.6 Friedman’stest . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 VI.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 VII Stochasticanalysisanditsapplicationsinstatistics 282 VII.1 Theempiricaldistributionfunctionassociatedwithasample . . . 282 VII.2 Convergenceofstochasticvariables . . . . . . . . . . . . . . . . 284 VII.3 TheGlivenko–Cantellitheorem . . . . . . . . . . . . . . . . . . 303 VII.4 TheKolmogorov–Smirnovteststatistic . . . . . . . . . . . . . . 311 VII.5 Metricsonthesetofdistributionfunctions . . . . . . . . . . . . 316 VII.6 Smoothingtechniques . . . . . . . . . . . . . . . . . . . . . . . 328 VII.7 Robustnessofstatistics . . . . . . . . . . . . . . . . . . . . . . . 334 VII.8 Trimmedmeans,themedianandtheirrobustness . . . . . . . . . 341 VII.9 Statisticalfunctionals. . . . . . . . . . . . . . . . . . . . . . . . 358 VII.10 ThevonMisesderivativeandinfluencefunctions . . . . . . . . . 372 VII.11 Bootstrapmethods . . . . . . . . . . . . . . . . . . . . . . . . . 384 VII.12 Estimationofdensitiesbymeansofkerneldensities . . . . . . . 392 VII.13 Estimationofdensitiesbymeansofhistograms . . . . . . . . . . 401 VII.14 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 VIII Vectorialstatistics 419 VIII.1 Linearalgebra . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 VIII.2 Theexpectationandcovarianceofstochasticvectors . . . . . . . 434 VIII.3 Vectorialsamples . . . . . . . . . . . . . . . . . . . . . . . . . . 446 Contents ix VIII.4 Vectorialnormaldistributions . . . . . . . . . . . . . . . . . . . 450 VIII.5 ConditionalprobabilitydistributionsrelatedtoGaussianones . . 463 VIII.6 VectorialsamplesfromGaussiandistributedpopulations . . . . . 470 VIII.7 Vectorialversionsofthefundamentallimittheorems . . . . . . . 486 VIII.8 Normalcorrelationanalysis . . . . . . . . . . . . . . . . . . . . 497 VIII.9 Multipleregressionanalysis . . . . . . . . . . . . . . . . . . . . 505 VIII.10 Themultiplecorrelationcoefficient . . . . . . . . . . . . . . . . 517 VIII.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524 Appendix A Lebesgue’sconvergencetheorems 529 B Productmeasures 532 C Conditionalprobabilities 536 D ThecharacteristicfunctionoftheCauchydistribution 540 E Metricspaces,equicontinuity 543 F TheFouriertransformandtheexistenceofstoutlytaileddistributions 553 Listofelementaryprobabilitydensities 559 Frequentlyusedsymbols 561 Statisticaltables 568 Bibliography 587 Index 591