Table Of Content

Introduction to Probability and Statistical Inference This Page Intentionally Left Blank Introduction to Probability and Statistical Inference George Roussas UniversityofCalifornia,Davis Amsterdam Boston London NewYork Oxford Paris SanDiego SanFrancisco Singapore Sydney Tokyo SeniorSponsoringEditor BarbaraHolland ProjectManager NancyZachor EditorialCoordinator TomSinger CoverDesign ShawnGirsberger Copyeditor MaryPrescott Composition InternationalTypesettingandComposition Printer Maple-Vail Thisbookisprintedonacid-freepaper.(cid:2)∞ Copyright2003,ElsevierScience(USA) Allrightsreserved. Nopartofthispublicationmaybereproducedortransmittedinanyformorbyany means,electronicormechanical,includingphotocopy,recording,oranyinformation storageandretrievalsystem,withoutpermissioninwritingfromthepublisher. Requestsforpermissiontomakecopiesofanypartoftheworkshouldbemailedto: PermissionsDepartment,Harcourt,Inc.,6277SeaHarborDrive,Orlando, Florida32887-6777. AcademicPress AnimprintofElsevierScience 525BStreet,Suite1900,SanDiego,California92101-4495,USA http://www.academicpress.com AcademicPress AnimprintofElsevierScience 200WheelerRoad,Burlington,Massachusetts01803,USA http://www.academicpressbooks.com AcademicPress AnimprintofElsevierScience 84Theobald’sRoad,LondonWC1X8RR,UK http://www.academicpress.com LibraryofCongressControlNumber:2002110812 InternationalStandardBookNumber:0-12-599020-0 PRINTEDINTHEUNITEDSTATESOFAMERICA 02 03 04 05 06 9 8 7 6 5 4 3 2 1 Tomywifeandsons, andtheunforgettableBeowulf This Page Intentionally Left Blank Contents Preface xi 1 SOMEMOTIVATINGEXAMPLESANDSOME FUNDAMENTALCONCEPTS 1 1.1 SomeMotivatingExamples 1 1.2 SomeFundamentalConcepts 8 1.3 RandomVariables 19 2 THECONCEPTOFPROBABILITYANDBASICRESULTS 23 2.1 DefinitionofProbabilityandSomeBasicResults 24 2.2 DistributionofaRandomVariable 33 2.3 ConditionalProbabilityandRelatedResults 41 2.4 IndependentEventsandRelatedResults 51 2.5 BasicConceptsandResultsinCounting 59 3 NUMERICALCHARACTERISTICSOFARANDOM VARIABLE,SOMESPECIALRANDOMVARIABLES 68 3.1 Expectation,Variance,andMomentGeneratingFunction ofaRandomVariable 68 3.2 SomeProbabilityInequalities 77 3.3 SomeSpecialRandomVariables 79 3.4 MedianandModeofaRandomVariable 102 4 JOINTANDCONDITIONALP.D.F.’S,CONDITIONAL EXPECTATIONANDVARIANCE,MOMENT GENERATINGFUNCTION,COVARIANCE, ANDCORRELATIONCOEFFICIENT 109 4.1 Jointd.f.andJointp.d.f.ofTwoRandomVariables 110 4.2 MarginalandConditionalp.d.f.’s,Conditional ExpectationandVariance 117 4.3 ExpectationofaFunctionofTwor.v.’s,Joint andMarginalm.g.f.’s,Covariance,andCorrelation Coefficient 126 4.4 SomeGeneralizationstokRandomVariables 137 4.5 TheMultinomial,theBivariateNormal,andthe MultivariateNormalDistributions 139 vii viii Contents 5 INDEPENDENCEOFRANDOMVARIABLES ANDSOMEAPPLICATIONS 150 5.1 IndependenceofRandomVariablesandCriteria ofIndependence 150 5.2 TheReproductivePropertyofCertainDistributions 159 6 TRANSFORMATIONOFRANDOMVARIABLES 168 6.1 TransformingaSingleRandomVariable 168 6.2 TransformingTwoorMoreRandomVariables 173 6.3 LinearTransformations 185 6.4 TheProbabilityIntegralTransform 192 6.5 OrderStatistics 193 7 SOMEMODESOFCONVERGENCE OFRANDOMVARIABLES,APPLICATIONS 202 7.1 ConvergenceinDistributionorinProbabilityandTheir Relationship 202 7.2 SomeApplicationsofConvergenceinDistribution: TheWeakLawofLargeNumbersandtheCentral LimitTheorem 208 7.3 FurtherLimitTheorems 222 8 ANOVERVIEWOFSTATISTICALINFERENCE 227 8.1 TheBasicsofPointEstimation 228 8.2 TheBasicsofIntervalEstimation 230 8.3 TheBasicsofTestingHypotheses 231 8.4 TheBasicsofRegressionAnalysis 235 8.5 TheBasicsofAnalysisofVariance 236 8.6 TheBasicsofNonparametricInference 238 9 POINTESTIMATION 240 9.1 MaximumLikelihoodEstimation:Motivation andExamples 240 9.2 SomePropertiesofMaximumLikelihoodEstimates 253 9.3 UniformlyMinimumVarianceUnbiasedEstimates 261 9.4 Decision-TheoreticApproachtoEstimation 270 9.5 OtherMethodsofEstimation 277 10 CONFIDENCEINTERVALSANDCONFIDENCE REGIONS 281 10.1 ConfidenceIntervals 282 10.2 ConfidenceIntervalsinthePresenceofNuisance Parameters 289 Contents ix 10.3 AConfidenceRegionfor(μ,σ2)inthe N(μ,σ2) Distribution 292 10.4 ConfidenceIntervalswithApproximateConfidence Coefficient 294 11 TESTINGHYPOTHESES 299 11.1 GeneralConcepts,FormulationofSomeTesting Hypotheses 300 11.2 Neyman–PearsonFundamentalLemma,ExponentialType Families,UniformlyMostPowerfulTestsforSome CompositeHypotheses 302 11.3 SomeApplicationsofTheorems2and3 315 11.4 LikelihoodRatioTests 324 12 MOREABOUTTESTINGHYPOTHESES 343 12.1 LikelihoodRatioTestsintheMultinomialCase andContingencyTables 343 12.2 AGoodness-of-FitTest 349 12.3 Decision-TheoreticApproachtoTestingHypotheses 353 12.4 RelationshipBetweenTestingHypothesesand ConfidenceRegions 360 13 ASIMPLELINEARREGRESSIONMODEL 363 13.1 Setting-uptheModel—ThePrincipleofLeastSquares 364 13.2 TheLeastSquaresEstimatesofβ andβ ,andSome 1 2 ofTheirProperties 366 13.3 NormallyDistributedErrors:MLE’sofβ ,β ,andσ2, 1 2 SomeDistributionalResults 374 13.4 ConfidenceIntervalsandHypothesesTestingProblems 383 13.5 SomePredictionProblems 389 13.6 ProofofTheorem5 393 13.7 ConcludingRemarks 395 14 TWOMODELSOFANALYSISOFVARIANCE 397 14.1 One-WayLayoutwiththeSameNumberofObservations perCell 398 14.2 AMulticomparisonMethod 407 14.3 Two-WayLayoutwithOneObservationperCell 412 15 SOMETOPICSINNONPARAMETRICINFERENCE 428 15.1 SomeConfidenceIntervalswithGivenApproximate ConfidenceCoefficient 429 15.2 ConfidenceIntervalsforQuantilesofaDistribution Function 431