An introduction to probability and statistical inference

543 Pages·2003·2.301 MB·English

by  George G. Roussas

Introduction to Probability
and Statistical Inference
This Page Intentionally Left Blank
Introduction to Probability
and Statistical Inference
George Roussas
UniversityofCalifornia,Davis
Amsterdam Boston London NewYork Oxford Paris
SanDiego SanFrancisco Singapore Sydney Tokyo
SeniorSponsoringEditor BarbaraHolland
ProjectManager NancyZachor
EditorialCoordinator TomSinger
CoverDesign ShawnGirsberger
Copyeditor MaryPrescott
Composition InternationalTypesettingandComposition
Printer Maple-Vail
Thisbookisprintedonacid-freepaper.(cid:2)∞
Copyright2003,ElsevierScience(USA)
Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyany
means,electronicormechanical,includingphotocopy,recording,oranyinformation
storageandretrievalsystem,withoutpermissioninwritingfromthepublisher.
Requestsforpermissiontomakecopiesofanypartoftheworkshouldbemailedto:
PermissionsDepartment,Harcourt,Inc.,6277SeaHarborDrive,Orlando,
Florida32887-6777.
AcademicPress
AnimprintofElsevierScience
525BStreet,Suite1900,SanDiego,California92101-4495,USA
http://www.academicpress.com
AcademicPress
AnimprintofElsevierScience
200WheelerRoad,Burlington,Massachusetts01803,USA
http://www.academicpressbooks.com
AcademicPress
AnimprintofElsevierScience
84Theobald’sRoad,LondonWC1X8RR,UK
http://www.academicpress.com
LibraryofCongressControlNumber:2002110812
InternationalStandardBookNumber:0-12-599020-0
PRINTEDINTHEUNITEDSTATESOFAMERICA
02 03 04 05 06 9 8 7 6 5 4 3 2 1
Tomywifeandsons,
andtheunforgettableBeowulf
This Page Intentionally Left Blank
Contents
Preface xi
1 SOMEMOTIVATINGEXAMPLESANDSOME
FUNDAMENTALCONCEPTS 1
1.1 SomeMotivatingExamples 1
1.2 SomeFundamentalConcepts 8
1.3 RandomVariables 19
2 THECONCEPTOFPROBABILITYANDBASICRESULTS 23
2.1 DefinitionofProbabilityandSomeBasicResults 24
2.2 DistributionofaRandomVariable 33
2.3 ConditionalProbabilityandRelatedResults 41
2.4 IndependentEventsandRelatedResults 51
2.5 BasicConceptsandResultsinCounting 59
3 NUMERICALCHARACTERISTICSOFARANDOM
VARIABLE,SOMESPECIALRANDOMVARIABLES 68
3.1 Expectation,Variance,andMomentGeneratingFunction
ofaRandomVariable 68
3.2 SomeProbabilityInequalities 77
3.3 SomeSpecialRandomVariables 79
3.4 MedianandModeofaRandomVariable 102
4 JOINTANDCONDITIONALP.D.F.’S,CONDITIONAL
EXPECTATIONANDVARIANCE,MOMENT
GENERATINGFUNCTION,COVARIANCE,
ANDCORRELATIONCOEFFICIENT 109
4.1 Jointd.f.andJointp.d.f.ofTwoRandomVariables 110
4.2 MarginalandConditionalp.d.f.’s,Conditional
ExpectationandVariance 117
4.3 ExpectationofaFunctionofTwor.v.’s,Joint
andMarginalm.g.f.’s,Covariance,andCorrelation
Coefficient 126
4.4 SomeGeneralizationstokRandomVariables 137
4.5 TheMultinomial,theBivariateNormal,andthe
MultivariateNormalDistributions 139
vii
viii Contents
5 INDEPENDENCEOFRANDOMVARIABLES
ANDSOMEAPPLICATIONS 150
5.1 IndependenceofRandomVariablesandCriteria
ofIndependence 150
5.2 TheReproductivePropertyofCertainDistributions 159
6 TRANSFORMATIONOFRANDOMVARIABLES 168
6.1 TransformingaSingleRandomVariable 168
6.2 TransformingTwoorMoreRandomVariables 173
6.3 LinearTransformations 185
6.4 TheProbabilityIntegralTransform 192
6.5 OrderStatistics 193
7 SOMEMODESOFCONVERGENCE
OFRANDOMVARIABLES,APPLICATIONS 202
7.1 ConvergenceinDistributionorinProbabilityandTheir
Relationship 202
7.2 SomeApplicationsofConvergenceinDistribution:
TheWeakLawofLargeNumbersandtheCentral
LimitTheorem 208
7.3 FurtherLimitTheorems 222
8 ANOVERVIEWOFSTATISTICALINFERENCE 227
8.1 TheBasicsofPointEstimation 228
8.2 TheBasicsofIntervalEstimation 230
8.3 TheBasicsofTestingHypotheses 231
8.4 TheBasicsofRegressionAnalysis 235
8.5 TheBasicsofAnalysisofVariance 236
8.6 TheBasicsofNonparametricInference 238
9 POINTESTIMATION 240
9.1 MaximumLikelihoodEstimation:Motivation
andExamples 240
9.2 SomePropertiesofMaximumLikelihoodEstimates 253
9.3 UniformlyMinimumVarianceUnbiasedEstimates 261
9.4 Decision-TheoreticApproachtoEstimation 270
9.5 OtherMethodsofEstimation 277
10 CONFIDENCEINTERVALSANDCONFIDENCE
REGIONS 281
10.1 ConfidenceIntervals 282
10.2 ConfidenceIntervalsinthePresenceofNuisance
Parameters 289
Contents ix
10.3 AConfidenceRegionfor(μ,σ2)inthe N(μ,σ2)
Distribution 292
10.4 ConfidenceIntervalswithApproximateConfidence
Coefficient 294
11 TESTINGHYPOTHESES 299
11.1 GeneralConcepts,FormulationofSomeTesting
Hypotheses 300
11.2 Neyman–PearsonFundamentalLemma,ExponentialType
Families,UniformlyMostPowerfulTestsforSome
CompositeHypotheses 302
11.3 SomeApplicationsofTheorems2and3 315
11.4 LikelihoodRatioTests 324
12 MOREABOUTTESTINGHYPOTHESES 343
12.1 LikelihoodRatioTestsintheMultinomialCase
andContingencyTables 343
12.2 AGoodness-of-FitTest 349
12.3 Decision-TheoreticApproachtoTestingHypotheses 353
12.4 RelationshipBetweenTestingHypothesesand
ConfidenceRegions 360
13 ASIMPLELINEARREGRESSIONMODEL 363
13.1 Setting-uptheModel—ThePrincipleofLeastSquares 364
13.2 TheLeastSquaresEstimatesofβ andβ ,andSome
1 2
ofTheirProperties 366
13.3 NormallyDistributedErrors:MLE’sofβ ,β ,andσ2,
1 2
SomeDistributionalResults 374
13.4 ConfidenceIntervalsandHypothesesTestingProblems 383
13.5 SomePredictionProblems 389
13.6 ProofofTheorem5 393
13.7 ConcludingRemarks 395
14 TWOMODELSOFANALYSISOFVARIANCE 397
14.1 One-WayLayoutwiththeSameNumberofObservations
perCell 398
14.2 AMulticomparisonMethod 407
14.3 Two-WayLayoutwithOneObservationperCell 412
15 SOMETOPICSINNONPARAMETRICINFERENCE 428
15.1 SomeConfidenceIntervalswithGivenApproximate
ConfidenceCoefficient 429
15.2 ConfidenceIntervalsforQuantilesofaDistribution
Function 431