Introduction to Mathematical Statistics

762 Pages·2019·5.74 MB·english

by  Robert V. Hogg

Introduction
to
Mathematical Statistics
Eighth Edition
Global Edition
Robert V. Hogg
University of Iowa
Joseph W. McKean
Western Michigan University
Allen T. Craig
Late Professor of Statistics
University of Iowa
Director,PortfolioManagement: DeirdreLynch ManagingEditor,DigitalMedia,Global
CoursewarePortfolioManager: PatrickBarbera Edition: GargiBanerjee
PortfolioManagementAssistant: Morgan Manager,RightsandPermissions: Gina
Danna Cheselka
AssociateEditor,GlobalEdition: Shaoni ManufacturingBuyer: CarolMelville,
Mukherjee LSCCommunications
ContentProducer: LaurenMorse ManufacturingBuyer,GlobalEdition:
ManagingProducer: ScottDisanno KayHolman
ProductMarketingManager: YvonneVannatta ArtDirector: BarbaraAtkinson
FieldMarketingManager: EvanSt. Cyr ProductionCoordinationandIllustrations:
MarketingAssistant: JonBryant Integra
SeniorAuthorSupport/TechnologySpecialist: CoverDesign: LuminaDatamatics
JoeVetere CoverImage: AnnaMente/Shutterstock
PearsonEducationLimited
KAOTwo
KAOPark
HockhamWay
Harlow
Essex
CM179SR
UnitedKingdom
andAssociatedCompaniesthroughouttheworld
Visit us on the World Wide Web at: www.pearsonglobaleditions.com
©PearsonEducationLimited2020
The rights of Robert V. Hogg, Joseph W. McKean, and Allen T. Craig to be identified as the
authors of this work have been asserted by them in accordance with the Copyright, Designs and
PatentsAct1988.
Authorized adaptation from the United States edition, entitled Introduction to Mathematical
Statistics, 8th Edition, ISBN 978-0-13-468699-8, by Robert V. Hogg, Joseph W. McKean, and
Allen T. Craig, published by Pearson Education © 2019.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system,
or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or
otherwise, without either the prior written permission of the publisher or a license permitting
restricted copying in the United Kingdom issued by the Copyright Licensing Agency Ltd, Saf-
fronHouse,6–10KirbyStreet,LondonEC1N8TS.Forinformationregardingpermissions,request
forms and the appropriate contacts within the Pearson Education Global Rights & Permissions
department,pleasevisitwww.pearsoned.com/permissions/.
Alltrademarksusedhereinarethepropertyoftheirrespectiveowners. Theuseofanytrademark
inthistextdoesnotvestintheauthororpublisheranytrademarkownershiprightsinsuchtrade-
marks,nordoestheuseofsuchtrademarksimplyanyaffiliationwithorendorsementofthisbook
bysuchowners.
PEARSON and ALWAYS LEARNING are exclusive trademarks owned by Pearson Education,
Inc. or its affiliates in the U.S. and/or other countries. Unless otherwise indicated herein, any
third-partytrademarksthatmayappearinthisworkarethepropertyoftheirrespectiveowners
andanyreferencestothird-partytrademarks,logosorothertradedressarefordemonstrativeor
descriptive purposes only. Such references are not intended to imply any sponsorship, endorse-
ment, authorization, or promotion of Pearson’s products by the owners of such marks, or any
relationshipbetweentheownerandPearsonEducation,Inc. oritsaffiliates,authors,licenseesor
distributors.
This eBook is a standalone product and may or may not include all assets that were part of the
print version. It also does not provide access to other Pearson digital products like MyLab and
Mastering. ThepublisherreservestherighttoremoveanymaterialinthiseBookatanytime.
ISBN10: 1-292-26476-4
ISBN13: 978-1-292-26476-9
eBookISBN13: 978-1-292-26478-3
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
TypesetbyIntegraSoftwareServicesPvt. Ltd.
Dedicated to my wife Marge
and to the memory of Bob Hogg
This page intentionally left blank
Contents
Preface 11
1 Probability and Distributions 15
1.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.2 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2.1 Review of Set Theory . . . . . . . . . . . . . . . . . . . . . . 18
1.2.2 Set Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.3 The Probability Set Function . . . . . . . . . . . . . . . . . . . . . . 26
1.3.1 Counting Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.3.2 Additional Properties of Probability . . . . . . . . . . . . . . 32
1.4 Conditional Probability and Independence . . . . . . . . . . . . . . . 37
1.4.1 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
1.4.2 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
1.5 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
1.6 Discrete Random Variables . . . . . . . . . . . . . . . . . . . . . . . 59
1.6.1 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . 61
1.7 Continuous Random Variables . . . . . . . . . . . . . . . . . . . . . 63
1.7.1 Quantiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
1.7.2 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . 67
1.7.3 Mixtures of Discrete and Continuous Type Distributions . . . 70
1.8 Expectation of a Random Variable . . . . . . . . . . . . . . . . . . . 74
1.8.1 R Computation for an Estimation of the Expected Gain . . . 79
1.9 Some Special Expectations . . . . . . . . . . . . . . . . . . . . . . . 82
1.10 Important Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . 92
2 Multivariate Distributions 99
2.1 Distributions of Two Random Variables . . . . . . . . . . . . . . . . 99
2.1.1 Marginal Distributions . . . . . . . . . . . . . . . . . . . . . . 103
2.1.2 Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
2.2 Transformations: Bivariate Random Variables . . . . . . . . . . . . . 114
2.3 Conditional Distributions and Expectations . . . . . . . . . . . . . . 123
2.4 Independent Random Variables . . . . . . . . . . . . . . . . . . . . . 131
2.5 The Correlation Coefficient . . . . . . . . . . . . . . . . . . . . . . . 139
2.6 Extension to Several Random Variables . . . . . . . . . . . . . . . . 148
5
6 Contents
2.6.1 ∗Multivariate Variance-Covariance Matrix . . . . . . . . . . . 154
2.7 Transformations for Several Random Variables . . . . . . . . . . . . 157
2.8 Linear Combinations of Random Variables . . . . . . . . . . . . . . . 165
3 Some Special Distributions 169
3.1 The Binomial and Related Distributions . . . . . . . . . . . . . . . . 169
3.1.1 Negative Binomial and Geometric Distributions . . . . . . . . 173
3.1.2 Multinomial Distribution . . . . . . . . . . . . . . . . . . . . 174
3.1.3 Hypergeometric Distribution . . . . . . . . . . . . . . . . . . 176
3.2 The Poisson Distribution . . . . . . . . . . . . . . . . . . . . . . . . 181
3.3 The Γ, χ2, and β Distributions . . . . . . . . . . . . . . . . . . . . . 187
3.3.1 The χ2-Distribution . . . . . . . . . . . . . . . . . . . . . . . 192
3.3.2 The β-Distribution . . . . . . . . . . . . . . . . . . . . . . . . 194
3.4 The Normal Distribution. . . . . . . . . . . . . . . . . . . . . . . . . 200
3.4.1 ∗Contaminated Normals . . . . . . . . . . . . . . . . . . . . . 207
3.5 The Multivariate Normal Distribution . . . . . . . . . . . . . . . . . 212
3.5.1 Bivariate Normal Distribution. . . . . . . . . . . . . . . . . . 212
3.5.2 ∗Multivariate Normal Distribution, General Case . . . . . . . 213
3.5.3 ∗Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
3.6 t- and F-Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . 224
3.6.1 The t-distribution . . . . . . . . . . . . . . . . . . . . . . . . 224
3.6.2 The F-distribution . . . . . . . . . . . . . . . . . . . . . . . . 226
3.6.3 Student’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . 228
3.7 ∗Mixture Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . 232
4 Some Elementary Statistical Inferences 239
4.1 Sampling and Statistics . . . . . . . . . . . . . . . . . . . . . . . . . 239
4.1.1 Point Estimators . . . . . . . . . . . . . . . . . . . . . . . . . 240
4.1.2 Histogram Estimates of pmfs and pdfs . . . . . . . . . . . . . 244
4.2 Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
4.2.1 Confidence Intervals for Difference in Means. . . . . . . . . . 255
4.2.2 Confidence Interval for Difference in Proportions . . . . . . . 257
4.3 ∗Confidence Intervals for Parameters of Discrete Distributions . . . . 262
4.4 Order Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
4.4.1 Quantiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
4.4.2 Confidence Intervals for Quantiles . . . . . . . . . . . . . . . 275
4.5 Introduction to Hypothesis Testing . . . . . . . . . . . . . . . . . . . 281
4.6 Additional Comments About Statistical Tests . . . . . . . . . . . . . 289
4.6.1 Observed Significance Level, p-value . . . . . . . . . . . . . . 293
4.7 Chi-Square Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
4.8 The Method of Monte Carlo . . . . . . . . . . . . . . . . . . . . . . . 306
4.8.1 Accept–Reject Generation Algorithm. . . . . . . . . . . . . . 312
4.9 Bootstrap Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . 317
4.9.1 Percentile Bootstrap Confidence Intervals . . . . . . . . . . . 317
4.9.2 Bootstrap Testing Procedures . . . . . . . . . . . . . . . . . . 322
4.10 ∗Tolerance Limits for Distributions . . . . . . . . . . . . . . . . . . . 329
Contents 7
5 Consistency and Limiting Distributions 335
5.1 Convergence in Probability . . . . . . . . . . . . . . . . . . . . . . . 335
5.1.1 Sampling and Statistics . . . . . . . . . . . . . . . . . . . . . 338
5.2 Convergence in Distribution . . . . . . . . . . . . . . . . . . . . . . . 341
5.2.1 Bounded in Probability . . . . . . . . . . . . . . . . . . . . . 347
5.2.2 Δ-Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
5.2.3 Moment Generating Function Technique . . . . . . . . . . . . 350
5.3 Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 355
5.4 ∗Extensions to Multivariate Distributions . . . . . . . . . . . . . . . 362
6 Maximum Likelihood Methods 369
6.1 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . 369
6.2 Rao–Cram´er Lower Bound and Efficiency . . . . . . . . . . . . . . . 376
6.3 Maximum Likelihood Tests . . . . . . . . . . . . . . . . . . . . . . . 390
6.4 Multiparameter Case: Estimation . . . . . . . . . . . . . . . . . . . . 400
6.5 Multiparameter Case: Testing . . . . . . . . . . . . . . . . . . . . . . 409
6.6 The EM Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
7 Sufficiency 427
7.1 Measures of Quality of Estimators . . . . . . . . . . . . . . . . . . . 427
7.2 A Sufficient Statistic for a Parameter . . . . . . . . . . . . . . . . . . 433
7.3 Properties of a Sufficient Statistic . . . . . . . . . . . . . . . . . . . . 440
7.4 Completeness and Uniqueness . . . . . . . . . . . . . . . . . . . . . . 444
7.5 The Exponential Class of Distributions . . . . . . . . . . . . . . . . . 449
7.6 Functions of a Parameter . . . . . . . . . . . . . . . . . . . . . . . . 454
7.6.1 Bootstrap Standard Errors . . . . . . . . . . . . . . . . . . . 458
7.7 The Case of Several Parameters . . . . . . . . . . . . . . . . . . . . . 461
7.8 Minimal Sufficiency and Ancillary Statistics . . . . . . . . . . . . . . 468
7.9 Sufficiency, Completeness, and Independence . . . . . . . . . . . . . 475
8 Optimal Tests of Hypotheses 483
8.1 Most Powerful Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
8.2 Uniformly Most Powerful Tests . . . . . . . . . . . . . . . . . . . . . 493
8.3 Likelihood Ratio Tests . . . . . . . . . . . . . . . . . . . . . . . . . . 501
8.3.1 Likelihood Ratio Tests for Testing Means of Normal Distri-
butions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502
8.3.2 Likelihood Ratio Tests for Testing Variances of Normal Dis-
tributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509
8.4 ∗The Sequential Probability Ratio Test. . . . . . . . . . . . . . . . . 514
8.5 ∗Minimax and Classification Procedures . . . . . . . . . . . . . . . . 521
8.5.1 Minimax Procedures . . . . . . . . . . . . . . . . . . . . . . . 521
8.5.2 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
8 Contents
9 Inferences About Normal Linear Models 529
9.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529
9.2 One-Way ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 530
9.3 Noncentral χ2 and F-Distributions . . . . . . . . . . . . . . . . . . . 536
9.4 Multiple Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . 539
9.5 Two-Way ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545
9.5.1 Interaction between Factors . . . . . . . . . . . . . . . . . . . 548
9.6 A Regression Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 553
9.6.1 Maximum Likelihood Estimates. . . . . . . . . . . . . . . . . 554
9.6.2 ∗Geometry of the Least Squares Fit . . . . . . . . . . . . . . 560
9.7 A Test of Independence . . . . . . . . . . . . . . . . . . . . . . . . . 565
9.8 The Distributions of Certain Quadratic Forms. . . . . . . . . . . . . 569
9.9 The Independence of Certain Quadratic Forms . . . . . . . . . . . . 576
10 Nonparametric and Robust Statistics 583
10.1 Location Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583
10.2 Sample Median and the Sign Test. . . . . . . . . . . . . . . . . . . . 586
10.2.1 Asymptotic Relative Efficiency . . . . . . . . . . . . . . . . . 591
10.2.2 Estimating Equations Based on the Sign Test . . . . . . . . . 596
10.2.3 Confidence Interval for the Median . . . . . . . . . . . . . . . 598
10.3 Signed-Rank Wilcoxon . . . . . . . . . . . . . . . . . . . . . . . . . . 600
10.3.1 Asymptotic Relative Efficiency . . . . . . . . . . . . . . . . . 605
10.3.2 Estimating Equations Based on Signed-Rank Wilcoxon . . . 607
10.3.3 Confidence Interval for the Median . . . . . . . . . . . . . . . 608
10.3.4 Monte Carlo Investigation . . . . . . . . . . . . . . . . . . . . 609
10.4 Mann–Whitney–Wilcoxon Procedure . . . . . . . . . . . . . . . . . . 612
10.4.1 Asymptotic Relative Efficiency . . . . . . . . . . . . . . . . . 616
10.4.2 Estimating Equations Based on the Mann–Whitney–Wilcoxon 618
10.4.3 Confidence Interval for the Shift Parameter Δ. . . . . . . . . 618
10.4.4 Monte Carlo Investigation of Power . . . . . . . . . . . . . . 619
10.5 ∗General Rank Scores . . . . . . . . . . . . . . . . . . . . . . . . . . 621
10.5.1 Efficacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624
10.5.2 Estimating Equations Based on General Scores . . . . . . . . 626
10.5.3 Optimization: Best Estimates . . . . . . . . . . . . . . . . . . 626
10.6 ∗Adaptive Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . 633
10.7 Simple Linear Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 639
10.8 Measures of Association . . . . . . . . . . . . . . . . . . . . . . . . . 645
10.8.1 Kendall’s τ . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645
10.8.2 Spearman’s Rho . . . . . . . . . . . . . . . . . . . . . . . . . 648
10.9 Robust Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 652
10.9.1 Location Model . . . . . . . . . . . . . . . . . . . . . . . . . . 652
10.9.2 Linear Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 659
Contents 9
11 Bayesian Statistics 669
11.1 Bayesian Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . 669
11.1.1 Prior and Posterior Distributions . . . . . . . . . . . . . . . . 670
11.1.2 Bayesian Point Estimation . . . . . . . . . . . . . . . . . . . 672
11.1.3 Bayesian Interval Estimation . . . . . . . . . . . . . . . . . . 676
11.1.4 Bayesian Testing Procedures . . . . . . . . . . . . . . . . . . 677
11.1.5 Bayesian Sequential Procedures . . . . . . . . . . . . . . . . . 678
11.2 More Bayesian Terminology and Ideas . . . . . . . . . . . . . . . . . 680
11.3 Gibbs Sampler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686
11.4 Modern Bayesian Methods . . . . . . . . . . . . . . . . . . . . . . . . 693
11.4.1 Empirical Bayes . . . . . . . . . . . . . . . . . . . . . . . . . 696
A Mathematical Comments 701
A.1 Regularity Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 701
A.2 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 702
B R Primer 707
B.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 707
B.2 Probability Distributions. . . . . . . . . . . . . . . . . . . . . . . . . 710
B.3 R Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 712
B.4 Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 713
B.5 Input and Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . 714
B.6 Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 714
C Lists of Common Distributions 717
D Tables of Distributions 721
E References 729
F Answers to Selected Exercises 735
Index 747