Introduction to Mathematical Statistics

705 Pages·2012·6.678 MB·English

by  Robert V. Hogg, Joeseph McKean, Allen T Craig

Introduction
to
Mathematical Statistics
Seventh Edition
Robert V. Hogg
University of Iowa
Joseph W. McKean
Western Michigan University
Allen T. Craig
Late Professor of Statistics
University of Iowa
Boston Columbus Indianapolis New York San Francisco Upper Saddle River
Amsterdam Cape Town Dubai London Madrid Milan Munich Paris Montreal Toronto
Delhi Mexico City Sao Paulo Sydney Hong Kong Seoul Singapore Taipei Tokyo
Editor in Chief: Deirdre Lynch
Acquisitions Editor: Christopher Cummings
Sponsoring Editor: Christina Lepre
Editorial Assistant: Sonia Ashraf
Senior Managing Editor: Karen Wernholm
Senior Production Project Manager: Beth Houston
Digital Assets Manager: Marianne Groth
Marketing Manager: Erin K. Lane
Marketing Coordinator: Kathleen DeChavez
Senior Author Support/Technology Specialist: Joe Vetere
Rights and Permissions Advisor: Michael Joyce
Manufacturing Buyer: Debbie Rossi
Cover Image: Fotolia: Yurok Aleksandrovich
Creative Director: Jayne Conte
Designer: Suzanne Behnke
Many of the designations used by manufacturers and sellers to distinguish their
productsareclaimedastrademarks. Wherethosedesignationsappearinthisbook,
andPearsonEducationwasawareofatrademarkclaim,thedesignationshavebeen
printed in initial caps or all caps.
Library of Congress Cataloging-in-Publications Data
Hogg, Robert V.
Introduction to mathematical statistics / Robert V. Hogg, Joseph W.
McKean, Allen T. Craig. – 7th ed.
p. cm.
ISBN 978-0-321-79543-4
1. Mathematical statistics. I. McKean, Joseph W., 1944- II. Craig,
Allen T. (Allen Thorton), 1905- III. Title.
QA276.H59 2013
519.5–dc23
2011034906
Copyright 2013, 2005, 1995 PearsonEducation, Inc.
All rights reserved. No part of this publication may be reproduced, stored in a re-
trievalsystem,ortransmitted,inanyformorbyanymeans,electronic,mechanical,
photocopying, recording, or otherwise, without the prior written permission of the
publisher. Printed in the United States of America . For information on obtain-
ing permission for use of material in this work, please submit a written request to
Pearson Education, Inc., Rights and Contracts Department, 501 Boylston Street,
Suite 900 , Boston , MA 02116 , fax your request to 617-671-3447, or e-mail at
http://www.pearsoned.com/legal/permissions.htm.
1 2 3 4 5 6 7 8 9 10CRS15 14 13 12 11
www.pearsonhighered.com
ISBN 13: 978-0-321-79543-4
ISBN 10: 0-321-79543-1
To Ann and to Marge
This page intentionally left blank
Contents
Preface ix
1 Probability and Distributions 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 The Probability Set Function . . . . . . . . . . . . . . . . . . . . . . 10
1.4 Conditional Probability and Independence . . . . . . . . . . . . . . . 21
1.5 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.6 Discrete Random Variables . . . . . . . . . . . . . . . . . . . . . . . 40
1.6.1 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . 42
1.7 Continuous Random Variables. . . . . . . . . . . . . . . . . . . . . . 44
1.7.1 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . 46
1.8 Expectation of a Random Variable . . . . . . . . . . . . . . . . . . . 52
1.9 Some Special Expectations . . . . . . . . . . . . . . . . . . . . . . . 57
1.10 Important Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2 Multivariate Distributions 73
2.1 Distributions of Two Random Variables . . . . . . . . . . . . . . . . 73
2.1.1 Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
2.2 Transformations: Bivariate Random Variables . . . . . . . . . . . . . 84
2.3 Conditional Distributions and Expectations . . . . . . . . . . . . . . 94
2.4 The Correlation Coefficient . . . . . . . . . . . . . . . . . . . . . . . 102
2.5 Independent Random Variables . . . . . . . . . . . . . . . . . . . . . 110
2.6 Extension to Several Random Variables . . . . . . . . . . . . . . . . 117
2.6.1 ∗Multivariate Variance-CovarianceMatrix . . . . . . . . . . . 123
2.7 Transformations for Several Random Variables . . . . . . . . . . . . 126
2.8 Linear Combinations of Random Variables . . . . . . . . . . . . . . . 134
3 Some Special Distributions 139
3.1 The Binomial and Related Distributions . . . . . . . . . . . . . . . . 139
3.2 The Poisson Distribution . . . . . . . . . . . . . . . . . . . . . . . . 150
3.3 The Γ, χ2, and β Distributions . . . . . . . . . . . . . . . . . . . . . 156
3.4 The Normal Distribution. . . . . . . . . . . . . . . . . . . . . . . . . 168
3.4.1 Contaminated Normals . . . . . . . . . . . . . . . . . . . . . 174
v
vi Contents
3.5 The Multivariate Normal Distribution . . . . . . . . . . . . . . . . . 178
3.5.1 ∗Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
3.6 t- and F-Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . 189
3.6.1 The t-distribution . . . . . . . . . . . . . . . . . . . . . . . . 189
3.6.2 The F-distribution . . . . . . . . . . . . . . . . . . . . . . . . 191
3.6.3 Student’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . 193
3.7 Mixture Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . 197
4 Some Elementary Statistical Inferences 203
4.1 Sampling and Statistics . . . . . . . . . . . . . . . . . . . . . . . . . 203
4.1.1 Histogram Estimates of pmfs and pdfs . . . . . . . . . . . . . 207
4.2 Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
4.2.1 Confidence Intervals for Difference in Means. . . . . . . . . . 217
4.2.2 Confidence Interval for Difference in Proportions . . . . . . . 219
4.3 Confidence Intervals for Parameters of Discrete Distributions . . . . 223
4.4 Order Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
4.4.1 Quantiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
4.4.2 Confidence Intervals for Quantiles . . . . . . . . . . . . . . . 234
4.5 Introduction to Hypothesis Testing . . . . . . . . . . . . . . . . . . . 240
4.6 Additional Comments About Statistical Tests . . . . . . . . . . . . . 248
4.7 Chi-Square Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
4.8 The Method of Monte Carlo . . . . . . . . . . . . . . . . . . . . . . . 261
4.8.1 Accept–Reject Generation Algorithm. . . . . . . . . . . . . . 268
4.9 Bootstrap Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . 273
4.9.1 Percentile Bootstrap Confidence Intervals . . . . . . . . . . . 273
4.9.2 Bootstrap Testing Procedures . . . . . . . . . . . . . . . . . . 276
4.10 ∗Tolerance Limits for Distributions . . . . . . . . . . . . . . . . . . . 284
5 Consistency and Limiting Distributions 289
5.1 Convergence in Probability . . . . . . . . . . . . . . . . . . . . . . . 289
5.2 Convergence in Distribution . . . . . . . . . . . . . . . . . . . . . . . 294
5.2.1 Bounded in Probability . . . . . . . . . . . . . . . . . . . . . 300
5.2.2 Δ-Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
5.2.3 Moment Generating Function Technique . . . . . . . . . . . . 303
5.3 Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 307
5.4 ∗Extensions to Multivariate Distributions . . . . . . . . . . . . . . . 314
6 Maximum Likelihood Methods 321
6.1 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . 321
6.2 Rao–Cram´erLower Bound and Efficiency . . . . . . . . . . . . . . . 327
6.3 Maximum Likelihood Tests . . . . . . . . . . . . . . . . . . . . . . . 341
6.4 Multiparameter Case: Estimation . . . . . . . . . . . . . . . . . . . . 350
6.5 Multiparameter Case: Testing . . . . . . . . . . . . . . . . . . . . . . 359
6.6 The EM Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
Contents vii
7 Sufficiency 375
7.1 Measures of Quality of Estimators . . . . . . . . . . . . . . . . . . . 375
7.2 A Sufficient Statistic for a Parameter . . . . . . . . . . . . . . . . . . 381
7.3 Properties of a Sufficient Statistic . . . . . . . . . . . . . . . . . . . . 388
7.4 Completeness and Uniqueness . . . . . . . . . . . . . . . . . . . . . . 392
7.5 The Exponential Class of Distributions . . . . . . . . . . . . . . . . . 397
7.6 Functions of a Parameter . . . . . . . . . . . . . . . . . . . . . . . . 402
7.7 The Case of Several Parameters . . . . . . . . . . . . . . . . . . . . . 407
7.8 Minimal Sufficiency and Ancillary Statistics . . . . . . . . . . . . . . 415
7.9 Sufficiency, Completeness, and Independence . . . . . . . . . . . . . 421
8 Optimal Tests of Hypotheses 429
8.1 Most Powerful Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
8.2 Uniformly Most Powerful Tests . . . . . . . . . . . . . . . . . . . . . 439
8.3 Likelihood Ratio Tests . . . . . . . . . . . . . . . . . . . . . . . . . . 447
8.4 The Sequential Probability Ratio Test . . . . . . . . . . . . . . . . . 459
8.5 Minimax and Classification Procedures . . . . . . . . . . . . . . . . . 466
8.5.1 Minimax Procedures . . . . . . . . . . . . . . . . . . . . . . . 466
8.5.2 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . 469
9 Inferences About Normal Models 473
9.1 Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
9.2 One-Way ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478
9.3 Noncentral χ2 and F-Distributions . . . . . . . . . . . . . . . . . . . 484
9.4 Multiple Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . 486
9.5 The Analysis of Variance . . . . . . . . . . . . . . . . . . . . . . . . 490
9.6 A Regression Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 497
9.7 A Test of Independence . . . . . . . . . . . . . . . . . . . . . . . . . 506
9.8 The Distributions of Certain Quadratic Forms. . . . . . . . . . . . . 509
9.9 The Independence of Certain Quadratic Forms . . . . . . . . . . . . 516
10 Nonparametric and Robust Statistics 525
10.1 Location Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525
10.2 Sample Median and the Sign Test. . . . . . . . . . . . . . . . . . . . 528
10.2.1 Asymptotic Relative Efficiency . . . . . . . . . . . . . . . . . 533
10.2.2 Estimating Equations Based on the Sign Test . . . . . . . . . 538
10.2.3 Confidence Interval for the Median . . . . . . . . . . . . . . . 539
10.3 Signed-Rank Wilcoxon . . . . . . . . . . . . . . . . . . . . . . . . . . 541
10.3.1 Asymptotic Relative Efficiency . . . . . . . . . . . . . . . . . 546
10.3.2 Estimating Equations Based on Signed-Rank Wilcoxon . . . 549
10.3.3 Confidence Interval for the Median . . . . . . . . . . . . . . . 549
10.4 Mann–Whitney–Wilcoxon Procedure . . . . . . . . . . . . . . . . . . 551
10.4.1 Asymptotic Relative Efficiency . . . . . . . . . . . . . . . . . 555
10.4.2 Estimating Equations Based on the Mann–Whitney–Wilcoxon 556
10.4.3 Confidence Interval for the Shift Parameter Δ. . . . . . . . . 557
10.5 General Rank Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . 559
viii Contents
10.5.1 Efficacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562
10.5.2 Estimating Equations Based on General Scores . . . . . . . . 563
10.5.3 Optimization: Best Estimates . . . . . . . . . . . . . . . . . . 564
10.6 Adaptive Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
10.7 Simple Linear Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 576
10.8 Measures of Association . . . . . . . . . . . . . . . . . . . . . . . . . 581
10.8.1 Kendall’s τ . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582
10.8.2 Spearman’s Rho . . . . . . . . . . . . . . . . . . . . . . . . . 584
10.9 Robust Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588
10.9.1 Location Model . . . . . . . . . . . . . . . . . . . . . . . . . . 589
10.9.2 Linear Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 595
11 Bayesian Statistics 605
11.1 Subjective Probability . . . . . . . . . . . . . . . . . . . . . . . . . . 605
11.2 Bayesian Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . 608
11.2.1 Prior and Posterior Distributions . . . . . . . . . . . . . . . . 609
11.2.2 Bayesian Point Estimation. . . . . . . . . . . . . . . . . . . . 612
11.2.3 Bayesian Interval Estimation . . . . . . . . . . . . . . . . . . 615
11.2.4 Bayesian Testing Procedures . . . . . . . . . . . . . . . . . . 616
11.2.5 Bayesian Sequential Procedures . . . . . . . . . . . . . . . . . 617
11.3 More Bayesian Terminology and Ideas . . . . . . . . . . . . . . . . . 619
11.4 Gibbs Sampler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626
11.5 Modern BayesianMethods . . . . . . . . . . . . . . . . . . . . . . . . 632
11.5.1 Empirical Bayes . . . . . . . . . . . . . . . . . . . . . . . . . 636
A Mathematical Comments 641
A.1 Regularity Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . 641
A.2 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 642
B R Functions 645
C Tables of Distributions 655
D Lists of Common Distributions 665
E References 669
F Answers to Selected Exercises 673
Index 683
Preface
Changes for Seventh Edition
In the preparation of this seventh edition, our goal has remained steadfast: to
produceanoutstandingtextinmathematicalstatistics. Inthisnewedition,wehave
addedexamplesandexercisestohelpclarifytheexposition. Forthesamereason,we
havemovedsomematerialforward. Forexample,we movedthe discussiononsome
properties of linear combinations of random variables from Chapter 4 to Chapter
2. This helps in the discussion of statistical properties in Chapter 3 as well as in
the new Chapter 4.
One of the major changes was moving the chapter on “Some Elementary Sta-
tistical Inferences,” from Chapter 5 to Chapter 4. This chapter on inference covers
confidence intervals and statistical tests of hypotheses, two of the most important
conceptsinstatisticalinference. We beginChapter4withadiscussionofarandom
sample and point estimation. We introduce point estimation via a brief discussion
of maximum likelihood estimation (the theory of maximum likelihood inference is
still fullly discussed in Chapter 6). In Chapter 4, though, the discussion is illus-
trated with examples. After discussing point estimation in Chapter 4, we proceed
onto confidence intervals and hypotheses testing. Inference for the basic one- and
two-sample problems (large and small samples) is presented. We illustrate this
discussion with plenty of examples, several of which are concerned with real data.
We have also added exercises dealing with real data. The discussion has also been
updated; for example, exact confidence intervals for the parameters of discrete dis-
tributions andbootstrapconfidenceintervals andtests ofhypotheses arediscussed,
both of which are being used more and more in practice. These changes enable
a one-semester course to cover basic statistical theory with applications. Such a
course would cover Chapters 1–4 and, depending on time, parts of Chapter 5. For
two-semester courses, this basic understanding of statistical inference will prove
quite helpful to students in the later chapters (6–8) on the statistical theory of
inference.
Another major change is moving the discussion of robustness concepts (influ-
ence function and breakdown) of Chapter 12 to the end of Chapter 10. To reflect
thismove,the titleofChapter10hasbeenchangedto“NonparametricandRobust
Statistics.” This additional material in the new Chapter 10 is essentially the im-
portantrobustnessconceptsfoundintheoldChapter12. Further,thesimplelinear
modelisdiscussedinChapters9and10. Hence,withthis movewehaveeliminated
ix