Introduction to Mathematical Statistics

655 Pages·2013·5.276 MB·English

by  Hogg, Robert V;McKean, Joeseph;Craig, Allen T

I
n
t
r
o
d
u
c
t
i
o
n

t
o

M
a
t
h
e
m
a
t
i
c
a
l

S
t
a
t
i
s
t
i
c
s

H
o
g
g

e
t

a
l Introduction to Mathematical Statistics
.

Robert V. Hogg Joeseph McKean

7 Allen T. Craig
e
ISBN 978-1-29202-499-8
Seventh Edition
9 781292 024998
Introduction to Mathematical Statistics
Robert V. Hogg Joeseph McKean
Allen T. Craig
Seventh Edition
ISBN 10: 1-292-02499-2
ISBN 13: 978-1-292-02499-8
Pearson Education Limited
Edinburgh Gate
Harlow
Essex CM20 2JE
England and Associated Companies throughout the world
Visit us on the World Wide Web at: www.pearsoned.co.uk
© Pearson Education Limited 2014
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without either the
prior written permission of the publisher or a licence permitting restricted copying in the United Kingdom
issued by the Copyright Licensing Agency Ltd, Saffron House, 6–10 Kirby Street, London EC1N 8TS.
All trademarks used herein are the property of their respective owners. The use of any trademark
in this text does not vest in the author or publisher any trademark ownership rights in such
trademarks, nor does the use of such trademarks imply any affi liation with or endorsement of this
book by such owners.
ISBN 10: 1-292-02499-2
ISBN 13: 978-1-292-02499-8
British Library Cataloguing-in-Publication Data
A catalogue record for this book is available from the British Library
Printed in the United States of America
122334456667409283831231175739577115
P E A R S O N C U S T O M L I B R AR Y
Table of Contents
1. Probability and Distributions
Robert V. Hogg/Joeseph McKean/Allen T. Craig 1
2. Multivariate Distributions
Robert V. Hogg/Joeseph McKean/Allen T. Craig 75
3. Some Special Distributions
Robert V. Hogg/Joeseph McKean/Allen T. Craig 141
4. Some Elementary Statistical Inferences
Robert V. Hogg/Joeseph McKean/Allen T. Craig 207
5. Consistency and Limiting Distributions
Robert V. Hogg/Joeseph McKean/Allen T. Craig 295
6. Maximum Likelihood Methods
Robert V. Hogg/Joeseph McKean/Allen T. Craig 327
7. Sufficiency
Robert V. Hogg/Joeseph McKean/Allen T. Craig 383
8. Optimal Tests of Hypotheses
Robert V. Hogg/Joeseph McKean/Allen T. Craig 439
9. Inferences About Normal Models
Robert V. Hogg/Joeseph McKean/Allen T. Craig 485
10. Nonparametric and Robust Statistics
Robert V. Hogg/Joeseph McKean/Allen T. Craig 537
11. Appendix: Mathematical Comments
Robert V. Hogg/Joeseph McKean/Allen T. Craig 617
12. Appendix: R Functions
Robert V. Hogg/Joeseph McKean/Allen T. Craig 621
13. Appendix: Tables of Distributions
Robert V. Hogg/Joeseph McKean/Allen T. Craig 631
I
664415
14. Appendix: Lists of Common Distributions
Robert V. Hogg/Joeseph McKean/Allen T. Craig 641
Index 645
II
Probability and Distributions
1 Introduction
Manykindsofinvestigationsmaybecharacterizedinpartbythefactthatrepeated
experimentation, under essentially the same conditions, is more or less standard
procedure. For instance, in medical research, interest may center on the effect of
a drug that is to be administered; or an economist may be concerned with the
prices of three specified commodities at various time intervals; or the agronomist
may wish to study the effect that a chemical fertilizer has on the yield of a cereal
grain. The only way in which an investigator can elicit information about any such
phenomenon is to perform the experiment. Each experiment terminates with an
outcome. But it is characteristic of these experiments that the outcome cannot be
predicted with certainty prior to the performance of the experiment.
Supposethatwehavesuchanexperiment,buttheexperimentisofsuchanature
thatacollectionofeverypossibleoutcomecanbedescribedpriortoitsperformance.
If this kind of experiment can be repeated under the same conditions, it is called
a random experiment, and the collection of every possible outcome is called the
experimental space or the sample space.
Example 1.1. In the toss of a coin, let the outcome tails be denoted by T and let
theoutcomeheadsbedenotedbyH. Ifweassumethatthecoinmayberepeatedly
tossed under the same conditions, then the toss of this coin is an example of a
random experiment in which the outcome is one of the two symbols T and H; that
is, the sample space is the collection of these two symbols.
Example 1.2. In the cast of one red die and one white die, let the outcome be
the ordered pair (number of spots up on the red die, number of spots up on the
whitedie). Ifweassumethatthesetwodicemayberepeatedlycastunderthesame
conditions, then the cast of this pair of dice is a random experiment. The sample
spaceconsistsofthe36orderedpairs: (1,1),...,(1,6),(2,1),...,(2,6),...,(6,6).
LetC denoteasamplespace, letcdenoteanelementofC, andletC representa
collectionofelementsofC. If,upontheperformanceoftheexperiment,theoutcome
FromChapter1ofIntroductiontoMathematicalStatistics,SeventhEdition.RobertV.Hogg,
JosephW.McKean,AllenT.Craig. Copyright(cid:2)c 2013byPearsonEducation,Inc.
Allrightsreserved.
1
Probability and Distributions
is in C, we shall say that the event C has occurred. Now conceive of our having
made N repeated performances of the random experiment. Then we can count the
number f of times (the frequency) that theevent C actually occurredthroughout
the N performances. The ratio f/N is called the relative frequency of the event
C in these N experiments. A relative frequency is usually quite erratic for small
values of N, as you can discover by tossing a coin. But as N increases, experience
indicates that we associate with the event C a number, say p, that is equal or
approximately equal to that number about which the relative frequency seems to
stabilize. Ifwedothis,thenthenumberpcanbeinterpretedasthatnumberwhich,
infutureperformancesoftheexperiment, therelativefrequencyoftheeventC will
either equal or approximate. Thus, although we cannot predict the outcome of
a random experiment, we can, for a large value of N, predict approximately the
relative frequency with which the outcome will be in C. The number p associated
with the event C is given various names. Sometimes it is called the probability that
theoutcomeoftherandomexperimentisinC;sometimesitiscalledtheprobability
oftheeventC;andsometimesitiscalledtheprobability measure ofC. Thecontext
usually suggests an appropriate choice of terminology.
Example 1.3. Let C denote the sample space of Example 1.2 and let C be the
collection of every ordered pair of C for which the sum of the pair is equal to seven.
ThusC isthecollection(1,6),(2,5),(3,4),(4,3),(5,2), and(6,1). Supposethatthe
dice are cast N =400 times and let f, the frequency of a sum of seven, be f =60.
ThentherelativefrequencywithwhichtheoutcomewasinC isf/N = 60 =0.15.
400
Thus we might associate with C a number p that is close to 0.15, and p would be
called the probability of the event C.
Remark 1.1. The preceding interpretation of probability is sometimes referred to
as the relative frequency approach, and it obviously depends upon the fact that an
experiment can be repeated under essentially identical conditions. However, many
persons extend probability to other situations by treating it as a rational measure
ofbelief. Forexample,thestatementp= 2 wouldmeantothemthattheirpersonal
5
orsubjective probabilityoftheeventC isequalto 2. Hence,iftheyarenotopposed
5
to gambling, this could be interpreted as a willingness on their part to bet on the
outcome of C so that the two possible payoffs are in the ratio p/(1−p)= 2/3 = 2.
5 5 3
Moreover,iftheytrulybelievethatp= 2 iscorrect,theywouldbewillingtoaccept
5
either side of the bet: (a) win 3 units if C occurs and lose 2 if it does not occur,
or (b) win 2 units if C does not occur and lose 3 if it does. However, since the
mathematical properties of probability given in Section 3 are consistent with either
oftheseinterpretations,thesubsequentmathematicaldevelopmentdoesnotdepend
upon which approach is used.
The primary purpose of having a mathematical theory of statistics is to provide
mathematical models for random experiments. Once a model for such an experi-
ment has been provided and the theory worked out in detail, the statistician may,
within this framework, make inferences (that is, draw conclusions) about the ran-
domexperiment. Theconstructionofsuchamodelrequiresatheoryofprobability.
One of the more logically satisfying theories of probability is that based on the
concepts of sets and functions of sets. These concepts are introduced in Section 2.
2
Probability and Distributions
2 Set Theory
The concept of a set or a collection of objects is usually left undefined. However,
a particular set can be described so that there is no misunderstanding as to what
collection of objects is under consideration. For example, the set of the first 10
positiveintegersissufficientlywelldescribedtomakeclearthatthenumbers 3 and
4
14 are not in the set, while the number 3 is in the set. If an object belongs to a
set, it is said to be an element of the set. For example, if C denotes the set of real
numbers x for which 0 ≤ x ≤ 1, then 3 is an element of the set C. The fact that
4
3 is an element of the set C is indicated by writing 3 ∈ C. More generally, c ∈ C
4 4
means that c is an element of the set C.
The sets that concern us are frequently sets of numbers. However, the language
of sets of points proves somewhat more convenient than that of sets of numbers.
Accordingly, we briefly indicate how we use this terminology. In analytic geometry
considerable emphasis is placed on the fact that to each point on a line (on which
an origin and a unit point have been selected) there corresponds one and only one
number,sayx;andthattoeachnumberxtherecorrespondsoneandonlyonepoint
on the line. This one-to-one correspondence between the numbers and points on a
line enables us to speak, without misunderstanding, of the “point x” instead of the
“number x.” Furthermore, with a plane rectangular coordinate system and with x
andynumbers,toeachsymbol(x,y)therecorrespondsoneandonlyonepointinthe
plane; and to each point in the plane there corresponds but one such symbol. Here
again,wemayspeakofthe“point(x,y),”meaningthe“orderednumberpairxand
y.” This convenient language can be used when we have a rectangular coordinate
system in a space of three or more dimensions. Thus the “point (x ,x ,...,x )”
1 2 n
meansthenumbersx ,x ,...,x intheorderstated. Accordingly,indescribingour
1 2 n
sets, we frequently speak of a set of points (a set whose elements are points), being
careful, of course, to describe the set so as to avoid any ambiguity. The notation
C = {x : 0 ≤ x ≤ 1} is read “C is the one-dimensional set of points x for which
0 ≤ x ≤ 1.” Similarly, C = {(x,y) : 0 ≤ x ≤ 1,0 ≤ y ≤ 1} can be read “C is the
two-dimensional set of points (x,y) that are interior to, or on the boundary of, a
square with opposite vertices at (0,0) and (1,1).”
We say a set C is countable if C is finite or has as many elements as there are
positiveintegers. Forexample,thesetsC ={1,2,...,100}andC ={1,3,5,7,...}
1 2
are countable sets. The interval of real numbers (0,1], though, is not countable.
We now give some definitions (together with illustrative examples) that lead to
an elementary algebra of sets adequate for our purposes.
Definition 2.1. If each element of a set C is also an element of set C , the set C
1 2 1
is called a subset of the set C . This is indicated by writing C ⊂C . If C ⊂C
2 1 2 1 2
and also C ⊂ C , the two sets have the same elements, and this is indicated by
2 1
writing C =C .
1 2
Example 2.1. Let C = {x : 0 ≤ x ≤ 1} and C = {x : −1 ≤ x ≤ 2}. Here the
1 2
one-dimensional set C is seen to be a subset of the one-dimensional set C ; that
1 2
is, C ⊂ C . Subsequently, when the dimensionality of the set is clear, we do not
1 2
make specific reference to it.
3
Probability and Distributions
Example 2.2. DefinethetwosetsC ={(x,y):0≤x=y ≤1}andC ={(x,y):
1 2
0 ≤ x ≤ 1,0 ≤ y ≤ 1}. Because the elements of C are the points on one diagonal
1
of the square, then C ⊂C .
1 2
Definition 2.2. If a set C has no elements, C is called the null set. This is
indicated by writing C =φ.
Definition 2.3. The set of all elements that belong to at least one of the sets C
1
and C is called the union of C and C . The union of C and C is indicated by
2 1 2 1 2
writing C ∪C . The union of several sets C ,C ,C ,... is the set of all elements
1 2 1 2 3
thatbelong to at least one of the several sets, denoted by C ∪C ∪C ∪···=∪∞ C
1 2 3 j=1 j
or by C ∪C ∪···∪C =∪k C if a finite number k of sets is involved.
1 2 k j=1 j
We refer to a union of the form ∪∞ C as a countable union.
j=1 j
Example 2.3. Define the sets C = {x : x = 8,9,10,11, or 11 < x ≤ 12} and
1
C ={x:x=0,1,...,10}. Then
2
C ∪C = {x:x=0,1,...,8,9,10,11, or 11<x≤12}
1 2
= {x:x=0,1,...,8,9,10 or 11≤x≤12}.
Example 2.4. Define C and C as in Example 2.1. Then C ∪C =C .
1 2 1 2 2
Example 2.5. Let C =φ. Then C ∪C =C , for every set C .
2 1 2 1 1
Example 2.6. For every set C, C∪C =C.
Example 2.7. Let
(cid:2) (cid:3)
C = x: 1 ≤x≤1 , k =1,2,3,... .
k k+1
Then∪∞ C ={x:0<x≤1}. Notethatthenumberzeroisnotinthisset,since
k=1 k
it is not in one of the sets C ,C ,C ,....
1 2 3
Definition 2.4. The set of all elements that belong to each of the sets C and C
1 2
is called the intersection of C and C . The intersection of C and C is indicated
1 2 1 2
by writing C ∩C . The intersection of several sets C ,C ,C ,... is the set of all
1 2 1 2 3
elements that belong to each of the sets C ,C ,C ,.... This intersection is denoted
1 2 3
by C ∩C ∩C ∩···=∩∞ C or by C ∩C ∩···∩C =∩k C if a finite number
1 2 3 j=1 j 1 2 k j=1 j
k of sets is involved.
We refer to an intersection of the form ∩∞ C as a countable intersection.
j=1 j
Example 2.8. Let C ={(0,0),(0,1),(1,1)} and C ={(1,1),(1,2),(2,1)}. Then
1 2
C ∩C ={(1,1)}.
1 2
Example 2.9. Let C = {(x,y) : 0 ≤ x+y ≤ 1} and C = {(x,y) : 1 < x+y}.
1 2
Then C and C have no points in common and C ∩C =φ.
1 2 1 2
Example 2.10. For every set C, C∩C =C and C∩φ=φ.
4
Probability and Distributions
C C C C
1 2 1 2
(a) (b)
Figure 2.1: (a) C ∪C and (b) C ∩C .
1 2 1 2
Example 2.11. Let
(cid:4) (cid:5)
C = x:0<x< 1 , k =1,2,3,...
k k
Then ∩∞ C = φ, because there is no point that belongs to each of the sets
k=1 k
C ,C ,C ,....
1 2 3
Example 2.12. Let C and C represent the sets of points enclosed, respectively,
1 2
by two intersecting ellipses. Then the sets C ∪C and C ∩C are represented,
1 2 1 2
respectively, by the shaded regions in the Venn diagrams in Figure 2.1.
Definition2.5. Incertaindiscussionsorconsiderations,thetotalityofallelements
that pertain to the discussion can be described. This set of all elements under
consideration is given a special name. It is called the space. We often denote
spaces by letters such as C and D.
Example 2.13. Let the number of heads, in tossing a coin four times, be denoted
by x. Of necessity, the number of heads is of the numbers 0,1,2,3,4. Here, then,
the space is the set C ={0,1,2,3,4}.
Example 2.14. Consider all nondegenerate rectangles of base x and height y. To
be meaningful, both x and y must be positive. Then the space is given by the set
C ={(x,y):x>0,y >0}.
Definition2.6. LetC denoteaspaceandletC beasubsetofthesetC. Thesetthat
consists of all elements of C that are not elements of C is called the complement
of C (actually, with respect to C). The complement of C is denoted by Cc. In
particular, Cc =φ.
5