An Introduction to Probability and Mathematical Statistics

235 Pages·1962·13.24 MB·English

by  Howard G. Tucker and Ralph P. Boas (Auth.)

Academic Press
Textbooks in Mathematics
Consulting Editor: Ralph P. Boas, Jr., Northwestern University
HOWARD G. TUCKER. An Introduction to Probability and
Mathematical Statistics
additional volumes in preparation
AN INTRODUCTION TO
PROBABILITY AND
MATHEMATICAL
STATISTICS
Howard G. Tucker
Department of Mathematics
University of California, Riverside
ACADEMIC PRESS
NEW YORK LONDON
COPYRIGHT 1962 BY ACADEMIC PRESS INC.
All Rights Reserved
NO PART OF THIS BOOK MAY BE REPRODUCED IN ANY FORM
BY PHOTOSTAT, MICROFILM, OR ANY OTHER MEANS,
WITHOUT WRITTEN PERMISSION FROM THE PUBLISHERS.
ACADEMIC PRESS INC.
Ill FIFTH AVENUE
NEW YORK, NEW YORK 10003
United Kingdom Edition
Published by
ACADEMIC PRESS INC. (LONDON) LTD.
BERKELEY SQUARE HOUSE, LONDON W. 1
Library of Congress Catalog Card Number 62-13131
First Printing, 1962
Second Printing, 1963
Third Printing, 1965
PRINTED IN THE UNITED STATES OF AMERICA
To My Mother and Father
Preface
This text in probability and mathematical statistics is designed for a
one-year course meeting three hours per week. It is designed for under­
graduate university students who are majoring in mathematics, who are
juniors or seniors, and who have already completed the standard freshman-
sophomore sequence of calculus courses. It is assumed that they are taking
at least one other undergraduate mathematics course concurrently; for
example, this course might be advanced calculus, linear algebra, modern
algebra, differential equations, set theory, or undergraduate topology. It
is not that any knowledge in any of these courses is a prerequisite to the
study of this book, but simply that at the time the student is using this
book he should be immersed in mathematics. Students who are majoring
in physics and who are favorably inclined toward abstract mathematics
are included in the class of students for whom this book is intended.
This book is not intended for anyone who is primarily interested in re­
sults and recipes. It should not be used as a text for service courses offered
for assorted majors in departments other than the mathematics and statis­
tics departments. This book is designed only for those students who love
mathematics and not for those who regard mathematics only as a tool.
As for the book itself, the student and instructor are hereby forewarned
that this is not a recipe book of statistical procedures. It is not a com­
pendium of statistical lore and is not even to be treated as a reference book.
Many topics usually found in junior-senior level statistics texts are pur­
posely omitted, e.g., sufficient statistics, the chi-square test, moment gen­
erating functions, sequential analysis, and nonparametric inference. In
addition, it is not an applied problem workbook. There are no long lists
of problems which require finding out whether one chewing gum chews
longer than another or whether a human being is less likely to get lung
cancer from one brand of cigarettes than another, i.e., this book is not
saturated with a repetitive list of numerical problems. Above all, this is
intended to be a mathematics book. Its title accurately reflects the original
desire of giving "an introduction" to probability theory and mathematical
statistics. The quoted pair of words found in the title of this book should
be considered in their exact meaning. The principal aim is to present the
vii
viii PREFACE
student with a solid foundation of probability theory from the mathemat­
ical point of view and to use this to introduce the essential ideas in depth of
three fields of statistical inference: point estimation, tests of hypotheses,
and confidence intervals. The entire presentation is decidedly from the
point of view of giving a sound mathematics course and of providing a
meeting place for other junior-senior level mathematics courses that the
student is now studying. Although the applications are not stressed and
are not of paramount interest, they are certainly not ignored. Indeed, the
exciting thing about probability and statistics brought out in the text is
that here is where purely abstract mathematics and physical reality do
come very close together.
The exercises found at the end of each section constitute a very impor­
tant part of this book. All of them should be assigned to the student as
he proceeds through the course. In keeping with the intended spirit of this
book, most exercises are of a theoretical nature. In solving the exercises at
the end of a section the student will often have to prove corollaries to the
theorems proved in that section. Also, he will frequently have to supply
missing steps to theorems proved in that section. Thus, the exercises are
not merely applications of results just proved; rather, they involve the
student in the entire mathematical development of that section and force
him to go over the details many times. Some of the exercises are rather
trivial, but some are quite difficult and will prove to be a definite challenge
to the better students. As was said before, these exercises are a very im­
portant part of the book, and as each section of the text is completed, all
of the exercises should be assigned to the student.
A few specific features of this book are the following:
1. Random variables are treated as measurable functions.
2. Sampling is treated in terms of product spaces.
3. Distributions are derived by the transformation method.
4. Probability is given an axiomatic treatment.
5. A chapter on the matrix theory needed is inserted in the middle of
the book.
6. The Neyman theory of confidence intervals is given a systematic
treatment.
7. A more natural definition of the multivariate normal distribution
is given.
8. Expectation is given a unified treatment; the expectation of a ran­
dom variable X is defined to be
PREFACE ix
provided that both of these improper Riemann integrals are finite. For­
mulas and properties in the discrete and absolutely continuous cases are
then derived from this definition.
The contents of this book were used in a one-year course which I taught
four times during the last six years and, except for the last half of Section
11.4, constitute no more nor no less than the material I actually presented
in class. My colleague, Professor Richard C. Gilbert, taught this same
course twice and based his lectures on notes of mine which eventually
became this book. I have always been able to cover the material in this
book in two semesters, each of fifteen weeks, in three hours of lecture per
week. The fall semester always ends somewhere in the middle of Chapter 8.
On the last day of the fall semester I always give a brief introduction to
hypothesis testing and present the notion of randomized tests as given in
the example in Section 11.4. In the spring semester I begin wherever I left
off at the end of the fall semester and proceed to the end of the book. I
usually spend from four to six weeks on Chapter 9 (Matrix Theory). Even
when every student in the class already knows some matrix theory, I
assume the contrary, and we proceed at a uniform rate through this chap­
ter. (It usually turns out that my assumption was correct.) During the
time spent on this chapter the homework assignments consist of complete,
detailed proofs of the theorems covered each hour.
I wish to express my appreciation to my colleague, Professor Richard C.
Gilbert, for many helpful discussions which greatly improved this book.
I am indebted to Professor Sir Ronald A. Fisher, F. R. S., Cambridge, and
to Dr. Frank Yates, F. R. S., Rothamsted, also to Messrs. Oliver and Boyd,
Ltd., Edinburgh for permission to include Table IV which is an abridgment
of their Table III from their book Statistical Tables for Biological, Agricul­
tural and Medical Research. Mrs. Julia Rubalcava typed the manuscript
speedily and accurately, and I wish to acknowledge my gratitude to her.
Riverside, California HOWARD G. TUCKER
January, 1962
CHAPTER 1
Events and
Probabilities
1.1 Combinatorial Probability
The notion of the probability of an event is approached by three different
methods. One method, perhaps the first historically, is to repeat an experi­
ment or game many times under identical conditions and compute the
relative frequency with which an event occurs. This means: divide the
total number of times that the specific event occurs by the total number
of times the experiment is performed or the game is played. This ratio is
called the relative frequency. Although this method of arriving at the notion
of probability is the most primitive and unsophisticated, it is the most
meaningful to the practical individual and the working scientist or engi­
neer who has to apply the results of probability theory to real-life situations.
Accordingly, whatever results one obtains in the theory of probability
and statistics, one should be able to interpret them in terms of relative
frequency.
A second way of approaching the notion of probability is from an
axiomatic point of view. That is, a minimal list of axioms is set down which
assumes certain properties of probabilities. From this minimal set of as­
sumptions the further properties of probability are deduced and applied.
The axiomatic approach will be used in this book.
The third method of arriving at the notion of probability is limited in
application but is extremely useful. It is the subject considered in this
section. This method is briefly stated as follows in terms of certain un­
defined words, which we introduce in quotation marks. Let us suppose
that an experiment or game has a certain number of mutually exclusive
"equally likely" outcomes. Let us also suppose that a certain event can
occur in any one of a specified number of these "equally likely" outcomes.
Then the probability of the event is defined to be the number of "equally
likely" ways in which the event can occur divided by the total number of
1
2 EVENTS AND PROBABILITIES [Chap. 1
possible "equally likely" outcomes. It must be emphasized here that the
number of equally likely ways in which the event can occur must be from
among the total number of equally likely outcomes. When we mentioned
above that this method is limited, we meant that in some games or experi­
ments not all the possible outcomes are "equally likely." Before we give
applications of this method, it will be necessary to review some notions
from high school algebra.
Let us suppose that we have n different objects, and we want to arrange
k of these in a row (where, of course, k n). We want to know in how
=
many ways we can accomplish this feat. As an example, suppose there are
five members of a club, call them A, B, C, D, E, and we want to know in
how many ways we can select a chairman and a secretary. When we
select the arrangement (C, A), we mean that C is the chairman and A is
the secretary. In this case n = 5 and k = 2. The different arrangements
are listed as follows:
(A,B) (A,Q (A, D) (A,E)
(B, A) (B, C) (B,D) (B,E)
(C, A) (C, B) (C, D) (c, m
(D,A) (D, B) (A O (D,E)
(E,A) (E, B) (E, C) (E, D)
One easily sees that there are 20 such arrangements. This number 20 can
also be obtained by the following reasoning: there are five ways in which
the chairman can be selected (which accounts for the five horizontal rows
of pairs), for each chairman selected there are four ways of selecting the
secretary (which accounts for the four vertical columns), and consequently
there are 20 such pairs. In general, if we want to arrange k out of n objects,
we reason as follows. There are n ways of selecting the first object. For
each way we select the first object there are n — 1 ways of selecting
the second object. Hence the total number of ways in which the first
two objects can be selected is n(n — 1). For every way in which the
first two objects are selected, there are n — 2 ways of selecting the third
object. Thus the number of ways in which the first three objects can be
selected is n(n — l)(n — 2). From this one can easily observe that
the number of ways in which the first k objects can be selected is
n(n — l)(n — 2) • • -(n — (k — 1)), which can also be written as the ratio
of factorials: n\/{n - k)l (Recall: 51 = 1 X 2 X 3 X 4 X 5 .) This is also
referred to as the number of permutations of n things taken & at a time.
In the above arrangements (or permutations), due regard must be given
to the order in which the k items were selected. Suppose, however, that
one is interested only in the number of ways k things can be selected but is
not interested in the order or arrangement. For example, in the case of the
Sec. 1.1] COMBINATORIAL PROBABILITY 3
club discussed above, the ways in which two members can be selected
out of the five to form a committee are as follows:
(A,B) (A,C) (A, D) (A, E)
(B, C) (B, D) (B, E)
(C, D) (C, E)
(D, E).
We do not list (D, B) as before because the committee denoted by (D, B)
and the committee denoted by (B, D) are the same. Thus, now we have
only half the number of selections. In general, if we want to find the number
of ways in which we can select k things out of n things, we reason it out
as follows: there are n\/(n — k)! ways of arranging (or permuting) n things
taken k at a time. However, we have too large a selection, since each time
we obtain k particular things there are k! ways of arranging them. Hence
we want to divide the number of ways of permuting n things taken k at
a time by k! to obtain the desired answer. The number of ways in which
we can select k objects out of n objects is usually referred to as the number
of combinations of n things taken k at a time and is denoted by the binomial
coefficient:
We shall also need the binomial theorem which states that
n
(a + b)
for every positive integer n, or, what amounts to the same thing,
Naturally, 0! is defined to be equal to one.
Now let us solve some problems. In each case we shall want to find the
probability, P, of a certain event. Accordingly, we first determine the
total number of all "equally likely" outcomes. Then we determine that
number of "equally likely" ways among these in which this event can
occur. Finally, we divide this last number by the preceding number to
obtain P.
Example 1. The numbers 1,2, • • • , n are arranged in random order, i.e.,
the n! ways in which these n numbers can be arranged are assumed to be
equally likely. We are to find the probability that the numbers 1 and 2
appear as neighbors in the order named.
As we have just noted, there are n! ways in which these integers can be