Introduction to Probability and Mathematical Statistics

658 Pages·2000·22.36 MB·English

by  Lee J. Bain, Max Engelhardt

Student's t

X
t(v)

v= 1,2,...

Snedecor's F

X
F(v1, y2)

Beta

X - BETA(a,b)

O <a
0<b

*Not tractable.
Does not exist.

Specia' Continuous Distributions

(y1 + v2

2
) (vii!
(y \
Ív2\ '\!'2)
\\2) \2

O

l<v

V
v-2

2<v

F'(a)F(b)
O<x<1

a
a+b
ab
(a+b+ l)(a+b)2

Notation and
Parameters
Continuous pdf fix)
Mean
Variance
MGF M(t)

2v(v1 +v2-2)
v1(v2 2)2(v2 4)
2

y1 = 1,2,.

y2 = 1,2,.
2<v2
4<v2
\
VJ

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

Cauchy

X CAU(O, ,j)
0<0

Pareto

X -
PAR(O, ,c)

0<0

Specia' Continuous Distributions

thr{1 + [(x )/0],}

¡ç

0(1 + X/O)K +1
0<x

y
0.5772
(Euler's
const.)

O
02K

Chi-Square

X -'- x2(v)
X''2 'e'2
V
2v
2/2 U(v/2)
v=1,2,...
0<x

ß

**

I 2t

Notation and
Parameters
Continuous pdf f(x)
Mean
Variance
MGF M(t)

WeibuH

X WEI(0,ß)
xe'
or(i +)
02[(t +
-

0<0
0<x
0<ß

Extreme
Value

X
EV(0, 17)
exp {[(x )/O] exp [(x-17)/0]}
- yO

0<0

'ç1
(-2)( 1)2

1<K
2<

2O2

6
&T(i + Ot)

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

INTRODUCTION
TO PROBABILITY
AND MATHEMATICAL
STATISTICS

SECOND EDITION

Lee J. Bain
University of Missouri - Rol/a

Max Engelhardt
University of Idaho

At Du::bur
Thomson Learñing

Australia Canada Mexico Singapore
Spain
United Kingdom United States

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

The Duxbuiy Classic Series is a collection of authoritativeworks from respected authors.
Reissued as paperbacks, these successful titles are now more affordable.

COPYRIGHT © 1992, 1987 by Brooks/Cole
Dux bury is an imprint of Brooks/Cole, a division of Thomson Learning
The Thomson Learning logo is a trademark used herein under license.

For more information about this or any óther Duxbury product, contact:
DUXBTJRY
511 Forest Lodge Road
Pacific Grove, CA 93950 USA
www.duxbury.com
1-800-423-0563 (Thomson Learning Academic Resource Center)

All rights reserved. No part of this work may be reproduced, transcribed or used in any form or by any
meansgraphic, electronic, or mechanical, including photocopying, recording, taping, Web distribution,
or information storage and/or retrieval systemswithout the prior written permission of the publisher.

For permission to use material from this work, contact us by
Web:
www.thomsonrights.com
fax:
1-800-730-2215
phone:
1-800-730-2214

Printed in the United States of America

10
9
8
7
6
5
4
3
2

Library of Congress Cataloging-in-Publication Data
Bain, Lee J.
Introduction to probability and mathematical statistics / Lee J. Bain, Max Engelhardt.-2°' ed.
p.
cm.(The Duxbuiy advanced series in statistics and decision sciences)
Includes bibliographical references and index.
ISBN 0-534-92930-3 (hard cover)
ISBN 0-534-38020-4 (paperback)
1. Probabilities.
2. Mathematical statistics.
I. Engelhardt, Max.
II. Title.
ifi. Series.
QA273.B2546
1991
519.2---dc2O
91-25923
dF

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

CONTENTS

CHA PTER

PROBABILITY
i

1.1
Introduction
1

1.2
Notation and terminology
2

1.3
Definition of probability
9

1.4
Some properties of probability
13

1.5
Conditional probability
16

1.6
Counting techniques
31
Summary
42
Exercises
43

CHAPTER

RANDOM VARIABLES AND THEIR
DISTRIBUTIONS 53

2.1
Introduction
53
2.2
Discrete random variables
56

2.3
Continuous random variables
62
2.4
Some properties of expected values
71

2.5
Moment generating functions
78
Summary
83
Exercises
83

V

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

Vi
CONTENTS

CHAPTER

SPECIAL PROBABILITY DISTRIBUTIONS
90

3.1
Introduction
90
3.2
Special discrete distributions
91
3,3
Special continuous distributions
109
3.4
Location and scale parameters
124
Summary
127
Exercises
128

CHAPTER

JOINT DISTRIBUTIONS
136

4.1
Introduction
136
4.2
Joint discrete distributions
137
4.3
Joint continuous distributions
144
4.4
Independent random variables
149
4.5
Conditional distributions
153
4.6
Random samples
158
Summary
165
Exercises
165

CHAPTER

PROPERTIES OF RANDOM VARIABLES 171

5.1
Introduction
.
171
5.2
Properties of expected values
172
5.3
Correlation
177
5.4
Conditional expectation
180
5;5
Joint moment generating functions
186
Summary
188
Exercises
189

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

FUNCTIONS OF RANDOM VARIABLES 193

6.1
Introduction
193
6.2
The CDF' technique
194

6.3
Transformation methods
197
6.4
Sums of random variables
209

6.5
Order statistics
214
Summary
226
Exercises
226

CHA PTER

LIMITING DISTRIBUTIONS
231

7.1
Introduction
231

7.2
Sequences of random variables
232
7.3
The central limit theorem
236
7.4
Approximations for the binomial distribution
240

7.5
Asymptotic normal distributions
243

7.6
Properties of stochastic convergence
245

7.7
Additional limit theorems
247
7.8* Asymptotic distributions of extreme-order statistics
250
Summary
259
Exercises
259

CHAPTER

STATISTICS AND SAMPLING
DISTRIBUTIONS 263

8.1
Introduction
263

8.2
Statistics
263

8.3
Sampling distributions
.
267
8.4
The t, F, and beta distributions
273

8.5
Large-sample approximations
280
Summary
283
Exercises
283

* Advanced (or optional) topics

CONTENTS
vu

CHAPTER

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

viii
CONTENTS

CHAPTER

POINT ESTIMATION 288

9.1
Introduction
288
9.2
Some methods of estimation
290

9.3
Criteria for evaluating estimators
302
9.4
Large-sample properties
311

9.5
Bayes and minimax estimators
319
Summary
327
Exercises
328

CHAPTER
10

INTERVAL ESTIMATION 358

11.1
Introduction
358
11.2
Confidence intervals
-
359
11.3
Pivotal quantity method
362
11.4
General method
369
11.5
Two-sample problems
377
11.6
Bayesian interval estimation
382
Summary
383
Exercises
384

SUFFICIENCY AND COMPLETENESS 335

10.1
Introduction
335
10.2
Sufficient statistics
337
10.3
Further properties of sufficient statistics
342
10.4
Completeness and exponential class
345
Summary
353
Exercises
353

CHAPTER
11

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

TESTS OF HYPOTHESES 389

12 1
Introduction
389

12.2
Composite hypotheses
395

12.3
Tests for the normal distribution
398

12.4
Binomial tests
404

12.5
Poisson tests
406

12.6
Most powerful tests
406

12.7
Uniformly most powerful tests
411

12.8
Generalized likelihood ratio tests
417

12.9
Conditional tests
426
12.10 Sequential tests
428
Summary
435
Exercises
-
436

CHAPTER
13

CONTINGENCY TABLES
AND GOODNESS-OF-FIT 442

13.1
Introduction
442

13.2
One-sample binomial case
443

13.3
r-Sample binomial test (completely specified H0)
444

13.4
One-sample multinomial
447

13.5
r-Sample multinomial
448

13.6
Test for independence, r x c contingency table
450

13.7
Chi-squared goodness-of-fit test
453

13.8
Other goodness-of-fit tests
457
Summary
461
Exercises
462

CHAPTER
14

NONPARAMETRIC METHODS 468

14.1
Introduction
468

14.2
One-sample sign test
469

14.3
Binomial test (test on quantiles)
471

14.4
Two-sample sign test
476

CONTENTS
ix

CI-/APTER
12

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

APPENDIXA REVIEWOFSETS 587

APPENDIX B
SPECIAL DISTRIBUTIONS 594

APPENDIX C TABLES OF DISTRIBUTIONS 598

ANSWERS TO SELECTED EXERCISES 619

REFERENCES 638

INDEX 641

Advanced (or optional) topics

CONTENTS

Wilcoxon paired-sample signed-rank test
Paired-sample randomization test
Wilcoxon and Mann-Whitney (WMW) tests
Correlation teststests of independence
Wald-Wolfowjtz runs test
Summary
Exercises

CHAPTER
J5*

477
482
483
486
492
494

495

14.5
14.6

14.7
14.8
14.9

15.1
15.2
15.3
15.4

15.5

16.1
16.2

16.3
16.4
16.5

REGRESSION AND LINEAR MODELS

Introduction
Linear regression
Simple linear regression
General linear model
Analysis of hivariate data
Summary
Exercises

*
CHAPTER

499

499
500

501
515
529
534

535

RELIABILITY AND SURVIVAL
DISTRIBUTIONS 540

Introduction
Reliability concepts
Exponential distribution
Weibull distribution
Repairable systems
Summary
Exercises

540
541
548
560
570
579

579

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

PREFACE

This book provides an introduction to probability and mathematical statistics.
Although the primary focus of the book is on a mathematical development of the
subject, we also have included numerous examples and exercises that are oriented
toward applications. We have attempted to achieve a level of presentation that is
appropriate for senior-level undergraduates and beginning graduate students.
The second edition involves several major changes, many of which were sug
gested by reviewers and users of the first edition. Chapter 2 now is devoted to
general properties of random variables and their distributions. The chapter now
includes moments and moment generating functions, which occurred somewhat
later in the first edition. Special distributions have been placed in Chapter 3.
Chapter 8 is completely changed. It now considers sampling distributions and
some basic properties of statistics. Chapter 15 is also new. It deals with regression
and related aspects of linear models
As with the first edition, the only prerequisite for covering the basic material is
calculus, with the lone exception of the material on general linear models in
Section 15.4; this assumes some familiarity with matrices. This material can be
omitted if so desired.
Our intent was to produce a book that could be used as a textbook for a
two-semester sequence in which the first semester is devoted to probability con-
cepts and the second covers mathematfcal statistics. Chapters 1 through 7 include
topics that usually are covered in a one-semester introductory course in probabil-
ity, while Chapters 8 through 12 contain standard topics in mathematical sta-
tistics. Chapters 13 and 14 deal with goodness-of-fit and nonparametric statistics.
These chapters tend to be more methods-oriented. Chapters 15 and 16 cover
material in regression and reliability, and these would be considered as optional
or special topics. In any event, judgment undoubtedly will be required in the

xi

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

xii
PREFACE

choice of topics covered or the amount of time allotted to topics if the desired
material is to be completed in a two-semester course.
lt is our hope that those who use the book will find it both interesting and
informative.

ACKNOWLEDG M ENTS

We gratefully acknowledge the numerous suggestions provided by the following
reviewers:

Dean H. Fearn
Alan M. Johnson
Calfornia State UniversityHayward
University of Arkansas, Little Rock

Joseph Glaz
Benny P. Lo
University of Connecticut
Ohione College

Victor Goodman
D. Ramachandran
Rensselaer Polytechnic Institute
California State UniversitySacramento

Shu-ping C. Hodgson
Douglas A. Wolfe
Central Michigan University
Ohio State University

Robert A. Hultquist
Linda J. Young
Pennsylvania State University
Oklahoma State University

Thanks also are due to the following users of the first edition who were kind
enough to relate their experiences to the authors: H. A. David, Iowa State Uni-
versity; Peter Griffin, California State UniversitySacramento.
Finally, special thanks are due for the moral support of our wives, Harriet Bain
and Linda Engelhardt.
Lee J. Bain
Max Engelhardt

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

C
H
A
P
T

1.1

INTRODUCTION

In any scientific study of a physical phenomenon, it is desirable to have a mathe-
matical model that makes it possible to describe or predict the observed value of
some characteristic of interest. As an example, consider the velocity of a falling
body after a certain length of time, t. The formula y = gt, where g
32.17 feet per
second per second, provides a useful mathematical model for the velocity, in feet
per second, of a body falling from rest in a vacuum. This is an example of a
deterministic model. For such a model, carrying out repeated experiments under
ideal conditions would result in essentially the same velocity each time, and this
would be predicted by the model. On the other hand, such a model may not be
adequate when the experiments are carried out under less than ideal conditions.
There may be unknown or uncontrolled variables, such as air temperature or
humidity, that might affect the outcome, as well as measurement error or other
factors that might cause the results to vary on different performances of the

PROBtJ3ILITY

I

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

2
CHAPTER 1
PROBABILITY

experiment. Furthermore, we may not have sufficient knowledge to derive a more
complicated model that could account for all causes of variation.
There are also other types of phenomena in which different results may natu-
rally occur by chance, and for which a deterministic model would not be appro.
priate. For example, an experiment may consist of observing the number of
particles emitted by a radioactive source, the time until failure of a manufactured
component, or the outcome of a game of chance.
The motivation for the study of probability is to provide mathematical models
for such nondeterministic situations; the corresponding mathematical models will
be called probability models (or probabilistic models). The term stochastic, which
is derived from the Greek word stochos, meaning "guess," is sometimes used
instead of the term probabilistic.
A careful study of probability models requires some familiarity with the nota-
tion and terminology of set theory. We will assume that the reader has some
knowledge of sets, but for convenience we have included a review of the basic
ideas of set theory in Appendix A.

L2

NOTATION AND TERMINOLOGY

The term experiment refers to the process of obtaining an observed result of some
phenomenon. A performance of an experiment is called a trial of the experiment,
and an observed result is called an outcome. This terminology is rather general,
and it could pertain to such diverse activities as scientific experiments or games
of chance. Our primary interest will be in situations where there is uncertainty
about which outcome will occur when the experiment is performed We will
assume that an experiment is repeatable under essentially the same conditions,
and that the set of all possible outcomes can be completely specified before
experimentation.

Definition 1.2.1

The set of all possible outcomes of an experiment is called the sample space, denoted
by S.
Note Chat one and only one of the possible outcomes will occur on any given trial
of the experiments.

Exaíipk i .2.1
An experiment consists of tossing two coins, and the observed face of each coin is
of interest. The set of possible outcomes may be represented by the sample space

S = {HH, HT, TH, TT}

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor

1.2
NOTATION AND TERMINOLOGY
3

which simply lists all possible pairings of the symbols H (heads) and T (tails). An
alternate way of representing such a sample space is to list all possible ordered
pairs of the numbers i and O, S = {(l, 1), (1, 0), (0, 1), (0, O)}, where, for example,
(1, 0) indicates that the first coin landed heads up and the second coin landed
tails up.

Example 1.2.2
Suppose that in Example 1.2.1 we were not interested in the individual outcomes
of the coins, but only in the total number of heads obtained from the two coins.
An appropriate sample space could then be written as S'1' = {0, 1, 2}. Thus, differ-
ent sample spaces may be appropriate for the same experiment, depending on the
characteristic of interest.

Exampl& 1.2.3
If a coin is tossed repeatedly until a head occurs, then the natural sample space is
S = {H, TH, TTH, . . .}. If one is interested in the number of tosses required to
obtain a head, then a possible sample space for this experiment would be the set
of all positive integers, S'1 = {1, 2, 3,
. . .}, and the outcomes would correspond
directly to the number of tosses required to obtain the first head. We will show in
the next chapter that an outcome corresponding to a sequence of tosses in which
a head is never obtained need not be included in the sample space.

Exampk 1.2.4
A light bulb is placed in service and the time of operation until it burns out is
measured, At least conceptually, the sample space for this experiment can be
taken to be the set of nonnegative real numbers, S = {tIO
t < c}.
Note that if the actual failure time could be measured only to the nearest hour,
then the sample space for the actual observed failure time would be the set of
nonnegative integers, S'i' = {O, 1, 2, 3, .
. .}. Even though S* may be the observable
sample space, one might prefer to describe the properties and behavior of light
bulbs in terms of the conceptual sample space S. In cases of this type, the dis-
creteness imposed by measurement limitations is sufficiently negligible that it can
be ignored, and both the measured response and the conceptual response can be
discussed relative to the conceptual sample space S.

A sample space S is said to be finite if it consists of a finite number of out-
comes, say S = {e1, e2 .....eN}, and it is said to be countably infinite if its out-
comes can be put into a one-to-one correspondence with the positive integers, say
S= {e1,e2,...}.

Definition 1.2.2

If a sample space S is either finite or countably infinite then it is called a thscrete
sample space.

PDF compression, OCR, web optimization using a watermarked evaluation copy of CVISION PDFCompressor