Probability and Statistical Inference: Volume 1: Probability

354 Pages·1985·6.106 MB·English

by  J. G. Kalbfleisch (auth.)

Springer Texts in Statistics
Advisors:
Stephen Fienberg Ingram Olkin
J. G. Kalbfleisch
Probability and
Statistical Inference
Volume 1: Probability
Second Edition
With 38 Illustrations
Springer Science+Business Media, LLC
J.G. Kalbfleisch
University of Waterloo
Department of Statistics and Actuarial Science
Waterloo, Ontario, N2L 3Gl
Canada
Editorial Board
Stephen Fienberg Ingram Olkin
York University Department of Statistics
North York, Ontario M3J IP3 Stanford University
CANADA Stanford, CA 94305
USA
AMS Classification: 60-01
Library of Congress Cataloging in Publication Data
Kalbfleisch, J. G.
Probability and statistical inference.
(Springer texts in statistics)
Includes indexes.
Contents: v. 1. Probability-v. 2. Statistical inference.
1. Probabilities. 2. Mathematical statistics.
I. Title. 11. Series.
QA273.K27 1985 519.5'4 85-12580
The first edition was published in two volumes, © 1979: Springer-Verlag New York, Inc.
Probability and Statistical Inference I (Universitext)
Probability and Statistical Inference 11 (Universitext)
© 1985 Springer Science+Business Media New York
Originally published by Springer-Verlag Berlin Heidelberg New York Tokyo in 1985
Softcover reprint of the hardcover 2nd edition 1985
All rights reserved. This work may not be translated or copied in whole or in part without the
written permission of the Springer Science+Business Media, LLC, except for brief excerpts in
connection with reviews or scholarly analysis. Use in connection with any form of information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication, even if
the former are not especially identified, is not to be taken as a sign that such names, as understood
by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.
Production managed by Dimitry L. LoselT; manufacturing supervised by Jacqui Ashri.
Typeset by H. Charlesworth & Co. Ltd., Huddersfield, England, and
by Asco Trade Typesetting Ltd., Hong Kong.
9 8 7 654 3
ISBN 978-1-4612-7009-6 ISBN 978-1-4612-1096-2 (eBook)
DOI 10.1007/978-1-4612-1096-2
Preface
This book is in two volumes, and is intended as a text for introductory
courses in probability and statistics at the second or third year university
level. It emphasizes applications and logical principles rather than math
ematical theory. A good background in freshman calculus is sufficient for
most of the material presented. Several starred sections have been included
as supplementary material. Nearly 900 problems and exercises of varying
difficulty are given, and Appendix A contains answers to about one-third
of them.
The first volume (Chapters 1-8) deals with probability models and with
mathematical methods for describing and manipulating them. It is similar
in content and organization to the 1979 edition. Some sections have been
rewritten and expanded-for example, the discussions of independent
random variables and conditional probability. Many new exercises have
been added.
In the second volume (Chapters 9-16), probability models are used as
the basis for the analysis and interpretation of data. This material has
been revised extensively. Chapters 9 and 10 describe the use of the like
lihood function in estimation problems, as in the 1979 edition. Chapter
11 then discusses frequency properties of estimation procedures, and in
troduces coverage probability and confidence intervals. Chapter 12 de
scribes tests of significance, with applications primarily to frequency data.
The likelihood ratio statistic is used to unify the material on testing, and
connect it with earlier material on estimation. Chapters 13 and 14 present
methods for analyzing data under the assumption of normality, with em
phasis on the importance of correctly modelling the experimental situation.
Chapter 15 considers sufficient statistics and conditional tests, and Chapter
16 presents some additional topics in statistical inference.
vi Preface
The content of volume two is unusual for an introductory text. The
importance of the probability model is emphasized, and general techniques
are presented for deriving suitable estimates, intervals, and tests from the
likelihood function. The intention is to avoid the appearance of a recipe
book, with many special formulas set out for type problems. A wide variety
of applications can be treated using the methods presented, particularly
if students have access to computing facilities.
I have omitted much of the standard material on optimality criteria for
estimators and tests, which is better left for later courses in mathematical
statistics. Also, I have avoided using decision-theoretic language. For in
stance, I discuss the calculation and interpretation of the observed sig
nificance level, rather than presenting the formal theory of hypothesis
testing. In most statistical applications, the aim is to learn from the data
at hand, not to minimize error frequencies in a long sequence of decisions.
I wish to thank my colleagues and students at the University of Waterloo
for their helpful comments on the 1979 edition, and on earlier drafts of
this edition. Special thanks are due to Professor Jock MacKay for his
many excellent suggestions, and to Ms. Lynda Hohner for superb technical
typing. Finally, I wish to express my appreciation to my wife Rebecca,
and children Jane, David, and Brian, for their encouragement and support.
I am grateful to the Biometrika trustees for permission to reproduce
material from Table 8 of Biometrika Tables for Statisticians. Vol. 1 (3rd
edition, 1966); to John Wiley and Sons Inc. for permission to reproduce
portions of Table II from Statistical Tables and Formulas by D. Hald
(1952); and to the Literary Executor of the late Sir Ronald Fisher, F.R.S.,
to Dr. Frank Yates, F.R.S., and to Longman Group Ltd., London, for
permission to reprint Tables I, III, and V from their book Statistical Tables
for Biological, Agricultural, and Medical Research (6th edition, 1974).
J. G. Kalbfleisch
Contents of Volume 1
Preface v
CHAPTER 1
Introduction
1.1 Probability and Statistics 1
1.2 Observed Frequencies and Histograms 3
1.3 Probability Models 12
1.4 Expected Frequencies 19
CHAPTER 2
Equi-Probable Outcomes 22
2.1 Combinatorial Symbols 22
2.2 Random Sampling Without Replacement 30
2.3 The Hypergeometric Distribution 38
2.4 Random Sampling With Replacement 44
2.5 The Binomial Distribution 48
2.6* Occupancy Problems 53
2.7* The Theory of Runs 56
2.8* Symmetric Random Walks 58
CHAPTER 3
The Calculus Of Probability 64
3.1 Unions and Intersections of Events 64
3.2 Independent Experiments and Product Models 70
3.3 Independent Events 76
3.4 Conditional Probability 81
3.5 Some Conditional Probability Examples 86
viii Contents
3.6 Bayes's Theorem 92
3.7* Union of n Events 99
Review Problems 104
CHAPTER 4
Discrete Variates 107
4.1 Definitions and Notation 107
4.2 Waiting Time Problems 116
4.3 The Poisson Distribution 124
4.4 The Poisson Process 128
4.5 Bivariate Distributions 134
4.6 Independent Variates 140
4.7 The Multinomial Distribution 146
Review Problems 151
CHAPTER 5
Mean and Variance 155
5.1 Mathematical Expectation 156
5.2 Moments; the Mean and Variance 164
5.3 Some Examples 171
5.4 Covariance and Correlation 176
5.5 Variances of Sums and Linear Combinations 182
5.6* Indicator Variables 189
5.7* Conditional Expectation 194
Review Problems 198
CHAPTER 6
Continuous Variates 200
6.1 Definitions and Notation 201
6.2 Uniform and Exponential Distributions 210
6.3* Transformations Based on the Probability Integral 215
6.4* Lifetime Distributions 220
6.5* Waiting Times in a Poisson Process 224
6.6 The Normal Distribution 229
6.7 The Central Limit Theorem 237
6.8 Some Normal Approximations 243
6.9 The Chi-Square Distribution 250
6.10 The F and t Distributions 255
Review Problems 262
CHAPTER 7
Bivariate Continuous Distributions 265
7.1 Definitions and Notation 265
7.2 Change of Variables 273
7.3 Transformations of Normal Variates 282
Contents ix
7.4* The Bivariate Normal Distribution 286
7.5* Conditional Distributions and Regression 290
CHAPTER 8
Generating Functions 295
8.1 * Preliminary Results 295
8.2* Probability Generating Functions 299
8.3* Moment and Cumulant Generating Functions 307
8.4* Applications 312
8.5* Bivariate Generating Functions 316
ApPENDIX A
Answers to Selected Problems 321
ApPENDIX B
Tables 329
Index 339
Contents of Volume 2
Preface
CHAPTER 9
Likelihood Methods
9.1 The Method of Maximum Likelihood
9.2 Combining Independent Experiments
9.3 Relative Likelihood
9.4 Likelihood for Continuous Models
9.5 Censoring in Lifetime Experiments
9.6 Invariance
9.7 Normal Approximations
9.8 Newton's Method
Review Problems
CHAPTER 10
Two-Parameter Likelihoods
10.1 Maximum Likelihood Estimation
10.2 Relative Likelihood and Contour Maps
10.3 Maximum Relative Likelihood
10.4 Normal Approximations
10.5 A Dose-Response Example
10.6 An Example from Learning Theory
10.7* Some Derivations
10.8* Multiparameter Likelihoods
XII Contents
CHAPTER 11
Frequency Properties
11.1 Sampling Distributions
11.2 Coverage Probability
11.3 Chi-square Approximations
11.4 Confidence Intervals
II.S Results for 2-Parameter Models
11.6* Expected Information and Planning Experiments
II. 7* Bias
CHAPTER 12
Tests of Significance
12. 1 Introduction
12.2 Likelihood Ratio Tests for Simple Hypotheses
12.3 Likelihood Ratio Tests for Composite Hypotheses
12.4 Tests for Binomial Probabilities
12.S Tests for Multinomial Probabilities
12.6 Tests for Independence in Contingency Tables
12.7 Cause and Effect
12.8 Testing for Marginal Homogeneity
12.9 Significance Regions
12.1O*Power
CHAPTER 13
Analysis of Normal Measurements
13.1 Introduction
13.2 Statistical Methods
13.3 The One-Sample Model
13.4 The Two-Sample Model
13.S The Straight Line Model
13.6 The Straight Line Model (continued)
13.7 Analysis of Paired Measurements
CHAPTER 14
Normal Linear Models
14.1 Matrix Notation
14.2 Parameter Estimates
14.3 Testing Hypotheses in Linear Models
14.4 More on Tests and Confidence Intervals
14.S Checking the Model
14.6* Derivations
CHAPTER 15
Sufficient Statistics and Conditional Tests
IS.1 The Sufficiency Principle
IS.2 Properties of Sufficient Statistics