Applied Probability

2008·6.2 MB·English

by  Paul E Pfeiffer

Applied Probability
By:
Paul E Pfeiffer

Applied Probability
By:
Paul E Pfeiffer
Online:
<http://cnx.org/content/col10708/1.6/ >
C O N N E X I O N S
RiceUniversity,Houston,Texas
'2008 Paul E Pfei(cid:27)er
This selection and arrangement of content is licensed under the Creative Commons Attribution License:
http://creativecommons.org/licenses/by/3.0/
Table of Contents
Preface to Pfei(cid:27)er Applied Probability ...........................................................1
1 Probability Systems
1.1 Likelihood ...................................................................................5
1.2 Probability Systems .........................................................................9
1.3 Interpretations .............................................................................14
1.4 Problems on Probability Systems ...........................................................19
Solutions ........................................................................................23
2 Minterm Analysis
2.1 Minterms ...................................................................................25
2.2 Minterms and MATLAB Calculations ......................................................34
2.3 Problems on Minterm Analysis .............................................................43
Solutions ........................................................................................48
3 Conditional Probability
3.1 Conditional Probability .....................................................................61
3.2 Problems on Conditional Probability .......................................................70
Solutions ........................................................................................74
4 Independence of Events
4.1 Independence of Events .....................................................................79
4.2 MATLAB and Independent Classes .........................................................83
4.3 Composite Trials ...........................................................................89
4.4 Problems on Independence of Events .......................................................95
Solutions .......................................................................................101
5 Conditional Independence
5.1 Conditional Independence .................................................................109
5.2 Patterns of Probable Inference .............................................................114
5.3 Problems on Conditional Independence ....................................................123
Solutions .......................................................................................129
6 Random Variables and Probabilities
6.1 Random Variables and Probabilities .......................................................135
6.2 Problems on Random Variables and Probabilities ..........................................148
Solutions .......................................................................................152
7 Distribution and Density Functions
7.1 Distribution and Density Functions ........................................................161
7.2 Distribution Approximations ..............................................................174
7.3 Problems on Distribution and Density Functions ...........................................184
Solutions .......................................................................................189
8 Random Vectors and joint Distributions
8.1 Random Vectors and Joint Distributions ...................................................195
8.2 Random Vectors and MATLAB ...........................................................202
8.3 Problems On Random Vectors and Joint Distributions .....................................211
Solutions .......................................................................................215
9 Independent Classes of Random Variables
9.1 Independent Classes of Random Variables .................................................231
9.2 Problems on Independent Classes of Random Variables ....................................242
iv
Solutions .......................................................................................247
10 Functions of Random Variables
10.1 Functions of a Random Variable ..........................................................257
10.2 Function of Random Vectors .............................................................263
10.3 The Quantile Function ...................................................................278
10.4 Problems on Functions of Random Variables .............................................287
Solutions .......................................................................................294
11 Mathematical Expectation
11.1 Mathematical Expectation: Simple Random Variables ....................................301
11.2 Mathematical Expectation; General Random Variables ...................................309
11.3 Problems on Mathematical Expectation ..................................................326
Solutions .......................................................................................334
12 Variance, Covariance, Linear Regression
12.1 Variance .................................................................................345
12.2 Covariance and the Correlation Coe(cid:30)cient ...............................................356
12.3 Linear Regression ........................................................................361
12.4 Problems on Variance, Covariance, Linear Regression .....................................366
Solutions .......................................................................................374
13 Transform Methods
13.1 Transform Methods ......................................................................385
13.2 Convergence and the central Limit Theorem ..............................................395
13.3 Simple Random Samples and Statistics ...................................................404
13.4 Problems on Transform Methods .........................................................408
Solutions .......................................................................................412
14 Conditional Expectation, Regression
14.1 Conditional Expectation, Regression .....................................................419
14.2 Problems on Conditional Expectation, Regression ........................................437
Solutions .......................................................................................444
15 Random Selection
15.1 Random Selection ........................................................................455
15.2 Some Random Selection Problems ........................................................464
15.3 Problems on Random Selection ...........................................................476
Solutions .......................................................................................482
16 Conditional Independence, Given a Random Vector
16.1 Conditional Independence, Given a Random Vector ......................................495
16.2 Elements of Markov Sequences ...........................................................503
16.3 Problems on Conditional Independence, Given a Random Vector .........................523
Solutions .......................................................................................527
17 Appendices
17.1 AppendixAtoAppliedProbability: Directoryofm-functionsandm-procedures............531
17.2 Appendix B to Applied Probability: some mathematical aids .............................592
17.3 Appendix C: Data on some common distributions ........................................596
17.4 Appendix D to Applied Probability: The standard normal distribution ...................598
17.5 Appendix E to Applied Probability: Properties of mathematical expectation ..............599
17.6 AppendixFtoAppliedProbability: Propertiesofconditionalexpectation,given
a random vector ...........................................................................601
17.7 Appendix G to Applied Probability: Properties of conditional independence,
given a random vector .....................................................................602
17.8 Matlab (cid:28)les for "Problems" in "Applied Probability" .....................................603
v
Solutions ........................................................................................??
Index ...............................................................................................615
Attributions ........................................................................................618
vi
Preface to Pfei(cid:27)er Applied Probability
1
The course
This is a "(cid:28)rst course" in the sense that it presumes no previous course in probability. The units are
modules taken from the unpublished text: Paul E. Pfei(cid:27)er, ELEMENTS OF APPLIED PROBABILITY,
USING MATLAB. The units are numbered as they appear in the text, although of course they may be used
in any desired order. For those who wish to use the order of the text, an outline is provided, with indication
of which modules contain the material.
Themathematicalprerequisitesareordinarycalculusandtheelementsofmatrixalgebra. Afewstandard
series and integrals are used, and double integrals are evaluated as iterated integrals. The reader who can
evaluate simple integrals can learn quickly from the examples how to deal with the iterated integrals used
in the theory of expectation and conditional expectation. Appendix B (Section 17.2) provides a convenient
compendium of mathematical facts used frequently in this work. And the symbolic toolbox, implementing
MAPLE, may be used to evaluate integrals, if desired.
In addition to an introduction to the essential features of basic probability in terms of a precise mathe-
matical model, the work describes and employs user de(cid:28)ned MATLAB procedures and functions (which we
refer to as m-programs, or simply programs) to solve many important problems in basic probability. This
should make the work useful as a stand alone exposition as well as a supplement to any of several current
textbooks.
MostoftheprogramsdevelopedherewerewritteninearlierversionsofMATLAB,buthavebeenrevised
slightly to make them quite compatible with MATLAB 7. In a few cases, alternate implementations are
available in the Statistics Toolbox, but are implemented here directly from the basic MATLAB program,
so that students need only that program (and the symbolic mathematics toolbox, if they desire its aid in
evaluating integrals).
Since machine methods require precise formulation of problems in appropriate mathematical form, it
is necessary to provide some supplementary analytical material, principally the so-called minterm analysis.
This material is not only important for computational purposes, but is also useful in displaying some of the
structure of the relationships among events.
A probability model
Much of "real world" probabilistic thinking is an amalgam of intuitive, plausible reasoning and of statistical
knowledge and insight. Mathematical probability attempts to to lend precision to such probability analysis
byemployingasuitablemathematicalmodel,whichembodiesthecentralunderlyingprinciplesandstructure.
A successful model serves as an aid (and sometimes corrective) to this type of thinking.
Certain concepts and patterns have emerged from experience and intuition. The mathematical formu-
lation (the mathematical model) which has most successfully captured these essential ideas is rooted in
measure theory, and is known as the Kolmogorov model, after the brilliant Russian mathematician A.N.
Kolmogorov (1903-1987).
1Thiscontentisavailableonlineat<http://cnx.org/content/m23242/1.7/>.
1
2
One cannot prove that a model is correct. Only experience may show whether it is useful (and not
incorrect). The usefulness of the Kolmogorov model is established by examining its structure and show-
ing that patterns of uncertainty and likelihood in any practical situation can be represented adequately.
Developments, such as in this course, have given ample evidence of such usefulness.
The most fruitful approach is characterized by an interplay of
• A formulation of the problem in precise terms of the model and careful mathematical analysis of the
problem so formulated.
• A grasp of the problem based on experience and insight. This underlies both problem formulation
and interpretation of analytical results of the model. Often such insight suggests approaches to the
analytical solution process.
MATLAB: A tool for learning
In this work, we make extensive use of MATLAB as an aid to analysis. I have tried to write the MATLAB
programs in such a way that they constitute useful, ready-made tools for problem solving. Once the user
understands the problems they are designed to solve, the solution strategies used, and the manner in which
these strategies are implemented, the collection of programs should provide a useful resource.
However, my primary aim in exposition and illustration is to aid the learning process and to deepen
insight into the structure of the problems considered and the strategies employed in their solution. Several
features contribute to that end.
1. Application of machine methods of solution requires precise formulation. The data available and the
fundamental assumptions must be organized in an appropriate fashion. The requisite discipline for
such formulation often contributes to enhanced understanding of the problem.
2. The development of a MATLAB program for solution requires careful attention to possible solution
strategies. One cannot instruct the machine without a clear grasp of what is to be done.
3. I give attention to the tasks performed by a program, with a general description of how MATLAB
carries out the tasks. The reader is not required to trace out all the programming details. However,
it is often the case that available MATLAB resources suggest alternative solution strategies. Hence,
for those so inclined, attention to the details may be fruitful. I have included, as a separate collection,
the m-(cid:28)les written for this work. These may be used as patterns for extensions as well as programs in
MATLAB for computations. Appendix A (Section 17.1) provides a directory of these m-(cid:28)les.
4. Some of the details in the MATLAB script are presentation details. These are re(cid:28)nements which are
not essential to the solution of the problem. But they make the programs more readily usable. And
theyprovideillustrationsofMATLABtechniquesforthosewhomaywishtowritetheirownprograms.
I hope many will be inclined to go beyond this work, modifying current programs or writing new ones.
An Invitation to Experiment and Explore
Because the programs provide considerable freedom from the burden of computation and the tyranny of
tables (with their limited ranges and parameter values), standard problems may be approached with a new
spirit of experiment and discovery. When a program is selected (or written), it embodies one method of
solution. Theremaybeotherswhicharereadilyimplemented. Thereaderisinvited, evenurged, toexplore!
The user may experiment to whatever degree he or she (cid:28)nds useful and interesting. The possibilities are
endless.
Acknowledgments
After many years of teaching probability, I have long since lost track of all those authors and books which
have contributed to the treatment of probability in this work. I am aware of those contributions and am