Probability Theory and Mathematical Statistics

360 Pages·2020·5.954 MB·English

by  Аканбай Н.

МINISTRY OF EDUCATION
AND SCIENCE OF THE REPUBLIC OF KAZAKHSTAN

AL-FARABI KAZAKH NATIONAL UNIVERSITY

Nursadyk Аkanbay

PROBABILITY THEORY
AND MATHEMATICAL
STATISTICS

I

Textbook

Translated from Russian by
L.M. Kovaleva and Z.I. Suleimenova

Almaty
«Qazaq University»
2020
UDC 519.2(075.8)
LBC 22.172я73
А 37

Recommended for publication
by the decision of the meeting of the educational-methodical association
of the Republican educational-methodological Council of the Ministry
of Education and Science of the Republic of Kazakhstan
on the basis of Al-Farabi Kazakh National University
in the specialties of higher and postgraduate education in the groups
«Natural Sciences», «Humanities», «Social Sciences, Economics and Business»,
«Technical sciences and technology», «Creativity»
(Minutes No.2 dated November 14, 2017);
Scientific Council of Mechanics and Mathematics
Faculty and RISO KazNU al-farabi
(Minutes No.5 dated June 27, 2019)

Rewievers:
Doctor of Physical and Mathematical Sciences, Professor B.E. Kanguzhin
Doctor of Physical and Mathematical Sciences, Professor М.N. Kalimoldaev
Doctor of Physical and Mathematical Sciences, Professor B.D. Koshanov

Translated from Russian by
I.M. Kovaleva and Z.I. Suleimenova

Akanbay N.
А 37 Probability Theory and Mathematical Statistics: in 2 parts. Part І: textbook /
N. Akanbay. – Almaty: Qazaq University, 2020. – 359 p.
ISBN 978-601-04-4563-5

In the textbook, the mathematical foundations of the probability theory are presented on
the basis of Kolmogorov's axiomatics. In the first chapter, materials on random events and their
probabilities are presented in the framework of a discrete probability space. The second chapter
is devoted to the general probability space. In the third chapter, a random variable is defined as
a measurable function. The concept of mathematical expectation considered in the fourth
chapter is introduced as a Lebesgue integral in a probability measure on a general probability
space, while the readers are not required to have any information about the Lebesgue integral.
The fifth chapter is devoted to limit theorems in the Bernoulli scheme. The last sixth and
seventh chapters contain materials on various types of convergence of sequences of random
variables and on the laws of large numbers.
The textbook is recommended for undergraduate, graduate and doctoral PhD students
studying in mathematical specialties and specialties related to mathematics.

UDC 519.2(075.8)
LBC 22.172я73

ISBN 978-601-04-4563-5 © Аkanbay N., 2020
© Al-Farabi KazNU, 2020
Глава

CONTENT

FOREWORD .............................................................................................................. 9

Chapter І. RANDOM EVENTS AND THEIR PROBABILITIES ........................ 15
§1. Sample space. Classical definition of probability.
Simplest probability properties ................................................................................. 15
1.1. Discrete probability space ...................................................................................... 16
1.1.1. Classical definition of probability ...................................................................... 17
1.1.2. Events. Operations on events .............................................................................. 19
1.2. Elements of combinatorics .................................................................................... 27
1.3. Distribution of balls into boxes.............................................................................. 35
1.4. Tasks for independent work ................................................................................... 42

§2. Some classical models and distributions ............................................................. 45
2.1. The Bernoulli scheme. Binomial distribution ........................................................ 45
2.2. Polynomial scheme. The polynomial distribution ................................................. 47
2.3. Hypergeometric and multidimensional hypergeometric distributions .................. 49
2.4. Tasks for independent work ................................................................................... 53

§3. Geometric probabilities ........................................................................................ 54
3.1. Tasks for independent work ................................................................................... 56

Chapter II. PROBABILITY SPACE ........................................................................ 59
§1. Axioms of probability theory. General probability space ................................. 59
1.1. The necessity of expanding the concept of space elementary events .................... 59
1.2. The probability on a measurable space .................................................................. 63
1.2.1. Probability properties .......................................................................................... 63
1.3. Tasks for independent work ................................................................................... 73

§2. Algebras, sigma-algebras and measurable spaces ............................................. 74
2.1. Algebras and sigma-algebras ................................................................................. 74
2.1.1.The theorem on the continuation of probability .................................................. 76
2.2. The most important examples of measurable spaces ............................................. 77
2.2.1. Measurable space (R, β(R)) ................................................................................. 77
2.2.2. Measurable space (Rn, β(Rn)) .............................................................................. 78
2.2.3. Measurable space (R∞, β(R∞)) ............................................................................. 81
2.3. Tasks for independent work ................................................................................... 82

§ 3. Methods for specifying probabilistic measures on measurable spaces ........... 85
3.1. Space (R, β(R)). Distribution function ................................................................... 85
5
3.2. Space (Rn, β(Rn)). Multidimensional distribution function .................................... 92
3.3. Space (R∞, β(R∞)) ................................................................................................... 97
3.4. Tasks for independent work .................................................................................. 98

§4. Conditional probability. Independence .............................................................. 100
4.1. Conditional probability. The formula for multiplying probabilities ...................... 100
4.2. Independence ......................................................................................................... 105
4.2.1. Independence of events ....................................................................................... 105
4.2.2. Independence of partitions and algebras. Independent trials.
Independence of σ-algebras .......................................................................................... 112
4.3. Total probability and Bayes formulas .................................................................... 120
4.3.1. Total probability formula .................................................................................... 121
4.3.2. The Bayes formulas ............................................................................................ 128
4.4. Tasks for independent work ................................................................................... 130

Chapter III. RANDOM VALUES ............................................................................. 135
§1. Random values and their distributions............................................................... 135
1.1. Discrete random variables ..................................................................................... 139
1.2. Absolutely continuous random variables ............................................................... 142
1.3. Equivalent definitions of a random variable .......................................................... 147
1.4. Functions of (one) random variable ....................................................................... 148
1.4.1. Distributions of functions of a random variable ................................................. 148
1.4.2. Structures of measurable functions ..................................................................... 150
1.5. The class of extended random variables and the closure of this class
with respect to pointwise convergence ......................................................................... 152
1.6. Tasks for independent work ................................................................................... 153

§2. Multidimensional random variables ................................................................... 156
2.1. Multidimensional random variables and their distributions.
Marginal distributions ................................................................................................... 156
2.1.1. Multidimensional discrete and absolutely continuous random variables ........... 158
2.2. Independence of random variables ........................................................................ 167
2.3. Functions of random variables ............................................................................... 171
2.3.1. Distributions of sums, relations, and products of random variables ................... 172
2.3.2. Linear transformation of random variables ........................................................ 176
2.4. Conditional distributions ....................................................................................... 186
2.5. Tasks for independent work ................................................................................... 190

Chapter ІV. MATHEMATICAL EXPECTATION ................................................ 197
§1. General definition of mathematical expectation. Properties ............................ 197
1.1. The multiplicative property ................................................................................... 205
1.2. Properties «almost sure» ....................................................................................... 206
1.3. Convergence properties ........................................................................................ 209
1.4. Formulas for computing expectation ..................................................................... 212
1.4.1. Fubini theorem and some of its applications ...................................................... 216
1.5. Variance ................................................................................................................. 219
1.6. Inequalities that are related to mathematical expectation ...................................... 224
1.7. Mathematical expectation and variance: examples of calculation ........................ 229
1.8. Tasks for independent work ................................................................................... 240

§2. Conditional probabilities and conditional mathematical expectations
with respect to partitions and sigma algebras .......................................................... 247
6
2.1. Conditional probabilities and conditional mathematical expectations
with respect to partitions ............................................................................................... 250
2.1.1. The conditional mathematical expectation of one simple random
variable relative to another simple random variable ..................................................... 252
2.2. Conditional probabilities and conditional mathematical expectations
with respect to sigma algebras ...................................................................................... 256
2.2.1. Existence of conditional expectation with respect to the σ-algebra.................... 258
2.2.2. Consistency of the definition of the conditional mathematical expectation
with respect to a partition with definition of conditional mathematical
expectation with respect to σ-algebras.......................................................................... 260
2.2.3. Properties of conditional expectation ................................................................. 261
2.3. The structure of the conditional mathematical expectation
of one random variable relative to the other ................................................................ 268
2.3.1. Properties and formulas for calculating the conditional
expectation M(ξ/η = y) .................................................................................................. 269
2.4. The conditional mathematical expectation and the optimal
(in the mean-square sense) estimator ............................................................................ 273
2.5. Tasks for independent work .................................................................................. 276

Chapter V. LIMIT THEOREMS IN THE BERNULLI SCHEME
§1. Laws of large numbers ......................................................................................... 281

§2. Limit theorems of Moivre-Laplace ..................................................................... 285
2.1. The local Moivre-Laplace theorem ....................................................................... 285
2.2. The Moivre-Laplace integral theorem ................................................................... 290
2.2.1. Deviation of the relative frequency from the constant probability
in independent tests ....................................................................................................... 294
2.2.2. Finding the probability of the number of successes in the given interval .......... 295

§3. Poisson's theorem .................................................................................................. 299

§4. Tasks for independent work .................................................................................... 301

Chapter VI. DIFFERENT TYPES OF CONVERGENCE
OF SEQUENCES OF RANDOM VARIABLES ..................................................... 303
§1. Different types of convergence of sequences of random variables
and their connection ................................................................................................... 303

§2. Weak convergence ................................................................................................ 311

§3. The Cauchy criterion for convergences in probability
and with probability 1 ................................................................................................ 315

§4. Tasks for independent work .................................................................................... 319

Chapter VII. THE LAWS OF LARGE NUMBERS .............................................. 323
§1. The weak law of large numbers ........................................................................... 323
1.1. Necessary and sufficient condition for the law of large numbers ......................... 326

§2. The strong law of large numbers ......................................................................... 330

§3. Tasks for independent work .................................................................................... 341
7
ANSWERS TO THE TASKS FOR INDEPENDENT WORK............................... 343

Appendix 1. Table of values of function(cid:77)(cid:11)x(cid:12)(cid:32) 1 e(cid:16)x2/2 ...................................... 353

2(cid:83)
Appendix 2. Table of values of function (cid:41) (cid:11)x(cid:12)(cid:32) 1 x(cid:179) e(cid:16)z2/2dz ........................... 354
0
2(cid:83)0
(cid:79)k
Appendix 3. Values of Poisson distribution (cid:83) ((cid:79))(cid:32)e(cid:16)(cid:79) .................................... 355
k k!

BIBLIOGRAPHY ...................................................................................................... 357

8
Глава

FOREWORD

«Theory of probability and Mathematical Statistics» is included in the cycle of
mandatory fundamental disciplines of mathematical specialties of universities. In
addition, this subject required, is studied in a number of other specialties (mathematical
and computer modeling, mechanics, computer science, physics, economics, actuarial
mathematics, etc.) of universities.
The proposed textbook was written by the author on the basis of his many years
of experience in teaching various (simplified, ordinary, complicated, etc.) courses of
this discipline to students, undergraduates and doctoral students of the PhD specialty
«Mathematics» of the Mechanics and Mathematics Department of Al-Farabi Kazakh
National University. The content of the textbook is fully consistent with the standard
curriculum of the discipline «Theory of probability and Mathematical Statistics» and
consist of even parts.
In accordance with tradition, the first chapter is devoted to elementary proba-
bility theory. The presentation begins with the construction of probabilistic models
with a finite number of outcomes and the introduction of basic probabilistic concepts,
such as elementary events, events, operations on events, event, probability etc. Then
the concepts of a discrete space of elementary events and a discrete probability space
are introduced. Further, special attention was paid to the classical definition of
probability: on the basis of this definition, a number of simple but important properties
of probability were proved; the concept of conditional probability is defined. Here are
also given: necessary information from combinatorics; sampling models with and
without return are considered; models for placing balls in boxes (statistics by Maxwell
– Boltzmann, Bose – Einstein, Fermi – Dirac); Bernoulli scheme; binomial and
polynomial models; hypergeometric and multidimensional hypergeometric distri-
butions.
At send of the chapter with a geometric definition of probability and provides
solutions to two classical examples (the meeting problem, the Buffon problem).
At the beginning of Chapter II, the rationale for the need to expand the concept
of probability is given, after the concepts of a measurable set (random event), a
measurable space are introduced and the axiomatics of probability theory of
A.N. Kolmogorov. The properties of probability are proved on the basis of the axioms
of probability theory also. Here, a theorem on the equivalence of the axioms of
continuity and countable additivity is proved given, the most important examples of
9
measurable spaces, and methods for defining probability measures on measurable
spaces using the distribution function are discussed. At the ends of chapter introducted
concepts of conditional probability and independence (two or more events; partitions,
algebras, and sigma – algebras; testing), proofs total probability of formulas, Bayes,
and a number of examples with solutions and discussion (the problem of ruining a
player, the problem of choosing a change strategy verbal exam, etc.).
In § 1 of Chapter III, the concept of a random variable (as a measurable function)
is introduced and other equivalent, usually easily verifiable, definitions are given. Since
the distribution function of a random variable is a distribution function in the sense of
the definition from Chapter II, based on the results known from the theory of functions,
found a random variable can be only types – discrete, (absolutely) continuous and
singular. A sufficient number of examples of discrete and continuous random variables
are given. Section 2 of this chapter is devoted to the consideration of multidimensional
(vector) random quantities and their distributions, questions of finding marginal and
conditional distributions, and distributions of a function of random variables. Here,
special attention is also paid to the concept of independence of random variables
(criterion for the independence of discrete and continuous random variables,
independence of (Borel) functions from independent random variables, composition
formula, etc.).
In §1 of Chapter IV, the theory of mathematical expectation is described – the
Lebesgue integral over a probability measure (axiomatically introduced in Chapter II).
Note that the reader is not supposed to know any preliminary information about
Lebesgue integration. The main properties of mathematical expectation (properties:
linearity, positivity, and finiteness; multiplicative property; almost certainly properties
and convergence properties) and related to the mathematical expectation of Cauchy-
Bunyakovsky, Jensen, Lyapunov, Gelder, Minkowski, Markov, and Chebyshev
inequalities are given with full proofs. The formulas for calculating the mathematical
expectation are obtained using the variable replacement formula in the Lebesgue
integral. Note also that the Carathéodory theorem on continuation of a measure known
from measure theory is given without proof.
The next paragraph (i.e., §2) of Chapter IV is devoted to the theory of conditional
expectation with respect to sigma – algebras. Here, the concept of conditional expec-
tation with respect to an event is first introduced as a natural generalization of ordinary
(unconditional) expectation, and conditional probability is defined as the conditional
expectation of an event indicator. After that, this definition is extended to the case of a
partition and a number of basic properties are proved (including the formula for full
mathematical expectation). Moreover, it is emphasized that this conditional expecta-
tion, by definition, is a random variable, measurable relative to the smallest sigma-
algebra generated by the original partition. After that, the concept of conditional
mathematical expectation with respect to sigma – algebra, like the concept of mathe-
matical expectation, is determined axiomatically (first for non-negative, after – for any
random variables). The existence of such a conditional mathematical expectation is
proved with the help of the Radon – Nicodemus theorem known from the theory
measure (this theorem is not proved in our textbook, it is only formulated in a form
convenient for us). At the end of §2, the structure of the conditional mathematical
10