Notes on Measure, Probability and Stochastic Processes Jo˜ao Lopes Dias

115 Pages·2017·0.55 MB·English

Notes on Measure, Probability and
Stochastic Processes
Joa˜o Lopes Dias
Departamento de Matema´tica, ISEG, Universidade de
Lisboa, Rua do Quelhas 6, 1200-781 Lisboa, Portugal
E-mail address: [e-mail]
December 22, 2017.
Contents
Chapter 1. Introduction 1
1. Classical definitions of probability 1
Chapter 2. Measure and probability 3
1. Algebras 3
2. Monotone classes 7
3. Product algebras 8
4. Measures 9
5. Examples 16
Chapter 3. Measurable functions 19
1. Definition 19
2. Simple functions 21
3. Convergence of sequences of measurable functions 22
4. Induced measure 24
5. Generation of σ-algebras 25
Chapter 4. Lebesgue integral 27
1. Definition 27
2. Properties 30
3. Examples 32
4. Convergence theorems 33
5. Fubini theorem 40
6. Weak convergence of sequences of probability measures 41
Chapter 5. Distributions 43
1. Definition 43
2. Simple examples 44
3. Distribution functions 46
4. Classification of distributions 48
5. Characteristic functions 49
6. Convergence in distribution 54
Chapter 6. Limit theorems 55
1. Independent random variables 55
2. Sums of random variables 58
3. Law of large numbers 60
4. Central limit theorem 61
iii
iv CONTENTS
Chapter 7. Conditional expectation 63
1. Radon-Nikodym theorem 63
2. Conditional expectation 67
3. Conditional probability 70
Chapter 8. Markov chains 73
1. The Markov property 73
2. Homogeneous Markov chains 77
3. Classification of states 79
4. Decomposition of chains 84
5. Stationary distributions 87
6. Limit distributions 88
Chapter 9. Martingales 91
1. The martingale strategy 91
2. General definition of a martingale 93
3. Examples 94
4. Stopping times 95
5. Stochastic processes with stopping times 96
Appendix A. Things that you should know before starting 99
1. Notions of mathematical logic 99
2. Set theory notions 101
3. Function theory notions 105
4. Greek alphabet 109
Bibliography 111
CHAPTER 1
Introduction
These are the lecture notes for the course “Probability Theory and
StochasticProcesses”oftheMasterinMathematicalFinance2016/2017
atISEG–UniversityofLisbon. Itisrequiredgoodknowledgeofcalculus
and basic probability.
1. Classical definitions of probability
Scienceisaboutobservinggivenphenomena,recordingdata,analysing
it and explaining particular features and behaviours using theoretical
models. This may be a rough description of what really means to make
science, but highlights the fact that experimentation is a crucial part
of obtaining knowledge.
Many experiments are of random nature. That is, their results are
not possible to predict, often due to the huge number of variables that
underlie the processes under scrutiny. One needs to repeat the exper-
iment and observe its different outcomes and events (sets of possible
outcomes). Our main goal is to quantify the likelihood of each event.
Thesegeneralideascanbeillustratedbytheexperimentofthrowing
a dice. We can get six possible outcomes depending on too many
different factors, so that it becomes impossible to predict the result.
Consider the event corresponding to an even number of dots, i.e. 2, 4
or 6 dots. How can we measure the probability of this event to occur
when we throw the dice once?
The way one usually thinks of probability is summarised in the
following relation:
number of favourable cases
Probability(“event”) =
number of possible cases
assuming that all cases are equally possible. This is the classical defi-
nition of probability (called the Laplace law).
Example 1.1. Tossing of a perfect coin in order to get either heads
or tails. The number of possible cases is 2. So,
Prob(“heads”) = 1/2
Prob(“heads at least once in two experiments”) = 3/4.
1
2 1. INTRODUCTION
Example 1.2.
1
Probability(“winning the Euromillions with one bet”) = (cid:39) 7×10−9.
C50C12
5 2
The above definition is far from what one could consider as a useful
definition of probability. For instance, we would like to examine also
“biased”experiments, that is, withinequally possibleoutcomes. Away
to deal with this question is defining probability by the frequency that
some event occurs when repeating the experiment many times under
the same conditions. So,
number of favourable cases in n experiments
Probability(“event”) = lim .
n→+∞ n
Example 1.3. In 2015 there was 85500 births in Portugal and
43685 were boys. So,
Probability(“it’s a boy!”) (cid:39) 0.51.
A limitation of this second definition of probability occurs if one
considers infinitely many possible outcomes.
The modern probability is based in measure theory, bringing a fun-
damental mathematical rigour and an abrangent concept (although
very abstract as we will see). This course is an introduction to this
subject.
CHAPTER 2
Measure and probability
1. Algebras
Given an experiment we consider Ω to be the set of all possible
outcomes. This is the probabilistic interpretation that we want to
associate to Ω, but in the point of view of the more general measure
theory, Ω is just any set.
The collection of all the subsets of Ω is denoted by
P(Ω) = {A: A ⊂ Ω}.
It is also called the set of the parts of Ω. When there is no ambiguity,
we will simply write P. We say that Ac = Ω\A is the complement of
A ∈ P in Ω.
As we will see later, a proper definition of the measure of a set re-
quires several properties. In some cases, that will restrict the elements
of P that are measurable. It turns out that the measurable ones just
need to verify the following conditions.
A collection A ⊂ P is an algebra of Ω iff
(1) ∅ ∈ A,
(2) If A ∈ A, then Ac ∈ A,
(3) If A ,A ∈ A, then A ∪A ∈ A.
1 2 1 2
An algebra F of Ω is called a σ-algebra of Ω iff given A ,A ,··· ∈ F
1 2
we have
+∞
(cid:91)
A ∈ F.
n
n=1
Remark 2.1. We can easily verify by induction that any finite
union of elements of an algebra is still in the algebra. What makes a
σ-algebra different is that the infinite countable union of elements is
still in the σ-algebra.
Example 2.2. Consider the set A of all the finite union of intervals
in R including R and ∅. Notice that the complementary of an interval
is a finite union of intervals. Therefore, A is an algebra. However, the
countable union of the sets A =]n,n + 1[∈ A, n ∈ N, is no longer
n
finite. That is, A is not a σ-algebra.
3
4 2. MEASURE AND PROBABILITY
Remark 2.3. Any finite algebra A (i.e. it contains only a finite
numberofsubsetsofΩ)isimmediatelyaσ-algebra. Indeed,anyinfinite
union of sets is in fact finite.
The elements of a σ-algebra F are called measurable sets. In prob-
ability theory they are also known as events. The pair (Ω,F) is called
a measurable space.
Exercise 2.4. Decide if F is a σ-algebra of Ω where:
(1) F = {∅,Ω}.
(2) F = P(Ω).
(3) F = {∅,{1,2},{3,4,5,6},Ω}, Ω = {1,2,3,4,5,6}.
(4) F = (cid:8)∅,{0},R−,R−,R+,R+,R\{0},R(cid:9), Ω = R.
0 0
Proposition 2.5. Let F ⊂ P such that it contains the comple-
mentary set of all its elements. For A ,A ,··· ∈ F,
1 2
+∞ +∞
(cid:91) (cid:92)
A ∈ F iff A ∈ F.
n n
n=1 n=1
Proof. (⇒) Using Morgan’s laws,
+∞ (cid:32)+∞ (cid:33)c
(cid:92) (cid:91)
A = Ac ∈ F
n n
n=1 n=1
because the complements are always in F.
(⇐) Same idea. (cid:3)
Therefore, the definitions of algebra and σ-algebra can be changed
to require intersections instead of unions.
Exercise 2.6. Let Ω be a finite set with #Ω = n. Compute
#P(Ω). Hint: Find a bijection between P and {v ∈ Rn: v ∈ {0,1}}.
i
Exercise 2.7. Let Ω be an infinite set, i.e. #Ω = +∞. Consider
the collection of all finite subsets of Ω:
C = {A ∈ P(Ω): #A < +∞}.
Is C an algebra? Is it a σ-algebra?
Exercise 2.8. Let (Ω,F) be a σ-algebra. Consider two disjoint
sets A,B ⊂ Ω and assume that A ∈ F. Show that A ∪ B ∈ F is
equivalent to B ∈ F?
1. ALGEBRAS 5
1.1. Generation of σ-algebras. In many situations we want to
consider some specific collection of subsets to be measurable because
of their relevance to the problem we are studying. If the collection is
not already a σ-algebra, we need to take a larger one that it is. That
is called the σ-algebra generated by the original collection.
Take I to be any set (of indices).
Theorem 2.9. If F is a σ-algebra, α ∈ I, then F = (cid:84) F is
α α∈I α
also a σ-algebra.
Proof.
(1) As for any α we have ∅ ∈ F , then ∅ ∈ F.
α
(2) Let A ∈ F. So, A ∈ F for any α. Thus, Ac ∈ F and Ac ∈ F.
α α
(cid:83)
(3) If A ∈ F, we have A ∈ F for any α. So, A ∈ F and
n n α n n α
(cid:83)
A ∈ F.
n n
(cid:3)
Consider now the collection of all σ-algebras:
Σ = {all σ-algebras of Ω}.
So, P ∈ Σ and {∅,Ω} ∈ Σ. In addition, let I ⊂ P be a collection of
subsets of Ω, i.e. I ⊂ P, not necessarily a σ-algebra. Define the subset
of Σ given by the σ-algebras that contain I,
Σ = {F ∈ Σ: I ⊂ F}.
I
The σ-algebra generated by I is the intersection of all σ-algebras
containing I,
(cid:92)
σ(I) = F.
F∈ΣI
Hence, σ(I) is the smallest σ-algebra containing I (i.e. it is contained
in all σ-algebras that contain I).
Example 2.10.
(1) Let A ⊂ Ω and I = {A}. Any σ-algebra containing I has to
include the sets ∅, Ω, A and Ac. Since these sets form already
a σ-algebra, we have
σ(I) = {∅,Ω,A,Ac}.
(2) Consider two disjoint sets A,B ⊂ Ω and I = {A,B}. The
generated σ-algebra is
σ(I) = {∅,Ω,A,B,Ac,Bc,A∪B,(A∪B)c}.
6 2. MEASURE AND PROBABILITY
(3) Consider now two different sets A,B ⊂ Ω such that A∩B (cid:54)= ∅,
and I = {A,B}. Then,
σ(I) ={∅,Ω,A,B,Ac,Bc,
A∪B,A∪Bc,Ac ∪B,(A∪B)c,(A∪Bc)c,(Ac ∪B)c,
Bc ∪(A∪Bc)c,(Bc ∪(A∪Bc)c)c,
((A∪Bc)c)∪((Ac ∪B)c),(((A∪Bc)c)∪((Ac ∪B)c))c}
={∅,Ω,A,B,Ac,Bc,
A∪B,A∪Bc,Ac ∪B,Ac ∩Bc,B \A,A\B,
(A∩B)c,A∩B,
(A∪B)\(A∩B),(Ac ∩Bc)∪(A∩B)}.
Exercise 2.11. Show that
(1) If I ⊂ I ⊂ P, then σ(I ) ⊂ σ(I ).
1 2 1 2
(2) σ(σ(I)) = σ(I) for any I ⊂ P.
Exercise 2.12. Consider a finite set Ω = {ω ,...,ω }. Prove that
1 n
I = {{ω },...,{ω }} generates P(Ω).
1 n
Exercise 2.13. Determine σ(C), where
C = {{x} ∈ P: x ∈ Ω}.
What is the smallest algebra that contains C.
1.2. Borel sets. A specially important collection of subsets of R
in applications is
I = {]−∞,x] ⊂ R: x ∈ R}.
It is not an algebra since it does not contain even the emptyset. An-
other collection could be obtained by considering complements and
intersections of pairs of sets in I. That is,
I(cid:48) = {]a,b] ⊂ R: −∞ ≤ a ≤ b ≤ +∞}.
Here we are using the following conventions
]a,+∞] =]a,+∞[ and ]a,a] = ∅
so that ∅ and R are also in the collection. The complement of ]a,b] ∈ I(cid:48)
is still not in I(cid:48), but is the union of two sets there:
]a,b]c =]−∞,a]∪]b,+∞].
So, the smallest algebra that contains I corresponds to the collection
of finite unions of sets in I(cid:48),
(cid:40) (cid:41)
N
(cid:91)
A(R) = I ⊂ R: I ,...,I ∈ I(cid:48),N ∈ N .
n 1 N
n=1
Clearly, I ⊂ I(cid:48) ⊂ A(R).