AR, MA and ARMA models • The autoregressive process of order p or AR(p)

37 Pages·2012·0.13 MB·English

AR, MA and ARMA models
(cid:15) The autoregressive process of order p or AR(p) is de(cid:12)ned
by the equation
p
X = (cid:30) X + !
(cid:0)
t j t j t
X
j=1
2
where ! (cid:24) N(0; (cid:27) )
t
(cid:15) (cid:30) = ((cid:30) ; (cid:30) ; : : : ; (cid:30) ) is the vector of model coe(cid:14)cients and
1 2 p
p is a non-negative integer.
(cid:15) The AR model establishes that a realization at time t is a
linear combination of the p previous realization plus some
noise term.
(cid:15) For p = 0, X = ! and there is no autoregression term.
t t
53
(cid:15) The lag operator is denoted by B and used to express
lagged values of the process so BX = X ,
t t(cid:0)1
2 3 d
B X = X , B X = X ,: : :, B X .
t t(cid:0)2 t t(cid:0)3 t(cid:0)d
(cid:15) If we de(cid:12)ne
p
j 2 p
(cid:8)(B) = 1 (cid:0) (cid:30) B = 1 (cid:0) (cid:30) B (cid:0) (cid:30) B (cid:0) : : : (cid:0) (cid:30) B
j 1 2 p
X
j=1
the AR(p) process is given by the equation
(cid:8)(B)X = ! ; t = 1; : : : ; n.
t t
(cid:15) (cid:8)(B) is known as the characteristic polynomial of the
process and its roots determine when the process is
stationary or not.
(cid:15) The moving average process of order q or MA(q) is
54
de(cid:12)ned as
q
X = ! + (cid:18) !
(cid:0)
t t j t j
X
j=1
(cid:15) Under this model, the observed process depends on
0
previous ! s
t
(cid:15) MA(q) can de(cid:12)ne correlated noise structure in our data
and goes beyond the traditional assumption where errors
are iid.
(cid:15) In lag operator notation, the MA(q) process is given by
q j
the equation X = (cid:2)(B)! where (cid:2)(B) = 1 + (cid:18) B .
t t j=1 j
P
(cid:15) The general autoregressive moving average process of
orders p and q or ARMA(p; q) combines both AR and
MA models into a unique representation.
55
(cid:15) The ARMA process of orders p and q is de(cid:12)ned as
p q
X = (cid:30) X + (cid:18) ! + !
(cid:0) (cid:0)
t j t j j t j t
X X
j=1 j=1
(cid:15) In lag operator notation, the ARMA(p; q) process is
given by (cid:8)(B)X = (cid:2)(B)! ; t = 1; : : : ; n
t t
(cid:15) Lets focus on the AR process and its characteristic
polynomial.
(cid:15) The characteristic polynomial can be expressed as:
p
(cid:8)(B) = (1 (cid:0) (cid:11) B)
i
Y
i=1
0
where the (cid:11) s are the reciprocal roots.
(cid:15) If (cid:12) ; (cid:12) ; : : : ; (cid:12) are such that (cid:8)((cid:12) ) = 0 (roots of the
1 2 p i
56
polynomial) then (cid:12) = 1=(cid:11) ; (cid:12) = 1=(cid:11) ; : : : ; (cid:12) = 1=(cid:11)
1 1 2 2 p p
(cid:15) Theorem: If X (cid:24) AR(p), X is a stationary process if and
t t
only if the modulus of all the roots of the characteristic
polynomial are greater than one, i.e. if jj(cid:12) jj > 1 for all
i
i = 1; 2; : : : ; p or equivalently if jj(cid:11) jj < 1; i = 1; 2; : : : p.
i
0
(cid:15) The (cid:11) s are also known as the poles of the AR process.
i
(cid:15) This theorem follows from the general linear process
theory.
(cid:15) Some of the poles or reciprocal roots can be real number
and some can be complex numbers and we need to
distinguish between the 2 cases.
57
(cid:15) For the complex case, we will use the representation
(cid:11) = r exp((cid:6)! i); i = 1; : : : ; C
i i i
so C is the total number of conjugate pairs and 2C is the
total number of complex poles.
(cid:15) r is the modulus of (cid:11) and ! its frequency.
i i i
(cid:15) The real reciprocal roots are denoted as
(cid:11) = r ; i = 1; 2; : : : R
i i
(cid:15) Example. Consider the AR(1) process X = (cid:30)X + ! .
t t(cid:0)1 t
(cid:15) In lag-operator notation this process is (1 (cid:0) (cid:30)B)X = !
t t
and the characteristic polynomial is (cid:8)(B) = (1 (cid:0) (cid:30)B).
(cid:15) If (cid:8)(B) = (1 (cid:0) (cid:30)B) = 0, the only characteristic root is
58
(cid:12) = 1=(cid:30) (assuming (cid:30) 6= 0).
(cid:15) The AR(1) process is stationary if only if j(cid:30)j < 1 or
(cid:0)1 < (cid:30) < 1.
(cid:15) The case where (cid:30) = 1 corresponds to a Random Walk
process with a zero drift, X = X + !
t t(cid:0)1 t
(cid:15) This is a non-stationary explosive process.
(cid:15) If we recursive apply the AR(1) equation, the Random
Walk process can be expressed as
X = ! + ! + ! + : : :. Then,
t t t(cid:0)1 t(cid:0)2
1 2
V ar(X ) = (cid:27) = 1.
t t=0
P
(cid:15) Example. AR(2) process X = (cid:30) X + (cid:30) X + !
t 1 t(cid:0)1 2 t(cid:0)2 t
(cid:15) The characteristic polynomial is now
59
2
(cid:8)(B) = (1 (cid:0) (cid:30) B (cid:0) (cid:30) B )
1 2
(cid:15) The solutions to (cid:8)(B) = 0 are
2 2
(cid:0)(cid:30) + (cid:30) + 4(cid:30) (cid:0)(cid:30) (cid:0) (cid:30) + 4(cid:30)
1 q 1 2 1 q 1 2
(cid:12) = ; (cid:12) =
1 2
2(cid:30) 2(cid:30)
2 2
(cid:15) The reciprocal roots are
2 2
(cid:30) + (cid:30) + 4(cid:30) (cid:30) (cid:0) (cid:30) + 4(cid:30)
1 q 1 2 1 q 1 2
(cid:11) = ; (cid:11) =
1 2
2 2
(cid:15) The AR(2) is stationary if and only if jj(cid:11) jj < 1 and
1
jj(cid:11) jj < 1
2
(cid:15) These two conditions imply that jj(cid:11) (cid:11) jj = j(cid:30) j < 1 and
1 2 2
jj(cid:11) + (cid:11) jj = j(cid:30) j < 2 which means (cid:0)1 < (cid:30) < 1 and
1 2 1 2
(cid:0)2 < (cid:30) < 2.
1
60
2
(cid:15) For (cid:11) and (cid:11) real numbers, (cid:30) + 4(cid:30) (cid:21) 0 which implies
1 2 1 2
(cid:0)1 < (cid:11) (cid:20) (cid:11) < 1 and after some algebra (cid:30) + (cid:30) < 1;
2 1 1 2
(cid:30) (cid:0) (cid:30) < 1
2 1
2
2 (cid:30)
(cid:15) In the complex case (cid:30) + 4(cid:30) < 0 or 1 > (cid:30)
1 2 (cid:0)4 2
(cid:15) If we combine all the inequalities we obtain a region
bounded by the lines (cid:30) = 1 + (cid:30) ; (cid:30) = 1 (cid:0) (cid:30) ; (cid:30) = (cid:0)1.
2 1 2 1 2
(cid:15) This is the region where the AR(2) process is stationary.
(cid:15) For an AR(p) where p (cid:21) 3, the region where the process
is stationary is quite abstract.
(cid:15) For the stationarity condition of the MA(q) process, we
need to rely on the general linear process.
(cid:15) A general linear process is a random sequence X of the
t
61
form,
1
X = a !
(cid:0)
t j t j
X
j=0
2
where ! is a white noise sequence with variance (cid:27) .
t
(cid:15) In lag operator notation, the general linear is given by the
(cid:0)1 (cid:0)1 1
j
expression X = (cid:8)(B) ! where (cid:8)(B) = a B .
t t j=0 j
P
(cid:15) Note (cid:12)rstly that by the de(cid:12)nition of the linear process,
E(X ) = 0.
t
(cid:15) Then, the covariance between X and X is
t s
1 1
E[X X ] = a a E[X X ]
t s j l t(cid:0)j s(cid:0)l
X X
j=0 l=0
62