Iterative Least Squares and M-Estimation for ARMA Processes With

23 Pages·2001·0.06 MB·English

IItteerraattiivvee LLeeaasstt SSqquuaarreess aanndd MM--EEssttiimmaattiioonn
ffoorr AARRMMAA PPrroocceesssseess WWiitthh IInnffiinniittee VVaarriiaannccee
Richard A. Davis
Colorado State University
[email protected]
Santa Barbara Dec 3-4, 1995
1
(cid:1)
Something old
(cid:1)
sample acf
(cid:1)
estimation
(cid:1)
Something new
(cid:1)
M-estimation
(cid:1)
LAD-estimation
(cid:1)
linearized estimates
(cid:1)
Something blue
(cid:1)
bootstrapping
(cid:1)
MLE
Santa Barbara Dec 3-4, 1995
2
Sample ACF .
∈
(α) α
Let {Z }~ IID, symmetric, Z D , 0 < < 2 and define
t 1
-1
a = inf{x: P(|Z | > x) < n }
n 1
-1
b = inf{x: P(|Z Z | > x) < n }
n 1 2
Then,
Σn Σn Σn
( )
-2 2 -1 . . . , -1
a Z , b Z Z , b Z Z
n t n t t+1 n t t+h
t=1 t=1 t=1
d . . . ,
( S , S , S ),
0 1 h
α . . . , α
where S is stable( /2) and S , S are stable( ).
0 1 h
. . . ,
Note: S , S , S are independent if E|Z | = and are
8
0 1 h 1
dependent if E|Z | <8 .
1
Santa Barbara Dec 3-4, 1995
3
Special Case of Pareto Tails.
n n n
Σ Σ Σ
( -2/α 2 -1/α . . ., -1/α )
n Z , (n ln n) Z Z , (n ln n) Z Z
t t t+1 t t+h
t=1 t=1 t=1
d . . . , . . . ,
( S , S , S ), (S , S , S indep.)
0 1 h 0 1 h
Then,
n n n
Σ Σ Σ
( 1/α . . ., 1/α )/ 2
(n/ln n) Z Z , (n /ln n) Z Z Z
t t+1 t t+h t
t=1 t=1 t=1
d . . . ,
(S , S )/ S ,
1 h 0
Santa Barbara Dec 3-4, 1995
4
ARMA(p,q) Processes {X }.
t
. . . . . .
φ φ θ θ
X = X + + X + Z + Z + + Z ,
t 1 t-1 p t-p t 1 t-1 q t-q
α) α
{Z } ~ IID( with Pareto tails (0 < < 2)
t
Then,
8
Σ
1/α ρ^( − ρ( d ρ ρ ρ ρ
(n / ln n) ( h) h)) ( (h+j)+ (h-j)-2 (j) (h)) S / S
j 0
j=1
^
ρ( ρ(
where h) is the population ACF, and h) is the sample ACF.
Implication:
Estimation procedures which are inherently second
α
1/
order based will have scaling factor (n / ln n) . Examples are
(cid:127) moment estimation (Davis & Resnick `85, `86)
(cid:127) Yule-Walker estimation for AR's (Davis & Resnick `85, `86)
(cid:127) Whittle estimate (and max Gaussian likelihood?) (Mikosch,
Gadrich, Kluppelberg and Adler `95)
Santa Barbara Dec 3-4, 1995
5
M-Estimation
Ε
xample (AR(1)).
Data.
X , . . . , X
1 n
Model. φ
X = X + Z ,
t 0 t-1 t
φ α .
| | < 1, {Z } ~ IID( )
0 t
Loss function : ρ( ) ψ ρ
with influence function (x)= '(x)
satisfying
1. τ (α−1,0).
Lipschitz of order > max
2. Ε| ψ α 1.
(Z )| < if <
8
1
Ε ψ ψ α 1 .
3. (Z ) = 0 and Var( (Z ) ) < if >
8
1 1
Santa Barbara Dec 3-4, 1995
6
Minimize n
Σ
φ) (ρ − φ − ρ( )
T ( = (X X ) Z )
n t t-1 t
t=1
n
Σ
(ρ − (φ −φ − ρ( )
= ( Z ) X ) Z )
t 0 t-1 t
t=1
1/ α φ−φ ) φ -1/α ),
Set v= n ( ), and put S (v = T ( + vn i.e.,
0 n n 0
n
Σ
) (ρ − -1/α − ρ( )
S (v = ( Z vn X ) Z )
n t t-1 t
t=1
Santa Barbara Dec 3-4, 1995
7
n
Σ
Properties of ) = (ρ − -1/α − ρ( ):
S (v ( Z vn X ) Z )
n t t-1 t
t=1
(cid:1) )
S (v has convex sample paths.
n
(cid:1) ) d ) ,
S (v S (v (finite dimensional distribution
n
)
convergence, where S(v is a random process).
(cid:1) d
S S on C(R).
n
Result.
(Davis, Knight and Liu `92)
^
v := argmin(S (v))
n n
1/ α φ^−φ d ^
= n ( ) v:= argmin(S (v))
0
Santa Barbara Dec 3-4, 1995
8
Ε
xample (MA(1)).
Data.
X , . . . , X
1 n
Model. θ
X = Z + Z ,
t t t-1
θ α)
| | < 1, {Z } ~ IID(
0 t
LAD estimation:
Minimize
n
Σ
θ) ( θ θ )
T ( = |Z ( )| - |Z ( )|
n t t 0
t=1
n
Σ
. . .
(| − θ θ2 θ t-1 θ )
= X X + X - (- ) X | - |Z ( )|
t t-1 t-2 1 t 0
t=1
1/α θ−θ
Set u=n ( ),
0
Santa Barbara Dec 3-4, 1995
9
) θ -1/α
S (u = T ( + un )
n n 0
n
Σ
( θ -1/α θ )
= |Z ( + un )| - |Z ( )|
t 0 t 0
t=1
(Not a convex function of u!)
θ -1/α
Linearize Z ( + un ) to get
t 0
n
Σ
) ∼ ( θ -1/α ' θ θ )
S (u |Z ( ) + un Z ( ) | - |Z ( )|
n t 0 t 0 t 0
t=1
' θ
where - Z ( ) is the AR(1) process
t 0
−θ
Y = Y + Z .
t 0 t-1 t
Result :
Same limit result as in the AR(1) case, i.e.
^
n1/α(θ − θ ) d u^ = argmin S(u).
LAD 0
Santa Barbara Dec 3-4, 1995
10