Marcinkiewicz Law of Large Numbers for Outer-products of Heavy-tailed, Long-range Dependent Data

0.27 MB·English

by  Michael A. Kouritzin

Marcinkiewicz Law of Large Numbers
for Outer-products of Heavy-tailed,
5 Long-range Dependent Data
1
0
2 Michael A. Kouritzin and Samira Sadeghi∗

n Department of Mathematical and Statistical Sciences
a Universityof Alberta, Edmonton, ABT6G 2G1 Canada
J
n
1 1
1 Abstract: The Marcinkiewicz Strong Law, nl→im∞np1 kX=1(Dk−D) = 0
T
] a.s. with p ∈ (1,2), is studied for outer products Dk = XkXk, where
T {Xk},{Xk}arebothtwo-sided(multivariate)linearprocesses(withcoeffi-
S cientmatrices(Cl),(Cl)andi.i.d.zero-meaninnovations{Ξ},{Ξ}).Matrix
sequences Cl andCl candecayslowlyenough(as|l|→∞)that{Xk,Xk}
h. havelong-rangedependencewhile{Dk}canhaveheavytails.Inparticular,
t theheavy-tailandlong-range-dependencephenomenafor{Dk}arehandled
a simultaneouslyandanewdecouplingpropertyisprovedthatshowsthecon-
m vergencerateisdeterminedbytheworstoftheheavy-tailsorthelong-range
dependence,butnotthecombination.Themainresultisappliedtoobtain
[
MarcinkiewiczStrongLawofLargeNumbersforstochasticapproximation,
1 non-linearfunctions formsandautocovariances.
v
Primary62J10, 62J12,60F15;secondary62L20.
5 Keywordsandphrases:covariance,linearprocess,Marcinkiewiczstrong
1 law of large numbers, heavy tails, long-range dependence, stochastic ap-
4 proximation.
2
0
1. 1. Intoduction
0
5 Let D = X XT be random matrices with {X }, {X } being Rd-valued (pos-
k k k k k
1 sibly two-sided, multivariate) linear processes
:
v
∞ ∞
i
X Xk = Ck−lΞl, Xk = Ck−lΞl. (1)
r l=X−∞ l=X−∞
a
defined on some probability space (Ω,F,P).
Ξ =(ξ(1),...,ξ(m)),Ξ =(ξ(1),...,ξ(m)) , l∈Z
l l l l l l
n(cid:16) (cid:17) o
are i.i.d. zero-mean random Rm+m-vectors (innovations) such that E[|Ξ |2] <
1
∞, E[|Ξ1|2] < ∞ and (Cl)l∈Z, (Cl)l∈Z are Rd×m-matrix sequences satisfying
sup|l|σkCk < ∞, sup|l|σkC k < ∞ for some (σ,σ) ∈ 1,1 . Hence, {D } can
l l 2 k
l∈Z l∈Z
have heavy tails as well as long-range dependence. (cid:0) (cid:3)
∗Correspondingauthor.E-mail:[email protected]
1
M.A. Kouritzin and S. Sadeghi/MSLLN for outer-products of LinearModels 2
Linearprocessmodels areheavilyusedinfinance,engineering,econometrics,
and statistics. In fact, classical time-series theory mainly involves the statis-
tical analysis of stationary linear processes. Current applications in network
theory and financial mathematics leads us to study time series models where
{D } can have heavy tails and long memory. Heavy-tailed data exhibits fre-
k
quent extremes and infinite variance, while positively-correlated long memory
data displays great serial momentum or inertia. Heavy-tailed data with long-
rangedependencehasbeenobservedinaplethoraofempiricaldatasetoverthe
lastfiftyyearsandso.Forinstance,Mandelbrot[11]observedthatlongmemory
time series often were heavy-tailed and self-similar.
The possible rates of the convergence is affected by both long-range depen-
dence and heavy-tailed. There are two broad types of dependence for linear
processes.Ifthe coefficients(C )areabsolutelysummableandinnovationshave
l
secondmoments,thenthecovariancesofX aresummableandwesaythat{X }
k k
isshort-rangedependence(SRD).Onthecontrary,wegenericallysaythat{X }
k
islong-rangedependence(LRD)ifitscovariancesarenotabsolutelysummable.
Practically, by choosing appropriate coefficients, matrix sequence (C ) can de-
l
cay slowly enough (as |l|→∞) such that {X } shows LRD. We consider {D }
k k
to have LRD too in this {C } non-summable case even though the second mo-
l
ments for D may not exist. There are also two generalkinds of randomness.If
k
each D fails to have a second moment, then we say it has heavy-tailed (HT)
k
and is otherwise light-tailed(LT). In our setting, D will either have HT or LT
k
depending upon the moments of and dependence between Ξ and Ξ .
1 1
There few general Marcinkiewicz Strong Law of Large Numbers (MSLLN)
results for partial sums of X under both heavy-tailed and the long-range de-
k
pendenceandtheMSLLNforpartialsumsofnonlinearfunctionsofX isalmost
k
untouched. Our purpose here is to establish a method and a structure under
which certain MSLLN for heavy-tailed and the long-range-dependent phenom-
ena can be handled properly. Technically, our goal is to prove:
n
1 1
lim (D −D)=0 a.s. for p< ∧α∧2,
n→∞np1 k 2−σ−σ
k=1
X
when max suptαP(|ξ(i)ξ(j)|>t)<∞ for some α>1 and sup|l|σkC k<∞,
1 1 l
1≤i,j≤mt≥0 l∈Z
sup|l|σkC k < ∞ when (σ,σ) ∈ 1,1 . This format of {D } is critical for our
l 2 k
l∈Z
result since, it allows LRD and H(cid:0)T c(cid:3)onditions decouple and convergence rate
be determined by the worst of the HT requirement p < (α∧2) and the LRD
conditionp< 1 ,butnotthecombination.Abifurcationhappens.Consider
2−σ−σ
∞
the summation, D = C ΞC Ξ , broken into off-diagonal and
k k−l l k−m m
l,m=−∞
X
diagonal terms. Due to the independence of (Ξ ,Ξ ) from (Ξ ,Ξ ), the off-
l l m m
diagonal sum C C Ξ Ξ does not have heavy tails ( when α > 1 ).
k−l k−m l m
l6=m
P
M.A. Kouritzin and S. Sadeghi/MSLLN for outer-products of LinearModels 3
∞
Conversely, since σ + σ > 1 the diagonal sum C C Ξ Ξ does not
k−l k−l l l
l=−∞
experiencelong-rangedependence.Inaddition,thePrateofconvergencedepends
on the worst of (α∧2) and 1 , so whenever we are in the LRD dominant
2−σ−σ
case, (α > 1 ), the off-diagonal terms dictate the rate of convergence by
2−σ−σ
the LRD effect (p < 1 ) and in the HT dominant case, (α < 1 ),
2−σ−σ 2−σ−σ
the diagonal terms dictate the rate of convergence by HT effect (p < α). The
bifurcation point is when α= 1 and α<2.
2−σ−σ
2. Background
In this section we give a review of some existing literature on MSLLN or weak
convergencefor partialsums, sample covarianceand non-linearfunction ofpar-
tialsumswithheavy-tailedand/orlong-rangedependence.Manyexistingresults
were only established in the scalar case. For ease of assimilation we use {x },
k
(c ), {d } and {ξ } to denote these scalar versions of {X }, (C ), {D } and
l k k k l k
{Ξ } and {x } for {X } when it is a shifted version of {x }.
k k+h k k
2.1. Partial Sums
There are many of publications that consider almost sure rates of convergence
for linear processes having either LRD or HT. However, there are only a few
like Louhchiand Soulier [10] thatconsideredthe combinationof these two phe-
nomena. They stated the following result for linear symmetric α-stable (SαS)
processes.
Theorem 1 Let {ξj}j∈Z be i.i.d. sequence of SαS random variables with 1 <
α < 2 and {cj}j∈Z be a bounded collection such that |cj|s < ∞ for some
j∈Z
s∈[1,α). Set x = c ξ . Then, for p∈(1,2) satisPfying 1 >1− 1 + 1
k k−j j p s α
j∈Z
P
n
1
x →0 a.s.
1 i
np
i=1
X
Theconditions<αensures |c |α <∞andtherebyconvergenceof c ξ .
j k−j j
j∈Z j∈Z
Moreover,{x } not only exhPibits heavytails but also long-rangedepePndence if,
k
for example, c = |j|−σ for j 6= 0 and some σ ∈ 1,1 . Notice there is interac-
j 2
tions betweenthe heavy tailconditionandthe long rangedependent condition.
(cid:0) (cid:1)
In particular for a given p, heavier tails (α becomes smaller) implies that you
cannothaveaslongrangedependence (s mustbecomesmaller)andvice versa.
Moreover,thisresultisdifficultorevenimpossibletoapplyinourouterproduct
setting due to the fact that x ’s are linear processes with SαS innovations and
k
sox cannotbedecomposedtoproductoftwovariableseveninthescalarcase.
k
M.A. Kouritzin and S. Sadeghi/MSLLN for outer-products of LinearModels 4
2.2. Non-linear function of partial sums
The limit behavior of suitably normalized partial sums of stationary random
variables that demonstrate either LRD or HT has been subject of study by
many authors. Applications can be found in geophysics, economics, hydrology
andstatistics.Forinstance,incontextslike Whittle approximation,the asymp-
totic behavior of quadratic forms of stationary sequences have an important
role. In addition, the efficacy of “R/S-statistic”theory that was introduced for
estimating the long-run, non-periodic statistical dependence of time series by
Hurst and developed by Mandelbrot [12], can be confirmed by convergence of
these limit functions.
There are many results that deal with the existence and description of limit
distributions of sums
[nt]
S (t)= (h(x )−E(h(x ))), t≥0, (2)
n,h k k
k=1
X
wherehisa(nonlinear)function.ThelimitbehaviorforaGaussianLRDprocess
{x }, firstly was studied by Rosenblatt [14]. Afterward, Dobrushin and Major
k
[4] explained it in more generalform. Then Taqqu[18] showedthat the limit in
distributionofparticularnormalizedsumsS (t)isdeterminedbytheHermite
n,h
rank m∗ ∈ {1,2,...} of h(x), which is the index of the first nonzero coefficient
in the Hermite expansion. On the other hand, the behavior of nonlinear non-
GaussianLRDprocessesismuchlesscommonlyknown.Oneofthemoststudied
modelsofnon-GaussianLRDprocessesisthe one-sidedlinear(movingaverage)
process,
∞
x = c ξ , (3)
k j k−j
j=0
X
in which, innovations ξ ,k ∈ Z, are independent and identically distributed
k
(i.i.d.), have zero mean with finite variance, and coefficients c satisfy:
j
c ∼c j−σ, j ≥1 (4)
j σ
for some constant c 6=0, c =1 and σ ∈(1,1).
σ 0 2
Surgailis [16] considered the limit behavior of partial sum processes S (t)
n,h
of polynomial h of linear process {xk}k∈Z. Later, Giraitis and Surgailis [5][6],
Avram and Taqqu [1] noticed that the only difference between this case and
Gaussian case is that the Hermite rank m∗ of h(x) has to be replaced by the
Appell rank m.
Vaiciulis [19] investigated distributional convergence for normalized partial
sums of Appell polynomials A (x ) of linear processes x having both long-
m k k
memory and heavy-tails in the sense EA2 (x )=∞. In particular, he assumed
m k
x hadtheform(3)withinnovations{ξm}belongingtothedomainofattraction
k k
M.A. Kouritzin and S. Sadeghi/MSLLN for outer-products of LinearModels 5
of an α-stable law with 1 < α < 2 and c following (4). The limit was: i)
j
an α-stable Levy process, ii) an mth order Hermite process, or iii) the sum
of two mutually independent α-stable Levy and mth order Hermite processes,
depending on the value of α,m and σ where σ ∈(1,1).
2
Thereafter, Surgailis [17] considered the bounded, infinitely differentiable h
case where {x } was LRD and had innovations with probability tail decay of
k
x−2α for 1 < α < 2. Suppose x satisfies (3) and (4). Then he showed three
k
different limiting behaviors corresponding to three different LRD-HT setting:
n1−(2σ−1)m∗/2Sn,h(t), n2α1σSn,h(t) or n21Sn,h(t) converge in distribution to re-
spectivelya Hermiteprocessoforderm∗,a2ασ-stable LevyprocessoraBrow-
nian motion, all at time t, for certain range of α and σ.
2.3. Sample Covariances
Auto-covariance functions play a substantial role in time series analysis and
have diverse applications in inference problems, including hypothesis testing
and parameter estimation. The natural estimator of auto-covariance is sample
covariance. Hence, the convergence properties of the sample covariance is of
great interest. In the case of LRD and HT, it is an area of active research.
Davis and Resnick [3]studied the distributional convergenceof sample auto-
covariances for two-sided linear processes with innovations that were i.i.d. and
had regularly varying tail probabilities of index α>0.
P(|ξ |>x)=x−2αL(x),
k
P(ξ >x) P(ξ <−x)
k k
→p and →q, as x→∞, (5)
P(|ξ |>x) P(|ξ |>x)
k k
L(aj)
where L(.) is a function slowly varying at infinity lim =1 and 0 ≤
j→∞ L(j)
(cid:18) (cid:19)
p ≤ 1, q = 1−p. They considered the case where the innovations had finite
variance(ι)butinfinitefourthmoment,i.e.1<α<2withabsolutelysummable
coefficients c with form of (4).
j
Note:Wechoosetoscaleourconstants,hereandinthesequel,sothatα<2
always mean HT of the object of interest, which is x x or more generally
k k+h
X X .
k k
In case of infinite fourth moment for {ξk}k∈Z, the asymptotic distribution
ofnormalizedsample autocovariancesoflong-memoryprocesseswasstudiedby
Horva´thandKokoszka[7].Supposeweobservetherealizationx ,x ,...,x , n>
1 2 n+v
1,v≥0,thesampleautocovariancesandpopulationautocovariancesaredefined
as
n
1
γˆ(n) = x x , h=0,1,...,v, and
h n k k+h
k=1
X
∞
γ =E[x x ]=ι c c , (6)
h 0 h j j+h
j=0
X
M.A. Kouritzin and S. Sadeghi/MSLLN for outer-products of LinearModels 6
respectively. Horva´th and Kokoszka [7, Theorem 3.1] studied the asymptotic
distribution [γˆ(n) −γ ], h = 0,1,...,v for linear process of form (3) with co-
h h
efficients and innovations satisfying (4) and (5) and a norming constant a =
n
inf{x:P(|ξ1|>x)≤n−1} (roughly of order n21α) satisfying
lim nP[|ξ |>a x]=x−2α, x>0. (7)
k n
n→∞
We quote this result in our notations as the following theorem.
Theorem 2 Suppose, conditions (3), (4), (5) and (7) hold.
(a) If 1− 1 <σ <1 and 1<α<2, then
2α
∞
na−2[γˆ(n)−γ ]→d S− α c c , h=0,1,...,H.
n h h α−1  j j+h
(cid:16) (cid:17) Xj=0
 
where S is an α-stablerandom variable. For theabove tohold for σ =3/4,
we must additionally assume that a−4nlnn→0.
n
(b) If 1 <σ <1− 1 and 1<α<2, then
2 2α
n2σ−1[γˆ(n)−γ ]→d ιc2 [U (1)], h=0.1,...,H.
h h σ σ
where U is a Rosenblatt process. The Rosenblatt process is often defined
σ
by the iterated stochastic integral:
U (t)=2 t(τ −w )−σ(τ −w )−σdτ W(dw )W(dw ),
σ w1<w2<t 0 1 + 2 + 1 2
in which WR(.) is thehRstandard Wiener process oni the real line.
This theorem works for one-sided linear processes with a regularly varying
tail condition and gives us weak convergence.
NoticethatinTheorem2,case(a)representstheHTdominant,(α< 1 ),
2−2σ
sothe diagonaltermsdictate convergenceto anα-stable distribution.However,
case (b) represents the LRD dominant, (α > 1 ), hence off-diagonal terms
2−2σ
take over and we get convergence to Rosenblatt process.
3. Main results
Ourfirstresultisinthe scalarcase.Later,wewillextractthe fullvector-valued
result as a second main theorem. All proofs are delayed until the next section
after we have discussed the applications.
Theorem 3 Let (ξ ,ξ ) be i.i.d. zero-mean random variables such that
l l l∈Z
E[ξ12]<∞, E[ξ21]<(cid:8) ∞ an(cid:9)d suptαP(|ξ1ξ1|>t)<∞ for some α>1. Moreover,
t≥0
suppose (cl)l∈Z,(cl)l∈Z satisfy
1
sup|l|σ|c |<∞, sup|l|σ|c |<∞ for some σ,σ ∈ ,1 ,
l l
l∈Z l∈Z (cid:18)2 (cid:21)
M.A. Kouritzin and S. Sadeghi/MSLLN for outer-products of LinearModels 7
∞ ∞
d = c c ξ ξ and d = E[ξ ξ ] c c . Then, for p satisfying
k k−l k−m l m 1 1 l l
l,m=−∞ l=−∞
p< 1P∧α∧2 P
2−σ−σ
n
1
lim (d −d)=0 a.s.
n→∞np1 k
k=1
X
Remark 1 The tail probability bound ensures that E[|ξ ξ |r] < ∞ for any
1 1
r ∈ (1,(α∧2)) and E[d ] exists but it is possible that E[d2] = ∞ so we are
1 1
handling heavy tails for {d }. On the other hand, E[|ξ ξ |α] < ∞ implies our
k 1 1
tail condition by Markov’s inequality. σ, σ bound the amount of long-range de-
∞ ∞
pendence in x = c ξ , x = c ξ . If σ can be taken larger than
k k−l l k k−l l
l=−∞ l=−∞
∞ P P
1, then E[x x ]<∞ and there is no long-range dependence in {x }. σ > 1
0 k k 2
k=1
P ∞
with E[ξ2]<∞ ensures that c ξ converges a.s.
1 k−l l
l=−∞
P
Remark 2 Notice that the constraints to handle long-range dependence, p <
1 , and to handle the heavy tails, p < (α∧2), decouple. This decoupling
2−σ−σ
appears to be due to the structureof d . Dueto the independence of (ξ ,ξ ) from
k l l
(ξ ,ξ ),theoff-diagonalsum c c ξ ξ doesnothaveheavytails.Con-
m m k−l k−m l m
l6=m
P ∞
versely, since σ+σ >1 the diagonal sum c c ξ ξ does not experience
k−l k−l l l
l=−∞
long-range dependence. P
We will give a simple example to verify conditions in Theorem 3. Recall, a
non-negative random variable ξ obeys a power law with parameters β >1 and
x >0, written ξ ∼PL(x ,β), if it has density
min min
β−1 x
f(x)= ( )−β ∀ x≥x
min
x x
min min
xr ( β−1 ) r <β−1
so E|ξ|r = min β−1−r .
∞ r ≥β−1
(cid:26)
It has a folded t distribution with parameterβ >1,written ξ ∼Ft(β), if it has
density
2Γ(β) x2 −β2
f(x)= 2 1+ ∀ x>0
Γ(β−1) (β−1)π (β−1)
2 (cid:18) (cid:19)
so E(|ξ|r) exists if and onply if r <β−1.
Example 1 Suppose p,q,α,β,β > 1 are such that 1 + 1 = 1, β > pα+1,
p q
β >qα+1andpα,qα≥2.Ifξ andξ havepowerlawdistribution,letssayξ ∼
1 1 1
Pl(x ,β), ξ ∼ Pl(x ,β) for some x ,x > 0, then E[ξ2], E[ξ2] < ∞
min 1 min min min 1 1
andsuptαP(|ξ ξ |>t)<∞.Ifξ ∼Ft(β),ξ ∼Ft(β),thenE[ξ2], E[ξ2]<∞
1 1 1 1 1 1
t≥0
M.A. Kouritzin and S. Sadeghi/MSLLN for outer-products of LinearModels 8
and suptαP(|ξ ξ | > t) < ∞. Either way, the Theorem 3 applies with properly
1 1
t≥0
chosen (c ,c ).
l l
We now consider the case where X and X are (multivariate) linear pro-
k k
cesses.
Theorem 4 Let {Ξ } and Ξ be i.i.d. zero-mean random Rm-vectors such
l l
that Ξl = ξl(1),...,ξl(m) , Ξl(cid:8)= (cid:9)ξ(l1),...,ξ(lm) , max suptαP(|ξ1(i)ξ(1j)|>t)<
1≤i,j≤mt≥0
(cid:16) (cid:17) (cid:16) (cid:17)
∞ for some α>1, E[|Ξ |2]<∞ and E[|Ξ |2]<∞. Moreover, suppose matrix
1 1
sequences (Cl)l∈Z,(Cl)l∈Z ∈Rd×m satisfy
1
sup|l|σkCk<∞, sup|l|σkC k<∞ for some (σ,σ)∈ ,1 ,
l l
l∈Z l∈Z (cid:18)2 (cid:21)
T T
X , X take form of (1), D =X X and D =E[X X ]. Then, for p satisfy-
k k k k k 1 1
ing p< 1 ∧α∧2
2−σ−σ
n
1
lim (D −D)=0 a.s.
n→∞np1 k
k=1
X
This theorem follows by linearity of limits and Theorem 3.
3.1. Applications
We give some applications of our theorems.
3.1.1. Stochastic Approximation
Stochasticapproximation(SA)isoftenusedinoptimizationproblemsforlinear
models. Hence, the convergence properties of SA algorithms driven by linear
models is of utmost interest. For illustration, we assume {z ,k = 1,2,..} and
k
{y ,k = 2,3,...} are respectively Rd− and R−valued stochastic processes, de-
k
fined on some probability space (Ω,F,P), that satisfy
y =zTh+ǫ , ∀k =1,2,..., (8)
k+1 k k
wherehisanunknownd-dimensionalparameterorweightvectorofinterestand
{ǫ } is a noise sequence. We want to estimate the parameter vector h through
k
the stochastic approximation algorithm:
h =h +µ (b −A h ), (9)
k+1 k k k k k
where µ is the kth step gain of the form µ = k−χ for some χ ∈ 1,1 ,
k k 2
A =z zT and b =y z .
k k k k k+1 k (cid:0) (cid:3)
KouritzinandSadeghi[9]studiedtheconvergenceandalmostsureratesofcon-
vergenceforthe algorithm(9). Now,we cancombine ourmainresult(Theorem
4 ) with [9, Corollary 2] to obtain a powerful rate of convergence result for
stochastic approximation.
M.A. Kouritzin and S. Sadeghi/MSLLN for outer-products of LinearModels 9
Theorem 5 Let {Ξ } be i.i.d. zero-mean random Rm-vectors such that
l
suptαP(|Ξ |2 >t)<∞ for some α∈(1,2)
1
t≥0
(Cl)l∈Z be R(d+1)×m-matrices such that sup|l|σkClk<∞for someσ ∈ 12,1 ,
l∈Z
(cid:0) (cid:3)
∞
(zT,y )T = C Ξ ,
k k+1 k−l l
l=−∞
X
A =z zT and b =y z and A=E[z zT] and b=E[y z ].
k k k k k+1 k k k k+1 k
Then,|h −h|=o(n−γ)asn→∞a.s.for anyγ <γ(χ) =. (χ−1)∧(χ+2σ−2).
n 0 α
Proof. By Theorem 4 when 1 = χ −γ, XT = XT = (zT,y ), Ξ = Ξ ,
p k k k k+1 l l
z zT y z
C =C , σ =σ, and D = k k k+1 k ,
l l k y zT y2
(cid:18) k+1 k k+1 (cid:19)
n
1
(D −D)→0 a.s.,
nχ−γ k
k=1
X
A b n
whereD = .Thefirstd-rowsof 1 (D −D)→0a.s.
bT E[y2 ] nχ−γ k
(cid:18) k+1 (cid:19) k=1
then establish the MSLLN P
n n
1 1
(A −A)→0 and (b −b)→0 a.s.
nχ−γ k nχ−γ k
k=1 k=1
X X
Now, we apply [9, Corollary 2] to complete the proof. (cid:3)
Remark 3 Note that χ−γ satisfies the required conditions χ−γ >2−2σ and
χ−γ > 1 in Theorem 4. Theorem 5 also appears in [9, Theorem 7].
α
3.1.2. Non-linear Function of linear processes
As mentionedin Background,Vaiciulis [19]showedthe convergenceofdistribu-
tionsofthe partialsumprocesseswithnon-linearh(x )intermsofconvergence
k
of Appell polynomials A (x ) of a long-memory moving average process {x }
m k k
with i.i.d. innovations {ξ } in the case where the variance EA2 (x )= ∞, and
k m k
the distribution of ξm belongs to the domain of attraction of an α-stable law
1
with 1<α<2.
Practically,thesimplestexamplesoffunctionsh(x)withagivenAppellrank
mareAppellpolynomialsh=A relativetothemarginaldistributionx ofthe
m 1
linearprocess(3).Incasem=2theAppellpolynomialisA (x)=x2−µ where
2 2
µ =Ex2.Viaiciulis[19,Theorems1.1and1.2]provedthatwhenm(2σ−1)<1,
2
m ≥ 2 and σ ∈ (1,1) the limit distribution of partial sums of mth Appell
2
polynomialis either (i) an α-stable Levy process for 2−2σ <1+ 2(1 −1), or
m α
M.A. Kouritzin and S. Sadeghi/MSLLN for outer-products of LinearModels 10
(ii) an mth order Hermite process for 2−2σ > 1+ 2(1 −1) or (iii) the sum
m α
of two mutually independent processes depending on the value of α,m and σ,
for 2−2σ =1+ 2(1 −1).
m α
Taking into account all his conditions ( when t=1 ) and transforming it to
our case we write our complementary almost sure rate-of-convergencetheorem.
Theorem 6 Suppose A represents the Appell polynomials with rank 2 relative
2
∞
to the marginal distribution x of the linear process x = c ξ , for p ∈
1 k k−j j
j=0
X
[1, 1 ∧α) when
2−2σ
suptαP(ξ2 >t)<∞ for some α∈(1,2), (10)
1
t≥0
1
sup|l|σ|c |<∞ for some σ ∈ ,1 . (11)
l
l∈Z (cid:18)2 (cid:21)
Then,
n
1
lim A (x )=0 a.s.
n→∞np1 2 k
k=1
X
One might wonderif we haveobtainedthe best possible MSLLN. Indeed,we
have. For example for m = 2, Viaiciulis [19] shows convergence in distribution
n
of 1 A (x ) to different non-trivial limits in cases (2 − 2σ) > 1
n(2−2σ)∧α1 k=1 2 k α
(LRD dominaPnt) or (2 − 2σ) < 1 (HT dominant), respectively. Therefore,
α
n
1
A (x ) cannot converge to zero almost surely. Theorem 6 gives
n(2−2σ)∧α1 2 k
k=1
X
MSLLN for Appell polynomials with rank 2 or in other word gives the conver-
gence and almost sure rates of convergence for partial sums of second Appell
polynomial when 1 > (2−2σ)∨ 1. Our result is optimal in polynomial sense
p α
and we cannot do better than that in terms of MSLLN.
3.1.3. Autocovariances
Asmentionedinthebackground,autocovarianceestimationunderHTandLRD
conditionsisanactiveareaofresearch.Wewillhandlethe asymptoticbehavior
ofsamplecovariancefunctionforprocesseswithLRD,innovationsofinfinite4th
momentand finite varianceι. If we define the sample aurtocovarianceandpop-
ulation autocovariance functions by γˆ(n)(h) and γ(h), as (6), we have following
almost sure result.
∞
Theorem 7 Assume γˆ(n)(h) and γ(h), as (6) in which x = c ξ and
k k−j j
j=0
satisfies (10) and (11) with E[ξ2]=ι. Then for p satisfying p< P1 ∧α∧2
1 2−2σ
n1−p1[γˆh(n)−γh]→0 a.s. (12)