The Failure of the Ergodic Assumption

0.17 MB·English

by  M. Ignaccolo

The Failure of the Ergodic Assumption
M. Ignaccolo1, M. Latka2 and B.J. West1,3
Thewellestablishedprocedureofconstructingphenomenologicalensemblefromasinglelongtime
seriesis investigated. Itisdetermined thatatimeseries generated byasimpleUhlenbeck-Ornstein
Langevinequationismeanergodic. Howevertheprobabilityensembleaverageyieldsavariancethat
isdifferentfromthatdeterminedusingthephenomenologicalensemble(timeaverage). Weconclude
that the latter ensemble is often neither stationary nor ergodic and consequently the probability
ensemble averages can misrepresent theunderlyingdynamic process.
0 PACSnumbers:
1
0
2 The ergodic hypothesis, which states the equivalence of the ergodic assumption we begin with a review of its
of time averages and phase space averages, began with introduction into the physics literature.
n
a Boltzmann [1], who conjectured that a single trajectory A stochastic physicalprocess is denoted by a dynamic
J can densely cover a surface of constant energy in phase variableX(t) thatcanbethevelocityofaparticleunder-

2 space. This ’proof ’of the hypothesis as well as many going Brownian motion, the erratic voltage measured in
2 subsequentproofs were subsequently shownto be fatally an electroencephalogram or the time series from any of
flawed. It was not until metric decomposability was in- avastnumber ofothercomplexphysicalorphysiological

] troducedbyBirkoff[2]thatarigorousmathematicalthe- phenomena. In any event in the absence of direct evi-
n
a ory of ergodicity began to take shape. Kinchin [3], who dence to the contrary it is assumed that the underlying
- wrote a seminal work on the mathematical foundations processisstationaryintimeimplyingthatthisobservable
a
ofstatisticalmechanics,offeredasecondmeaningforthe can be determined experimentally from a single histori-
t
a ergodichypothesis,thatbeing,toassumethetruthofthe calrecordX(t)measuredoverasufficientlylongtime. As
d
hypothesisandjudgethetheoryconstructedonthisbasis stated in a classic paper by Wang and Uhlenbeck (WU)
.
s byitspracticalsuccessorfailure. Thislatterperspective in 1945 [11]:
c
is the one adopted by the vast majority of physicists,
i
s withthesubsequentreplacementofphasespaceaverages ..One can then cut the record in pieces of
y
with averages over an ensemble probability density. We length T (where T is long compared to all
h
callthislatteruseofergodicitythe’ergodicassumption’. ”periods” occurring in the process), and one
p
[ mayconsiderthedifferentpiecesasthediffer-
The ergodicassumptionwasquite evidentinquantum
entrecordsofanensembleofobservations. In
1 measurements, which historically employed single parti-
computing average values one has in general
v cleensemblestoevaluateaverages. Howevertheadvances
to distinguish between an ensemble average
3 ininstrumentationandmeasurementtechniquesoverthe
0 and a time average. However, for a station-
past decade or so enabled the measurement of single
1 ary process these two ways of averaging will
molecule time series and these relatively recent experi-
4 always give the same result...
ments forced the ergodic assumption out of the shadows
.
1 intotheforeground. MargolinandBarkai[4]haveargued
0 This technique for generating empirical realizations of
thatawidevarietyofcomplexphenomenawhosedynam-
0 physicalensemblessubsequentlyprovedtobeveryuseful
ics consistof randomswitching between two states, such
1 andhasbeentheformoftheergodicassumptionadopted
as the fluorescent intermittency of single molecules [5]
: todetermineaveragequantitiesfromtimeseriesforthree
v andnanocrystals[6]maydisplaynon-ergodicity,whichis
i generations of scientists. The stationarity assumption
X to say the ergodic assumption breaks down. The break
impliesthatsinglevariabledistributionsareindependent
downoccursbecausethe probabilitydensity forsojourn-
r of time, so that their ensemble averages and time aver-
a ing within one of the two states is given by the inverse
power law ψ(t) ∝ t−µ so that when µ < 2 the average agesarethe same. Moreover,the jointdistributionfunc-
tions for pairs of dynamic variables depends only on
sojourn time within a state diverges and the time series
the difference in time between the measurements of the
is manifestly non-ergodic . A similar breakdown is ob-
two variables. Formally, this implies that the difference
seved in neuroscience where the cognitive activity asso-
variable Z(t,τ) = X(t+τ)−X(t) is only a function of
ciated with perceptual judgement is determined to span
the time difference τ, that is, Z(t,τ) = Z(τ). However
the interval 1 < µ < 3 [8] and consequently to manifest
a stationary time series need not have stationary incre-
non-ergoic behavior in a variety of experiments [9, 10].
ments and the difference variable can depend on t and τ
In this Letter we show that the ergodic assumption is separately. Since the introduction of phenomenological
evenmoredelicatethandeterminedintheseexperiments, ensembles over a half century ago the phenomena of in-
which is to say it is less applicable to complex networks terest have become more complex and consequently the
in general, or physical networks in particular, than was WU approach to evaluating averages is often not ade-
previouslybelieved. Toestablishtheextentofthefailure quate. To establish this inadequacy we go back to the
2
prequel of WU, another classic, and consider a process If g(x) = x, then (3) is a generalization of the law of
first analyzedin detail by Uhlenbeck and Ornstein(UO) large numbers and in the limit of long time lags the
[12] in 1930. time average of the dynamic variable exists in proba-
LetusconsidertheUOstochasticprocessaboutwhich bility [14]. The process in this case is mean ergodic,
weknoweverythingsoastotesttheergodicassumption. or first-order ergodic, which requires that the autocor-
Consider the UO Langevin equation relation function have the appropriate asymptotic be-
havior, that being, for C (τ) = hX(t)X(t+τ)i we
X T
dX(t) T
=−λX(t)+F(t) (1) must have 1 C (τ)dτ → 0 as T → ∞ in which
dt 2T X
R0
case (5) is true [15]. If we consider the quadratic vari-
where in a physics context X(t) is the one-dimensional
able Z(t) = X(t)X(t+τ) then Z(t) is mean ergodic if
velocity, the particle mass is set to one, λ is the dissi-
T
pation parameter and F(t) is a random force. In other 1 C (τ)dτ →0asT →∞,whichimpliesthatX(t)is
contexts the dynamic variable has been taken to be the 2T Z
R0
voltagemeasuredinanEEGrecord[13],oranyofanum- second-orderergodic. The question remains whether the
ber of other interesting observables. The solution to the phenomenological ensemble averages constructed from a
UO Langevin equation, as we show below, actually vi- single historical trajectory of long but finite length T
olates the stationarity assumption made by Wang and satisfies (5) for any g(x) other than linear as explicitly
Uhlenbeck in 1945. To establish this violation we calcu- assumed in WU. It is probably obvious but it should be
late the variance using the phenomenological ensemble said that first-order ergodicity is important in statistical
and find that it is different from that obtained using the physicsbecauseitguaranteesthatthemicrocanonicalen-
multi-trajectory ensemble (MTE), that is, the ensemble semble of Gibbs produces the correct statistical average
obtained by solving the stochastic differential equation in phase space for Hamiltonian systems [14].
(1)usinganensembleofrealizationsoftherandomforce.
Toanswerthequestionregardingthevalidityof(5)for
The significance of this result can not be overestimated
quantities higher than first-order consider the solution
since it directly contradicts a half century of analyses
to the UO Langevin equation, which since the dynamic
made assuming the ergodicity of the phenomenological
equation is linear has the same statistics as the random
ensembles constructed from the time series.
force. Assumingtherandomforceisazero-centereddelta
The formal solution of the OU Langevin equation is,
correlated Gaussian process, we know that the solution
with the initial condition X(0),
(2) is completely determined by the mean and variance.
The solution with X(0) = 0 yields the average values
t
for the dynamic variable hXi = hXi = 0. The
X(t)=e−λtX(0)+Z F(t′)eλt′dt′. (2) autocorrelation function is C T(τ) = σF2 Me−TλEτ so that the
X 2λ
 0  T
condition 1 C (τ)dτ →0 as T →∞ is satisfied and
Aproperinterpretationofthissolutionrequiresthespec- 2T X
R0
ification of the statistics of the random force. If we as- consequently the solution to the UO Lagevinequationis
sumetherandomforceisazero-centereddeltacorrelated first-order ergodic.
Gaussian process, as is often done, then the solution (2)
Now let us consider the variance using the MTE av-
is generally accepted as a stationary ergodic process.
erages consisting of an infinite number of trajectories all
startingfromX(0)=0(withoutlossofgenerality)yield-
Recallthatanergodicprocessisoneforwhichthetime
ing
average of an analytic function of the dynamic variable
σ2
T σ2(t)≡ X2(t) −hX(t)i2 = F 1−e−2λt .
1 MTE MTE 2λ
′ ′
hg(X)i ≡ lim g(X(t))dt (3) (cid:10) (cid:11) (cid:2) ((cid:3)6)
T T→∞T Z
Note that (6) is also the variance obtained using the
0
Gaussian solution to the Fokker-Planckequation for the
and the MTE average ensemble probability distribution.
Now let us examine the single trajectory case. To cre-
ateanensembleoftrajectoriesconsiderdifferentportions
hg(X)i ≡ g(x)P(x)dx (4)
MTE Z of the single trajectory. Let X(t) (X(0)=0) be a single
trajectory of maximum finite length T and consider the
are equal set of trajectories for t∈[0,T −τ]
hg(X)i =hg(X)i . (5) Z(t,τ)=X(t+τ)−X(t). (7)
T MTE
3
Using (2), we can write X(t+τ) as
200
τ
X(t+τ)=e−λτX(t)+ F(t+t′)eλt′dt′ (8)
Z
 0 
100
where the initial condition for the trajectory at t+τ is
τ)
thetimeseriesattimet. Inserting(8)into(7),weobtain σ(
for t∈[0,T −τ]
τ λ =λ/2
Z(t,τ)=e−λτ F(t+t′)eλt′dt′+X(t) e−λτ −1 eff λ
Z STE/WU
0 (cid:2) (cid:3) MTE
30
(9)
1 10 100 1000
Note that the first term of the rhs of (9) is the solution
τ
oftheOULangevinequation(1)witht=τ andX(0)=0,
apart from the fact that in the integrand of (9) only the FIG.1: ThevariancecalculatedfortheOUprocessusingthe
time in the range [t,t+τ] are considered for the function singlelongtimeseries andthephenomenological WUensem-
F (instead of the range (0,τ) as in (2)). We now define ble. Note the difference in the dissipation rates for the two
the solution starting from zero to be ways of calculating the variance.
τ
X0(t,τ)=e−λτ F(t+t′)eλt′dt′ (10) Inserting the mean of the phenomenological second mo-
Z
ment into the integrand yields
0
sothatthetermX(t)in(9)isequivalenttoX (0,t),and
0
therefore with t∈[0,T −τ] (9) reduces to T−τ
Z(t,τ)=X0(t,τ)+X0(0,t) e−λτ −1 (11) σs2traj(τ)= T −1 τ Z dtnσ2(τ)+(cid:2)e−λτ −1(cid:3)2σ2(t)o
(cid:2) (cid:3) 0
This equation reveals that the statistics of the variable (14)
Z(t,τ) depends on the particular value of t through the and using (6) allows to write after integrating over time
second term on the rhs. In fact the term X (0,t) has
0
zero mean and variance that depends on t according to
σ2 (τ) = σ2(τ)+ e−λτ −1 2
(6). Alsonoticethatthe firsttermoftheaboveequation straj
depends on t but the ”dependence” is a time transla- σ2 (cid:2) 1 (cid:3) e−2λ(T−τ)−1
tion of the variable F in the integrand of (10). But the × F 1+ (15)
(cid:20)2λ(cid:21)(cid:20) T −τ 2λ (cid:21)
random force F is stationary by assumption and thus
X0(t,τ) is statistically equivalent to X0(0,τ). The func- Since λT≫1 (because T is the length of the record) we
tion Z(t,τ) is the sum of two statistically independent can write
GaussianvariablesandthereforeisitselfaGaussianvari-
able with hZ(t,τ)iT = hZ(t,τ)iMTE = 0 and second- σ2 (τ)≈σ2(τ)+ e−λτ −1 2 σF2 (16)
moment straj (cid:20)2λ(cid:21)
(cid:2) (cid:3)
Z2(t,τ) =σ2(τ)+ e−λτ −1 2σ2(t). (12) and substitution from (6) yields
MTE
(cid:10) (cid:11) (cid:2) (cid:3)
Weusethisvariancetodemonstratethatthephenomeno-
logical ensemble created is not equivalent to MTE be- σ2
causethe trajectoriesZ(t,τ) arenotstatistically equiva- σs2traj(τ)≈ λF(1−e−λτ). (17)
lent to those of X(t). We show that σ2 , the variance
straj
calculated with the ensemble Z(t,τ), is different from Thelastexpressionshowsthatthephenomenologicalen-
that calculated using the MTE (6). semble trajectories Z(t,τ) behave as a MTE with an ef-
To calculate the variance σ2 (τ) using the phe- fective dissipation that is half the dissipation rate in the
straj
nomenologicalensemble,we simply needto calculatethe Langevin equation generating the process, that is,
meanvalue(amongthetrajectories)using(12). Herethe
variance is calculated using the time average since only
the time series is assumed to be available to us λeff =λ/2. (18)
T−τ Consequently,it takes halfas long for the MTE variance
1
σ2 (τ)= dt Z2(t,τ) . (13) to decay as it does for the phenomenological variance of
straj T −τ Z MTE
(cid:10) (cid:11) the time series.
0
4
Consider what has been established in this Letter. using the phenomenological ensemble. The inescapable
Firstalongtimeseriesisgeneratednumericallyfromthe conclusion is that the phenomenological prescription for
U) Langevin equation. This long series is separated into calculating the average of any nonlinear function of the
a large number of equal length time series as prescribed dynamic variable using time series is not equivalent to
by Wang and Uhlenbeck [11] to form a phenomenolog- that using a MTE. The fact that (5) is not satisfied for
ical ensemble of realizations of the UO process. This g(x)6=x implies that the ergodic assumption is violated
phenomenologicalensembleoftrajectoriesisshowntobe by any finite time phenomenological ensemble. Given
mean ergodic. However when they are used to calculate the simplicity of the dynamic model discussed here; it is
thevarianceoftheprocessanticipatingthatthesamere- meanergodic;combinedwiththerecentfindingsconcern-
sultwillbeobtainedasthatforaMTEaverage,theprob- ing the non-ergodicity of experimental data mentioned
ability ensemble average, of the variance. The expected earlier,it is notunreasonableto concludethat the appli-
result turns out to be wrong. The dissipation parameter cation of the ergodic assumption in general is probably
determined by the MTE variance is twice that obtained unwarranted.
[1] L.Boltzmann,LecturesonGasTheory,firstpublishedin Rept. 468, 1-99 (2008).
1895,translatedbyS.G.Brush,Dover,NewYork(1995). [8] P. Grigolini, G. Aquino, M. Bologna, M. Lukovic and
[2] G.D. Birkoff, ”Proof of the Ergodic Theorem”, PNAS B.J. West, ”A theory of 1/f noise in human cognition”,
17, 656-660 (1931). Physica A 388, 4192 (2009).
[3] A.I. Kinchin, Mathematical Foundaitons of Statistical [9] D.L. Gilden, T. Thornton, and M.W. Mallon, ”1/f noise
Mechanics, translated by G. Gamow, Dover, New York in Human Cognition”, Science 267, 1837 (1995).
(1949). [10] J. correll, ”1/f noise and effort on implicit measures of
[4] G. Margolin and E. Barkai, ”Nonergodicity of blinking bias”, J. Personality and Social Psych. 94, 48 (2008).
nanocrystalsandotherL´evy-walkprocesses”,Phys.Rev. [11] M.C. Wang and G.E. Uhlenbeck, ”On the theory of the
Lett. 94, 080601 (2005). Brownian motion II”,Rev. Mod. Phys. 17, 323 (1945).
[5] M. Haase, c.G. Hu¨bner, E. Reuther, A. Herrmann, K. [12] G.E.UhlenbeckandL.S.Ornstein,”Onthetheoryofthe
Mu¨llen and Th. Basch´e, J. Phys. Chem. B 108, 10445 Brownian motion”, Phys. Rev. 36, 823 (1930).
(2004). [13] M.Ignaccolo, M.Latka,W.Jernajczyk,P.Grigoliniand
[6] M.Nirmal,B.O.Dabbousi,M.G.Bawendi,J.J.Macklin, B.J. West, ”The dynamics of EEG entropy”, J. Biol.
J.K.Trautman,T.D.Harris,andL.E.Brus,Nature 383, Phys. (2009).
802 (1996); X. Brokmann, J. P. Hermier, G. Messin, P. [14] J.L. McCauley, ”Time vs. ensemble averages for nonsta-
Desbiolles, J. P. Bouchaud and M. Dahan, Phys. Rev. tionary time series”, Physica a 387 , 5518-5522 (2008).
Lett. 90, 120601 (2003). [15] B.V. Gnedeko, Theory of Probability (trans. from Rus-
[7] B.J. West,E.L. Geneston andP.Grigolini, ”Maximizing sian byB.D. Seckler) Chelsea, NY (1968).
informationexchangebetweencomplexnetworks”,Phys.