Markovian Statistics on Evolving Systems

0.2 MB·

by  Ulrich Faigle

Markovian Statistics on Evolving Systems
UlrichFaigle1,GerhardGierz2
1 MathematischesInstitut
7 Universta¨tzuKo¨ln
1 Weyertal80,50931Ko¨ln,Germany
0 [email protected]
2
2 DepartmentofMathematics
n
UniversityofCaliforniaatRiverside
a
Riverside,CA92521,USA
J
[email protected]
2
2

] Abstract. Anovelframeworkfortheanalysisofobservation statisticsontime
T
discretelinearevolutionsinBanachspaceispresented.Themodeldiffersfrom
S traditionalmodelsforstochasticprocessesand,inparticular,clearlydistinguishes
. betweenthedeterministicevolutionofasystemandthestochasticnatureofob-
h
t servations on the evolving system. General Markov chains are defined in this
a
context and it isshown how typical traditional models of classical or quantum
m
random walks and Markov processes fit into the framework and how a theory
[ of quantum statistics (sensu Barndorff-Nielsen, Gill and Jupp) may be devel-

oped from it.Theframework permitsageneral theory of joint observability of
1
twoormoreobservationvariableswhichmaybeviewedasanextensionofthe
v
Heisenberguncertaintyprincipleand,inparticular,offersanovelmathematical
0
4 perspective ontheviolationof Bell’sinequalitiesinquantummodels. Mainre-
1 sultsincludeageneralsamplingtheoremrelativetoRieszevolutionoperatorsin
6 thespiritofvonNeumann’smeanergodictheoremfornormaloperatorsinHilbert
0 space.
.
1
0
Keywords: Banachspace,Bellinequality,ergodictheorem,evolution,Heisenbergun-
7
certainty, Hilbert space, Markov chain, observable, quantum statistics, random walk,
1
: sampling,Rieszoperator,stochasticprocess
v
i
X
1 Introduction
r
a
ConsiderasystemSthatisobservedatdiscretetimest = 1,2,3,...relativetoapre-
specifiedeventSthatmayormaynotoccurattimet.Astatisticalanalysisisinterested
intherelativefrequencyoftheeventS.Inparticular,onewouldliketoknowwhether
the sample frequency of its occurrence converges to a definite limiting value. For a
mathematicalformulationofthisproblem,letusdefinetheindicatorfunctionsI(t) as
S
(0,1)-variableswiththeeventnotation{I(t) = 1}meaningthatS isobservedattime
S
t.Thesampledfrequencyuptotimetwouldthenbe
(t) 1 (m)
I = I .
S t S
m=1
X
WecallthestudyofthelimitingbehaviorofthesamplefrequencyofaneventS asthe
Markovianproblem.
MathematicalapproachestothisproblemtypicallythinkofSasarandomsource
that emits symbols from some (finite or infinite) alphabetA and thus givesrise to an
A-valuedstochasticprocess(X ).S isassumedtorepresentaparticulareventrelative
t
tothisprocessthatmayormaynotmaterializeattimet.Sotheobservationvariables
I(t) reflectanassociatedstochasticprocessintheirownrightandtheexpectedaverage
S
numberofobservationsofS attimetis
t t
E(I(t))= 1 E(I(m))= 1 I(m)Pr{I(m) =1}.
S t S t S
m=1 m=1
X X
No matter what the nature of the underlying stochastic process (X ) is, the mathe-
t
matical analysis of the statistical problem relative to the event S, i.e., the Markovian
problem,willactuallybeontheassociatedbinaryprocess(I(t)).
S
The present investigation is concerned with the Markovian problem of the rela-
tiveoccurrenceofsomeeventS ratherthanwiththemathematicalanalysisofgeneral
stochastic processes per se. Therefore,there is no loss in generality when we restrict
ourselvestoamodelwhereSisviewedassomesourcethatemitssymbolsfromsome
alphabetAoffinitecardinality|A|<∞.Infact,theassumptionofAasabinaryalpha-
bet(i.e.,|A|=2)wouldtheoreticallysuffice.However,itisconvenienttoalsoconsider
moregeneralalphabetsoccasionally.
Astochasticprocess(X )isusuallyunderstoodtobeasequenceofstochasticvari-
t
ablesX thataredefinedonsomeprobabilityspaceandoneisinterestedintheexpected
t
limitingbehaviorof(X ).However,ifonetakesstatisticson(X ),i.e.,averagestheob-
t t
servationof specialeventsovertime, there is notalwaysa clear asymptoticbehavior.
Indeed,thereareexamplesofevencompletelydeterministicprocesses(X )whereob-
t
servationstatisticsdonotconverge.Itisthelimitingbehaviorofobservationaverages
weareconcernedwithhere.
OurapproachismotivatedbyaMarkovianinterpretationof(X ):Arandomsource
t
S produces symbols a of an alphabet A over time t with {X = a} denoting these
t
events.S isthoughtto changeoverdiscrete time t, with the probabilityPr{X = a}
t
dependingonthecurrentstateofS.TheclassicalexamplegoesbacktoMarkov’s[21]
modelofasystemthatadmitsasetN ofgroundstatesandissubjecttoarandomwalk
onN withtransitionprobabilitiesp .Thesystemstatesarethentheprobabilitydistri-
ij
butionsp(t)ofthepositionsoftherandomwalkattimest.TheMarkovmodelhasbeen
verysuccessfulinapplicationmodeling.Statisticalmechanicsinphysics,forexample,
describes the behavior of ideal gases in this way. But also the behavior of economic
andsocialsystemsis oftenviewedasfollowingMarkovianprinciples.Internetsearch
enginessuccessfullyorganizeandranktheirsearchaccordingtoMarkovianstatistics.
Thesituationseemstobemorecomplicatedwithquantumsystemsthatdonotadmit
a classicalanalysis.Forexample,theresultofaquantummeasurementisnota deter-
ministic functionof the state of the system and the measuringinstrumentappliedbut
MarkovianStatistics 3
ratheranexpectedvaluerelativeto some(state dependent)probabilitydistributionon
thepossiblemeasurementoutcomes.Moreover,Heisenberg’suncertaintyprinciplesays
thatobservationsmaynotbesimultaneouslyfeasibleunlesstheyconformtoaspecial
condition.Experimentalevidencewithspincorrelations(Aspectetal.[2])furthermore
exhibitsadefiniteviolationofclassicalstatisticalprinciplesasexpressedinBell’s[4,5]
inequalities.WhiletheSchro¨dingerpictureofquantumstatesbeingdescribedbywave
functions yields a special theory of quantum probabilities with applications also to
quantum computing,active currentresearch effort is devoted to the quantum analogs
ofclassicalMarkovrandomwalks. Inthisspirit,Barndorff-Nielsen,GillandJupp[3]
haveputforwardatheoryofquantumstatisticalinference.
The present investigation proposes a quite general model for Markovian statisti-
cal analysis. Rather thanfollowingthe standardapproachto stochastic processes,our
modelislinearandmotivatedbythelinearalgebraicanalysisofclassicalMarkovchains
ofGilbert[14],Dharmakhari[8]andHeller[16],whichhasledtotheidentificationof
more general Markov type processes (e.g., Jaeger [19]). Addressing the issue of the
”dimension” of a stochastic evolution, the asymptotic behavior of even more general
stochasticprocessescouldbeclarified(FaigleandScho¨nhuth[11]).Generalizingthese
previousmodels, our setting is in Banach space and focusses on the evolution of lin-
earoperators,whichallowsustodealalsowiththestatisticsofdiscretequantumtype
evolutionsappropriately.
Thereareseveraladvantagesandnovelaspectsinourapproach.Notonlydoesour
model include typical Markovian models proposed so far (see the examples in Sec-
tion4),butitsgeneralityallowsustodevelopameaningfulnotionofjointlyobservable
statistical measuring instruments on an evolving system. Sets of classical stochastic
variablesarealwaysjointlyobservable(forthesimplereasonthattheyaremathemat-
ically based on the same underlying probability space). The Heisenberg uncertainty
principle,on the other hand,makes it clear that this propertyis no longer guaranteed
forstatisticalobservationsonquantumsystems.
WhiletheHeisenbergprincipleisformulatedforpairsofself-adjointoperators,our
modelallowsustodealwiththree(ormore)operatorsaswell.WeshowthattheHeisen-
berg principle correspondsto a very special case in our setting (Section 5). In fact, a
carefulmathematicalanalysisofthejointobservabilityof3measurementoperatorsmay
offerastraightforwardkeytotheunderstandingofBell’sinequalities(Section5.2).
Theseadvantagesaretheresultofaclearseparationoftheaspectofthe(determin-
istic) evolutionof a system fromthe aspectof statisticalobservationsonthe evolving
systeminthemathematicalmodel.
Ourpresentationis organizedasfollows.Section2introducesevolutionoperators
onBanachspacesanddiscussestheir ergodicity.Thensamplingfunctionsarestudied
andtheirconvergencebehaviorischaracterizedintheSamplingTheorem(Theorem3)
relativetofinitaryevolutions,whichincludeallevolutionsbasedonRieszoperators,for
example.ObservablesandgeneralizedMarkovchainsaredefinedin Section3. These
notions are illustrated by the examples in Section 4 with particular emphasis on ran-
domwalksandquantumstatistics.Theproofsofthemainresultsaredeferredintothe
Appendix.
2 Evolutions ofsystems
LetSbesomesystemthatisinacertainstateS atanytimet.ObservingSatdiscrete
t
timest = 0,1,2,...,we referto thesequenceǫ = (S ) asanevolutionofS. For
t t≥0
amathematicalanalysis,theevolutionneedstoberepresentedinsome(mathematical)
universe U. In the present investigation, we will always assume U to be be a vector
spaceoverthecomplexfieldC.ArepresentationoftheevolutionǫinU isthenamap
t7→s(t) ∈U suchthatthereisalinearoperatorψonU withtheproperty
s(t+1) =ψs(t) (t=0,1,2,...).
Wethinkofthevectors(t) ∈ U astherepresentationofthestateS ofSattimetand
t
callψanevolutionoperator.Clearly,anyevolutionǫofSadmitssucharepresentation.
Forexample,astationaryrepresentation,wheres(t) = s(0) foralltandtheevolution
operatorsareexactlythose(linear)operatorsonU thatfixs(0).
Ina practicalsystem analysis,it isthe firsttask ofthe modelerconsistsin the de-
termination of an appropriate representation of the evolution of the system S under
consideration.Here,however,wewillassumethattheevolutionisalreadyrepresented
insomeuniverseU sothattheevolutionsarevectorsequencesΨ oftheform
Ψ =(ψ,s)=(ψts|t=0,1,2,...)
where ψ is an operator on U. We furthermore assume that U is endowed with some
norm k·k and is complete with respect to this norm (otherwise we replace U by its
completionU).
Remark1 By standard complexification argumentsin functionalanalysis (e.g., [7]),
theresultsweobtaininthissectionapplytouniversesovertherealfieldRaswell.We
chooseCformathematicalconvenience,withoutlossofgenerality.
The evolution space of the evolution Ψ = (ψ,s) in U is the linear subspace U
Ψ
generatedbyΨ,i.e.,
U =lin{ψts|t=0,1,...}.
Ψ
TheparameterdimΨ = dimU isthedimensionoftheevolutionΨ.We willreferto
Ψ
thevectorss(t) =ψtsasthestatesofΨ.
Notice that U is ψ-invariant (i.e., ψ(U ) ⊆ U ). So the restriction of ψ to U
Ψ Ψ Ψ Ψ
is an operatoron the normedspace U . Let U be the closureof U in U and recall
Ψ Ψ Ψ
from general operator theory3 that ψ extends to a unique norm bounded (and hence
continuous)operatorψ : U → U with the same (finite)norm,providedψ is norm
Ψ Ψ
boundedonU .ThenormofψonU is
Ψ Ψ
kψk =inf{c∈R|kψuk≤ckukforallu∈U }.
s Ψ
3e.g.,[7,9]
MarkovianStatistics 5
Inthe case ofa finite-dimensionalevolution(i.e.,dimΨ < ∞),forexample,kψk is
s
necessarilyfinite.ThenormoftheevolutionΨ =(ψ,s)isdefinedas
kΨk=inf{c∈R|kψtsk≤cksk,∀t≥0}
andΨ saidtobestableifkΨk<∞.ThusΨ isstableifkψk ≤1,forexample.
s
Lemma1. IfΨ =(ψ,s)isstable,thenΨ′ =(ψ,s′)isstableforeverys′ ∈U .Hence
Ψ
therestrictionofψtoU doesnotadmitanyeigenvalueλwith|λ|>1.
Ψ
Proof.Consideranys′ = kj=1ajψtjs∈UΨ.Thenthetriangleinequalityyields
P
k
kψts′k≤ kΨk |a | ksk forallt≥0.
j
j=1
(cid:0) X (cid:1)
(cid:4)
TheevolutionΨ =(ψ,s)isergodicifitsstatess(t) =ψtsconvergeinthenorm.Ψ
ismeanergodicifthestateaverages
t
1
s(t) = s(m)
t
m=1
X
convergetosomelimitstates∞ ∈U .Clearly,ifs(∞)exists(ifandkψk <∞holds),
Ψ s
Ψ isstationaryinthesense
ψs(∞) =s(∞).
Moreover,anergodicevolutionisalsomeanergodic,whiletheconverseconclusionis
generallyfalse.
2.1 Equivalentevolutions
LetuscalltwoevolutionsΦ=(ϕ,v)andΨ =(ψ,w)inU equivalentif
lim kψtw−ϕtvk=0.
t→∞
ByCauchy’sTheorem,equivalenceimplies
t
1
lim kψmw−ϕmvk=0.
t→∞ t
m=1
X
So, assuming equivalence,Φ is meanergodicexactlywhen Ψ is mean ergodicandin
eithercase,onehas
t t
1 1
lim ψmw = lim ϕmv. (1)
t→∞ t t→∞ t
m=1 m=1
X X
Finitaryevolutions WesaythattheevolutionΨ =(ψ,s)isfinitaryifΨ isequivalentto
afinite-dimensionalevolutionΦ=(ϕ,v).Acharacterizationofthemeanergodicityof
afinite-dimensionalevolutionΦfollowsfromtheanalysisofFaigleandScho¨nhuth[11]
andsaysinessence:ΦismeanergodicpreciselywhenΦisequivalenttoanevolution
Π =(π,v),whereπisaprojectionoperatoronU =U .
Φ Φ
Proposition1 ([11]).LetΦ=(ϕ,v)beafinite-dimensionalevolution.ThenΦismean
ergodicifandonlyifΦisstable.Moreover,ifΦisstable,onehas
t
1 0 ifλ=1isnotaneigenvalueofϕ
lim ϕmv =
t→∞ t P1v ifλ=1isaneigenvalueofϕ,
m=1 (cid:26)
X
whereP isaprojectionoperatorontotheeigenspaceE ={x∈U |x=ϕx}.
1 1 Φ
Proof. By Lemma1,Φis stableif andonlyif(φ,v′)isstableforallv′ ∈ U .So
Φ
Proposition1isadirectconsequenceofTheorem2anditsproofin[11].
(cid:4)
Riesz evolutions Recall that the spectrum σ(T) of a (linear) operator T : V → V
on a complexnormed vector space V consists of those λ ∈ C such that the operator
L =T−λ(withvaluesL v =Tv−λv)isnotinvertible.T iscalledaRieszoperator
λ λ
ifT isboundedand
(a) eachλ∈σ(T)\{0}isaneigenvalueofT withfinitealgebraicmultiplicity;
(b) 0istheonlypossibleaccumulationpointofσ(T).
Rieszoperatorsformaquitewideclassofoperatorsthatincludestheso-calledcom-
pact operators.In particular,everyoperatorT with finite-dimensionalrange is Riesz.
Furtherexamplesarethe Hilbert-Schmidtoperatorsona HilbertspaceH, namelythe
boundedoperatorsT :H→Hsuchthat
kTe k2 <∞
i
i∈I
X
holdsforsomeorthonormalbasis{e |i∈I}ofH.Notethateveryoperatoronafinite-
i
dimensionalvectorspaceV istriviallyHilbert-Schmidtrelativetoanyinnerproduct.
A Riesz evolution in our universe U is now an evolution Ψ = (ψ,s) such that ψ
extendstoaRieszoperatorψ :U →U .Inparticular,everyevolutionunderaRiesz
Ψ Ψ
evolutionoperatoronU isRiesz.
Thecharacterizationofmeanergodicfinite-dimensionalevolutions(Proposition1)
extendstogeneralRieszevolutions.
Theorem1. LetΨ =(ψ,s)beanyRieszevolutioninU.ThenΨ isfinitary.Moreover,
Ψ ismeanergodicifandonlyifΨ isstable.Inparticular,ifΨ isstable,onehas
t
1 0 ifλ=1isnotaneigenvalueofψ
lim ψmv =
t→∞ t m=1 (cid:26)P1v ifλ=1isaneigenvalueofψ,
X
whereP isaprojectionoperatorontotheeigenspaceE ={x∈Uˆ |x=ψx}.
1 1 Φ
MarkovianStatistics 7
TheessentialpartoftheproofofTheorem1consistsinshowingthatRieszevolu-
tionsarefinitary.WediscussthedetailsintheAppendix(cf.Proposition2there).
NormalevolutionsinHilbertspace LetHbeaHilbertspaceandrecallthatanoper-
atorT onHisnormalifT commuteswithitsadjointT∗(i.e.,TT∗ =T∗T).
Theorem2. LetψbeaboundednormaloperatoronHandΨ = (ψ,s)anevolution.
Thenthefollowingstatementsareequivalent:
(i) Ψ isstable.
(ii) Ψ ismeanergodic.
In this case, the averages s(t) converge to the orthogonal projection of s onto the
eigenspaceE ={x∈H|ψx=x}.
1
WeproveTheorem2intheAppendix.Theimplication”(i)⇒(ii)”iswell-known
andusuallystatedasvonNeumann’smeanergodictheorem:
Corollary1 (vonNeumann).Ifψ isanormaloperatoronHofnormkψk ≤ 1,then
everyevolutionΨ =(ψ,s)ismeanergodic. (cid:4)
Animportantspecialcaseistheevolutionofawavefunctionv ∈ Hofaquantum
system in discrete time. According to Schro¨dinger’s differential equation, there is a
unitaryoperatorU (i.e.UU∗ = I = U∗U)sothatthediscreteevolutionofv isgiven
as
v(t) =Utv (t=0,1,...).
Clearly,theoperatorv 7→Uvisnormalandbounded.Moreover,(U,v)isstableforany
v ∈H.SoSchro¨dingerevolutionsaremeanergodic.
2.2 Sampling
By a sampling function relative to the universe U we understand a continuous linear
map f : U → F, where F is a normed vector space of samples. With respect to an
evolutionψ =(ψ,s),thef =f(ψts)arethesamplingvalueswiththecorresponding
t
samplingaverages
t t
1 1
f = f = f(ψms) (t=1,2,...).
t t m t
m=1 m=1
X X
Inapplications,asamplingfunctionf willtypicallybeafunctionalintothescalarfield
ofU.ButmoregeneralsamplespacesF mayalsobeofinterest.
Thesamplingaverageswill,ofcourse,convergewhenΨ ismeanergodic.Unitary
evolutionsinHilbertspace,forexample,willguaranteeconvergingsamplingaverages.
But sampling averages may possibly also converge on evolutions that are not mean
ergodic.Thesamplingconvergenceonfinitaryevolutionsischaracterizedasfollows.
Theorem3 (Sampling Theorem). Let Ψ = (ψ,s) be an arbitrary finitary evolution
andf :U →F asamplingfunction.Thenthefollowingstatementsareequivalent:
(i) Thesamplingaveragesf converge.
t
(ii) Thesamplingvaluesf areboundedinnorm.
t
Again,wedefertheproof ofTheorem3totheAppendix.ChoosingF =U andf =I
astheidentityoperator,weimmediatelynote:
Corollary2. LetΨ =(ψ,s)beanarbitraryfinitaryevolutioninU.Then
Ψ ismeanergodic ⇐⇒ Ψ isstable.
(cid:4)
3 Observables and Markovchains
LetA be a finite or countableset. Anobservablewith rangeA onthe evolutionΨ =
(ψ,s)inU isacollectionX ={χ |a ∈A}ofcontinuouslinearfunctionalsχ such
a a
thattheχ (s(t))arerealnumberswiththeproperty
a
p(t) =χ (s(t))≥0 ∀a∈A and p(t) =1.
a a a
a∈A
X
WethinkofX asproducingtheevent{X =a}attimetwithprobability
t
Pr{X =a}=p(t).
t a
SotheobservableX yieldsasequence(X )ofstochasticvariablesX withprobability
t t
distributionsp(t).Wecall(X )a(generalized)MarkovchainonA.
t
Remark2 The probability distributionsp(t) of X may be viewed as ”stochastic ker-
a
nel”ofXandthusgeneralizetheideaofakernelofclassicalMarkovchaintheory(see,
e.g.[13,17]).AMarkov chaininoursense,however,doesnotneedtobeastochastic
processnordoesitsstochastickernelneedtoreflectanyconditionalprobabilitieswith
respecttostatetransitions.
Related to the (generalized) Markov chain (X ) is are (statistical) sampling pro-
t
cesses(Ya)withrespecttoanya∈A,where
t
1ifX =a
Ya = t
t 0otherwise.
(cid:26)
(Ya)isaMarkovchainonΨ initsownrightwithbinaryalphabet{0,1}andprobability
t
distributions
Pr{Ya =1}=p(t) and Pr{Ya =0}=1−p(t).
t a t a
MarkovianStatistics 9
AssumingthattheevolutionΨ isfinitary,forexample,Theorem3impliesthatthe
expectationoftheobservationaverages
t t t
1 1 1
E Ya = E(Ya)= Pr{Ya =1}
t m t m t m
!
m=1 m=1 m=1
X X X
convergestoadefinitelimitp(∞) ≥ 0sincethenumbersPr{Ya = 1}arebounded.In
a t
viewof
p(m) =1 forallm≥1,
a
a∈A
X
weconcludethatp(∞) = {p(∞)|a ∈ A}isaprobabilitydistributiononA.Itisinthat
a
sensethatwerefertop(∞) asthelimitdistributionoftheMarkovchain(X ).
t
4 Examples
4.1 Evolutionsofstochasticprocesses
Let(X ) be a discrete stochasticprocessthattakesvaluesin the alphabetA. Without
t
lossofgenerality,weassumethatAisbinary,sayA={0,1}.Asusual,wedenotethe
setofallfinitelengthwordsoverAas
∞
A∗ = An,
n=0
[
whereA0 = {(cid:3)}and (cid:3) is the emptyword.Foranyv = v v ...v ∈ An, |v| = n
1 2 n
is the length of v. Recall that A∗ is a semigroup with neutral element (cid:3) under the
concatenationoperation
(v ...v )(w ...w )=v ...v w ...w .
1 n 1 k 1 n 1 k
Moreover,weset
p(w ...w |v ...v )=Pr{X =w ,...X =w |X =v ,...,X =v }
1 k 1 n n+1 1 n+k k 1 1 k n
and p(v) = p(v|(cid:3)). If v ∈ At has been produced, the process is in a state that is
describedbythepredictionvectorPv withthecomponents
Pv =p(w|v) (w ∈A∗).
w
Theexpectedcoordinatevaluesofthenextpredictionvectorarethengivenbythecom-
ponentsofthevector
ψPv =p(0|v)Pv0+p(1|v)Pv1.
Binomialexpansion,therefore,immediatelyshowsthattheexpectedpredictionvector
attimetisgivenby
p(v)Pv =ψtP(cid:3).
v∈At
X
SincethecomponentsofthepredictionvectorsPvarebounded,theygenerateanormed
vectorspaceP withrespecttothesupremumnorm
kgk = sup |g |.
∞ w
w∈A∗
Moreover,isitnotdifficulttoseethatψextendstoauniquelinearoperatoronP.The
evolutionΨ =(ψ,P(cid:3))issaidtobetheevolutionofthestochasticprocess(X ).
t
Foranya∈A,onehas
(ψtP(cid:3)) = p(v)Pv =Pr{X =a}.
a a t+1
v∈At
X
Since coordinate projections are continuous linear functionals, we find that observa-
tionson(X )yieldobservablesinthesenseofSection3,whichallowsustoviewthe
t
stochasticprocess(X )asa(generalized)MarkovchainonA.
t
Remark3 TheevolutionofstochasticprocesseswasfirststudiedbyFaigleandScho¨n-
huth[11], towhichbereferforfurtherdetails.Thestochasticevolutionmodelgener-
alizesearlierlinearmodelsfortheanalysisofMarkovtypestochasticprocesses(e.g.,
Gilbert[14],Dharmadhikari[8],Heller[16]andJaeger[19]).
4.2 Finite-dimensionalevolutions
Finite-dimensionalevolutionmodelsareofparticularinterestinapplications.Anevo-
lution Ψ = (ψ,s) in Rn admits an (n × n)-matrix M such that ψx = Mx holds
for all x ∈ Rn. As observedin [11], Ψ is mean ergodicexactly when Ψ is stable (cf.
Corollary2).
LettingN = {1,...,n}andassumingthatsandallcolumnsofM areprobability
distributions on N, Ψ is clearly stable and hence mean ergodic. The Markov chain
relative to {χ |i ∈ N}, where the χ are the projections onto the n components of
i i
x ∈ Rn, yields the well-known model of a random walk on N with the transition
matrixM.ConsideringanymapX : N → AintosomealphabetA,inducedMarkov
chainwiththe”kernel”functionals
χ (x)= x (x=(x ,...,x )∈Rn)
a i 1 n
XX(i)=a
is classically known as a hidden Markov chain on A. Hidden Markov models have
provedveryusefulinpracticalapplications4.
Itisimportanttonote,however,thatevenfinite-dimensionalMarkovchainsinthe
general sense of Section 3 are not necessarily stochastic processes (see Example 1
in Section 5.2 below). Also quantum random walks (Section 4.4 are not necessarily
stochasticprocesses.
4see,e.g.,[6,10,26]