Reachability in Augmented Interval Markov Chains

0.14 MB·

by  Ventsislav Chonev

Reachability in
Augmented Interval Markov Chains
Ventsislav Chonev
IST Austria
Abstract—In this paper we propose augmented interval the goal is to determine whether some (or alternatively, all)
Markovchains(AIMCs):ageneralisationofthefamiliarinterval refinements satisfy a given property. In contrast, the at-every-
Markov chains (IMCs) where uncertain transition probabilities
7 step interpretation exhibits a more game-theoretic flavour by
are in addition allowed to depend on one another. This new
1 allowing a choice over the outgoing transition probabilities
model preserves the flexibility afforded by IMCs for describing
0 stochasticsystemswheretheparametersareunclear,forexample prior to every move. The goal is then to determine strate-
2
duetomeasurementerror,butalsoallowsustospecifytransitions gies which optimise the probability of some property being

n withprobabilitiesknowntobeidentical,therebylendingfurther satisfied. Originally introduced in [JL91], interval Markov
a expressivity.
chains have recently elicited considerable attention: see for
J The focus of this paper is reachability in AIMCs. We study
example references [SVA06], [CHS08] and [BLW13], which
1 the qualitative, exact quantitative and approximate reachability
studythecomplexityofmodelcheckingbranching-andlinear-
1 problem, as well as natural subproblems thereof, and establish
several upper and lower bounds for their complexity. We prove time properties, as well as [DLL+11], where the focus is on

] the exact reachability problem is at least as hard as the famous consistency and refinement.
C square-root sum problem, but, encouragingly, the approximate While IMCs are very natural for modelling uncertainty in
C version lies in NP if the underlying graph is known, whilst the
stochastic dynamics, they lack the expressivity necessary to
restrictionoftheexactproblemtoaconstantnumberofuncertain
s. edges is in P. Finally, we show that uncertainty in the graph capture dependencies between transition probabilities arising
c structure affects complexity by proving NP-completeness for outofdomain-specificknowledgeoftheunderlyingreal-world
[
the qualitative subproblem, in contrast with an easily-obtained system. Such a dependency could state for example that,
1 upper bound of P for the same subproblem with known graph although the probabilities of some set of transitions are only
v structure. known to lie within a given interval, they are all identical.
6
Disregardingthisinformationandstudyingonlyadependence-
9 I. INTRODUCTION
free IMC is impractical, as allowing these transitions to
9
2 Discrete-time Markov chains are a well-known stochastic vary independently of one another results in a vastly over-
0 model, one which has been used extensively to reason about approximated space of possible behaviours.
. software systems [CY95], [HJ94], [RKNP04]. They comprise Therefore, in the present paper we propose augmented
1
0 a finite set of states and a set of transitions labelled with interval Markov chains (AIMCs), a generalisation of IMCs
7 probabilities in such a way that the outgoing transitions from which allows for dependencies of this type to be described.
1 each state form a distribution. They are useful for modelling We study the effect of this added expressivity through the
:
v systemswith inherentlyprobabilisticbehaviour,aswell asfor prism of the (existentially quantified) reachability problem
i abstracting complexity away from deterministic ones. Thus, under the once-and-for-all interpretation. Our results are the
X
it is a long-standing interest of the verification community to following.First,weshowthatthefullproblemishardforboth
r
a developlogicsfordescribingpropertiesconcerningrealiability the famoussquare-rootsum problem(Theorem6) and for the
of software systems and to devise verification algorithms for class NP (Theorem 3). The former hardness is present even
these properties on Markov chains and their related generali- when the underlying graph structure is known and acyclic,
sations, such as Markov decision processes [Bel57], [Put14]. whilst the latter arises even in the qualitative subproblem
One well-known such generalisation is motivated by how whentransitionintervalsareallowedtoincludezero,rendering
the assumption of precise knowledge of a Markov chain’s the structure uncertain. Second, assuming known structure,
transition relation often fails to hold. Indeed, a real-world we show the approximate reachability problem to be in NP
system’sdynamicsarerarelyknownexactly,duetoincomplete (Theorem 11). Third, we show that the restriction of the
information or measurement error. The need to model this reachability problem to a constant number of uncertain (i.e.
uncertaintyandtoreasonaboutrobustnessunderperturbations interval-valued) transitions is in P (Theorem 4).
in stochastic systems naturally gives rise to interval Markov
II. PRELIMINARIES
chains (IMCs). In this model, uncertain transition probabili-
ties are constrained to intervals, with two different semantic A. Markov chains
interpretations. Under the once-and-for-all interpretation, the Adiscrete-timeMarkovchainorsimplyMarkovchain(MC)
given interval Markov chain is seen as representing an un- isa tuple M =(V,δ) whichconsistsofa finite setof vertices
countablyinfinite collection of Markovchains refining it, and or states V and a one-step transition function δ :V2 [0,1]
→
such that for all v V, we have δ(v,u) = 1. B. First-order theory of the reals
∈ u∈V
For the purposes of specifying MarkovPchains as inputs to We denote by the first-order language R +, ,0,1,<
decision problems, we will assume δ is given by a square L h ×
,= . Atomic formulas in this language are of the form
matrix of rational numbers. The transition function gives i
P(x ,...,x ) = 0 and P(x ,...,x ) > 0 for P
rise to a probability measure on Vω in the usual way. We 1 n 1 n ∈
Z[x ,...,x ] a polynomial with integer coefficients. We de-
denote the probability of reaching a vertex t starting from a note1by Th(nR) the first-order theory of the reals, that is, the
vertex s in M by PM(s ։ t). The structure of M is its setofallvalidsentencesinthelanguage .LetTh∃(R)bethe
underlying directed graph, with vertex set V and edge set L
existential first-order theory of the reals, that is, the set of all
E = (u,v) V2 : δ(u,v) = 0 . Two Markov chains with
{ ∈ 6 } valid sentences in the existential fragment of . A celebrated
the same vertex set are said to be structurally equivalent if L
result[Tar51]isthat admitsquantifierelimination:eachfor-
their edge sets are identical. L
mula φ (x¯) in is equivalentto some effectivelycomputable
1
L
formula φ (x¯) which uses no quantifiers. This immediately
AnintervalMarkovchain(IMC)generalisesthenotionofa 2
entails the decidability of Th(R). Tarski’s original result
Markovchain.Formally,itisapair(V,∆)comprisingavertex
setV andatransitionfunction∆fromV2 tothesetInt of had non-elementary complexity, but improvements followed,
[0,1]
culminating in the detailed analysis of [Ren92]:
intervals contained in [0,1]. For the purposes of representing
an input IMC, we will assume that each transition is given Theorem 1. 1) Th(R) is complete for 2-EXPTIME.
by a lower and an upper bound, together with two boolean 2) Th∃(R) is decidable in PSPACE.
flags indicating the strictness of the inequalities. A Markov 3) If m N is a fixed constant and we consider only
chain M =(V,δ) is said to refine and interval Markov chain existen∈tial sentences where the number of variables is
= (V,∆) with the same vertex set if δ(u,v) ∆(u,v) bounded above by m, then validity is decidable in P.
M ∈
for all u,v V. We denote by [ ] the set of Markov chains
whichrefine∈ .AnIMC’sstrucMtureissaidtobeknownifall We denote by R the class, introduced in [SŠ11], which
elementsof[M]arestructurallyequivalent.Moreover,ifthere lies between NP∃and PSPACE and comprisesall problems
exists some ǫM>0 such that for all M =(V,δ) [ ] and all reducible in polynomial time to the problem of deciding
u,v V, either δ(u,v) = 0 or δ(u,v) > ǫ, th∈enMthe IMC’s membership in Th∃(R).
∈
structure is ǫ-known. An IMC can have known structure but
C. Square-root sum problem
notǫ-knownstructureforexampleby havinganedgelabelled
with an open interval whose lower bound is 0. The square-root sum problem is the decision problem
where, given r ,...,r ,k N, one must determine whether
1 m
An augmented interval Markov chain (AIMC) generalises ∈
√r1 + + √rm k. Originally posed in [O’R81], this
the notion of an IMC further by equipping it with pairs of ··· ≥
problemarises naturally in computationalgeometry and other
edges whose transition probabilities are required to be identi-
contexts involving Euclidean distance. Its exact complexity
cal. Formally, an AIMC is a tuple (V,∆,C), where (V,∆) is
is open. Membership in PSPACE is straightforward via a
an IMC and C V4 is a set of edge equality constraints. A
reduction to the existential theory of the reals. Later this was
⊆
Markov chain (V,δ) is said to refine an AIMC (V,∆,C) if it
sharpened in [ABKPM09] to PosSLP, the complexity class
refinesthe IMC(V,∆)andforeach(u,v,x,y) C, wehave
whose complete problem is deciding whether a division-free
∈
δ(u,v)=δ(x,y). We extend the notation [ ] to AIMCs for
arithmeticcircuitrepresentsa positivenumber.Thisclass was
M
the set of Markov chains refining .
introducedandboundedabovebythefourthlevelofthecount-
M
inghierarchyCHinthesamepaper.However,containmentof
The reachability problem for AIMCs is the problem of
the square-root sum problem in NP is a long-standing open
deciding, given an AIMC = (V,∆,C), an initial vertex
M question,originallyposedin[GGJ76],andtheonlyobstacleto
s V, a target vertex t V, a threshold τ [0,1] and a
∈ ∈ ∈ provingmembershipin NPforthe exactEuclideantravelling
relation , , whether there exists M [ ] such that
PM(s։∼t∈) {≤τ.≥T}he qualitative subproblem∈is tMhe restriction salesman problem. This highlights a difference between the
∼ familiar integer model of computation and the Blum-Shub-
of the reachability problem to inputs where τ 0,1 .
∈{ } SmaleRealRAMmodel[BSS89],underwhichthesquare-root
Finally, in the approximate reachability problem, we are sumisdecidableinpolynomialtime[Tiw92].Seealso[EY09]
given a (small) rational number ε and a reachability problem for more background.
instance. If is , our procedure is required to accept if
there exists ∼some≥refining Markov chain with reachability III. QUALITATIVECASE
probability greater than τ +ε/2, it is required to reject if all
In this section, we will focuson the qualitativereachability
refining Markovchains have reachabilityprobabilityless than
problem for AIMCs. We show that, whilst membership in
τ ε/2, and otherwise it is allowed to do anything.Similarly
P is straightforward when the underlying graph is known,
−
if is . Intuitively, this is a promise problem: in the given
uncertainty in the structure renders the qualitative problem
∼ ≤
instance the optimal reachability probability is guaranteed to
NP-complete.
be outside the interval [τ ε/2,τ +ε/2].
−
Theorem 2. The qualitative reachability problem for AIMCs For all i 1,...,k ,
∈{ }
with known structure is in P.
1 1
∆(v ,S)=∆(v ,ϕ )= , .
Proof. LetthegivenAIMCbe ands,ttheinitialandtarget m m i (cid:20)k+1 k+1(cid:21)
M
vertices, respectively. Since the structure G = (V,E) of
M Finally, ∆(S,S) = ∆(F,F) = [1,1]. For all other pairs of
is known, the qualitative reachability problem can be solved
vertices u,v, we have ∆(u,v)=[0,0].
simply using standard graph analysis techniques on G. More
The edge equality constraints are:
precisely, for any M [ ], PM(s ։ t) = 1 if and only if
∈ M
there is no path in G which starts in s, does not enter t and C = (v ,x ,x ,v ),(v ,x ,x ,F)
i−1 i i i i−1 i i
{ }
ends in a bottom strongly connected component which does i=1[,...,m
not contain t. Similarly, PM(s ։ t) = 0 if and only if there
Intuitively, the sequence of ‘diamonds’ comprised by
is no path from s to t in G.
v ,...,v and the vertices corresponding to literals is a
0 m
Theorem 3. The qualitative reachability problem for AIMCs variable setting gadget. Choosing transition probabilities
is NP-complete. δ(vi−1,xi)=δ(xi,vi)=1, and hencenecessarily δ(xi,F)=
0, corresponds to setting x to true, whereas δ(v ,x ) =
i i−1 i
Proof. Membership in NP is straightforward. The equiva-
δ(x ,v ) = 1 and δ(x ,F) = 0 corresponds to setting x
i i i i
lence classes of [ ] under structure equivalence are at most to false. On the other hand, the branching from v into
2n2, where n is Mthe number of vertices, since for each pair ϕ ,...,ϕ and the edges from clauses to their literalsmmakes
1 k
(u,v) of vertices, either an edge (u,v) is present in the up the assignment testing gadget. Assigning non-zero proba-
structure or not. This upper bound is exponential in the size bility to the edge (ϕ ,l ) correspondsto selecting the literal
i i,j
of the input. Thus, we can guess the structure of the Markov l as witness that the clause ϕ is satisfied.
i,j i
chaininnondeterministicpolynomialtimeandthenproceedto Formally, we claim that there exists a Markov chain M
solveaninstanceofthequalitativereachabilityproblemonan [ ]suchthatPM(v ։S)=1ifandonlyifϕissatisfiable∈.
0
AIMCwithknownstructureinpolynomialtimebyTheorem2. MSupposefirstthatϕissatisfiableandchoosesomesatisfying
We now proceed to show NP-hardness using a reduction assignment σ : x ,...,x 0,1 . Let M = (V,δ)
1 m
from 3-SAT. Suppose we are given a propositionalformula ϕ [ ]betherefini{ngMarkovc}ha→inw{hich}assignsthefollowin∈g
in 3-CNF: trMansition probabilities to the interval-valued edges of .
M
ϕ ϕ1 ϕ2 ϕk, First, let
≡ ∧ ∧···∧
where each clause is a disjunction of three literals: δ(v ,x )=δ(x ,v )=δ(x ,F)=σ(x ),
i−1 i i i i i
δ(v ,x )=δ(x ,v )=δ(x ,F)=1 σ(x )
ϕ l l l . i−1 i i i i i
i i,1 i,2 i,3 −
≡ ∨ ∨
for all i 1,...,m . Second, for each clause ϕ , choose
Let the variables in ϕ be x ,...,x . i
1 m ∈ { }
some literal l which is true under σ and set δ(ϕ ,l )=1
Let =(V,∆,C) be the following AIMC, also depicted i,j i i,j
M and consequentlyδ(ϕ ,l)=0 for the other literals l. Now we
in Figure 1. The vertex set has 3m+k+3 vertices: i
can observe that the structure of M has two bottom strongly-
V = x ,...,x ,x ,...,x connected components, namely S and F , and moreover,
1 m 1 m
{ϕ1,...,ϕk } F is unreachable from v0. There{fo}re, PM{(v0}։S)=1.
∪{ } Conversely, suppose there exists some M = (V,δ) [ ]
∪{S,F}, such that PM(v ։ S) = 1. We will prove that ϕ∈haMs a
0
v ,...,v
0 m satisfying assignment. For each i 1,...,m , write
∪{ }
∈{ }
that is, one vertex for each possible literal over the given
p =δ(v ,x )=δ(x ,v )=δ(x ,F),
i i−1 i i i i
variables,onevertexforeachclause,twospecialsinkvertices
1 p =δ(v ,x )=δ(x ,v )=δ(x ,F).
S,F (success and failure) and m + 1 auxiliary vertices. − i i−1 i i i i
Throughaslightabuseofnotation,we usex ,x toreferboth Notice that
i i
to the literals over the variable x and to their corresponding
i PM(v x Fω)=PM(v x Fω)=p (1 p ),
verticesin ,andsimilarly,ϕi denotesboththeclauseinthe 0 1 0 1 1 − 1
M
formula and its corresponding vertex. so we can concludep 0,1 ,otherwise PM(v ։S)=1,
1 0
The transitions are the following. For all i 1,...,m , a contradiction. If p =∈1{, then} 6
∈ { } 1
we have:
PM(v x v x Fω)=PM(v x v x Fω)=p (1 p ),
0 1 1 2 0 1 1 2 2 2
−
∆(v ,x )=∆(v ,x )=∆(x ,v )=
i−1 i i−1 i i i
whereas if p =0, then
1
∆(x ,F)=∆(x ,F)=∆(x ,v )=[0,1].
i i i i
PM(v x v x Fω)=PM(v x v x Fω)=p (1 p ).
0 1 1 2 0 1 1 2 2 2
For all i 1,...,k and j 1,...,3 , we have: −
∈{ } ∈{ } Either way, we must have p 0,1 to ensure PM(v ։
2 0
∆(ϕ ,l )=[0,1]. ∈ { }
i i,j S)=1. Unrollingthisargumentfurthershowsp 0,1 for
i
∈{ }
alli.Inparticular,thereisexactlyonepathfromv tov and u) is independent of the choice of M [ ]. Denote these
0 m
∈ M
ithasprobability1.Letσ bethetruthassignmentx p ,we probabilities by α(w,u). They satisfy the system
i i
→
showthatσsatisfiesϕ.Indeed,ifsomeclauseϕ isunsatisfied
i α(w,u)=δ(w,u)+ δ(w,w′)α(w′,u),
under σ, then its three literals l ,...,l are all unsatisfied,
i,1 i,3
so δ(l ,F) > 0 for all j = 1,...,3. Moreover, for at least w∈W^,u∈U wX′∈W
i,j
oneofthesethreeliterals,sayl ,wewillhaveδ(ϕ ,l )>0, whichislinearandthereforeeasytosolvewithGaussianelim-
i,1 i i,1
so the path v ...v ϕ l Fω will have non-zero probability: ination.Thus,assumethatwehavecomputedα(w,u) Qfor
0 m i i,1 ∈
all w W and u U.
1 ∈ ∈
PM(v ...v ϕ l Fω)= δ(ϕ ,l )δ(l ,F)=0, Similarly, for all u ,u U, write β(u ,u ) for the
0 m i i,1 k+1 i i,1 i,1 6 probabilityofu u .1No2tic∈ethatβ(u ,u )is1a p2olynomial
1 2 1 2
which contradicts PM(v ։ S) = 1. Therefore, σ satisfies of degree at most 1 over the variables x, given by
0
ϕ, which completes the proof of NP-hardness and of the
β(u ,u )=δ(u ,u )+ δ(u ,w)α(w,u ).
Theorem. 1 2 1 2 1 2
wX∈W
Thus, assume we have computed symbolically β(u ,u )
1 2
IV. CONSTANT NUMBEROF UNCERTAIN EDGES Q[x] for all u ,u U. ∈
1 2
∈
We now shift our attention to the subproblem of AIMC Finally, for each u U, let y(u) be a variable and write y
∈
reachability which arises when the number of interval-valued for the vector of variables y(u) in some order. Consider the
transitions is fixed, that is, bounded above by some absolute followingformulain the existentialfirst-orderlanguageof the
constant. Our result is the following. real field:
ϕ x y.ϕ ϕ ϕ ,
Theorem 4. Fix a constant N N. The restriction of the ≡∃ ∃ 1∧ 2∧ 3
∈
reachability problem for AIMCs to inputs with at most N where
interval-valued transitions lies in P. Hence, the approximate
ϕ y(t)=1 y(u)= β(u,u′)y(u′),
reachability problem under the same restriction is also in P. 2 ≡ ∧
u∈U^\{t} uX′∈U
Proof. Let = (V,∆,C) be the given AIMC and suppose ϕ y(s) τ,
we wish toMdecide whether there exists M [ ] such that 3 ≡ ∼
PM(s։t) τ. LetU V bethesetofver∈ticeMswhichhave and ϕ1 is as above. Intuitively, ϕ1 states that the variables
at least one∼interval-valu⊆ed outgoing transition, together with x descibe a Markov chain in [ ], ϕ2 states that y gives
M
s and t: the reachability probabilities from U to t, and ϕ3 states that
the reachability probability from s to t meets the required
U = s,t u V : v V.∆(u,v) is not a singleton . threshold τ. The problem instance is positive if and only
{ }∪{ ∈ ∃ ∈ }
Notice that U N +2=const. Write W =V U, so that if ϕ is a valid sentence in the existential theory of the
| |≤ \ reals, which is decidable. Moreover, the formula uses exactly
U,W is a partition of V.
{ } 2U 2(N +2) = const variables, so by Theorem 1, the
Let x be a vector of variables, one for each interval-valued | | ≤
problem is decidable in polynomial time, as required.
transition of . For vertices v ,v , let δ(v ,v ) denote the
1 2 1 2
M
correspondingvariableinxifthetransition(v ,v )isinterval- Notice that removing the assumption of a constant number
1 2
valued, and the only element of the singleton set ∆(v ,v ) of interval-valued transitions only degrades the complexity
1 2
otherwise. Let ϕ be the following propositional formula upperbound,butnotthedescribedreductiontotheproblemof
1
over the variables x which captures the set of ‘sensible’ checkingmembershipin Th∃(R). As an immediate corollary,
assignments: we have:
ϕ δ(v ,v )=1 Theorem 5. The reachability problem and the approximate
1 1 2
≡
v1^∈V vX2∈V reachability problem for AIMCs are in ∃R.
δ(v1,v2) ∆(v1,v2) [0,1] Note that Theorem 5 can be shown much more easily,
∧ ∈ ∩
v1,^v2∈V without the need to consider separately U-vertices and W-
δ(a,b)=δ(c,d). verticesasintheproofofTheorem4.Itissufficienttouseone
∧ ^ variable per interval-valued transition to capture its transition
(a,b,c,d)∈C
probability as above and one variable per vertex to express
its reachability probability to the target. Then write down
There is clearly a bijection between [ ] and assignments of an existentially quantified formula with the the usual system
M
x which satisfy ϕ . of equations for reachability in a Markov chain obtained by
1
For vertices v ,v , use the notation v v to denote the conditioning on the first step from each vertex. While this
1 2 1 2
event ‘v is reached from v along a path consisting only of easilygivesthe R upperbound,itusesatleast V variables,
2 1
∃ | |
vertices in W, with the possible exception of the endpoints so it is insufficient for showing membership in P for the
v ,v ’. Notice that for all u U and w W, PM(w restriction to a constant number of interval-valuedtransitions.
1 2
∈ ∈
1 1
F S
p 1 p2 1 als [0,1]
1 − 1 − −1p −pm 1 liter ϕ
3 k+1 o 1
t
x x x x
1 2 3 m
p 1 p p 2 p p 3 p
1 2 m 1
v0 v1 v2 ... vm k+1 ...
1
1 p 1 1 p 2 1 p m k+1
−p x 1 − −p x 1 − −p x x 1 −
1 1 2 2 3 3 m
als ϕk
p3 er
p1 p2 p m olit [0, 1]
t
F
1
Figure1. ConstructionusedinTheorem3forshowingNP-hardnessofthequalitative AIMCreachabilityproblem.ThesinkF isduplicatedtoavoidclutter.
V. HARDNESS FORSQUARE-ROOT SUMPROBLEM Let M be a positive integer large enough to ensure
3√r
x∗ := (0,1).
2M ∈
In this section, we give a lower bound for the AIMC
Then choose a positive integer N large enough, so that
reachabilityproblem.This boundremains in place even when
the structure of the AIMC is ǫ-known and acyclic, except for 4M
α:= (0,1),
the self-loops on two sink vertices. N ∈
16M3
β := (0,1),
Theorem 6. The AIMC reachability problem is hard for the 27rN ∈
square-rootsumproblem,evenwhenthestructureoftheAIMC √r β
p := + (0,1).
isǫ-knownandisacyclic,exceptfortheself-loopsontwosink opt
N 4 ∈
vertices.
Proof. The reduction is based on the gadget depicted in Now, a straightforward calculation shows
Figure 2. It is an AIMC with two sinks, S and F (success
P(a։S)=P(ab c d eS)+P(ab c S)
and failure), each with a self-loop with probability 1, and 1 1 1 2 2
12 vertices: a,b1,...,b4,c1,...,c4,d1,d4,e . The structure +P(ab3c3S)+P(ab4c4d4S)
{ }
is acyclic and comprises four chains leading to S, namely, βx2(1 x) β(1 x)
= − + −
ab1c1d1eS, ab2c2S, ab3c3S and ab4c4d4S. From each vertex 4 4
other than a and S there is also a transition to F. αx βx(1 x)
+ + −
The probabilities are as follows. The transition (b ,c ) has 4 4
3 3
probabilityα,whilst(b ,c ),(b ,c ),(b ,c )haveprobability αx βx3+β
1 1 2 2 4 4 = − .
β, for rationals α,β to be specified later. Consequently, 4
the remaining outgoing transition to F out of each bi has Analysing the derivative of this cubic, we see that P(a։S)
probability 1 α or 1 β. The transitions (a,bi) for increases on [0,x∗), has its maximum at x = x∗ and then
− −
i = 1,...,4 all have probability 1/4. Finally, the transitions decreases on (x∗,1]. This maximum is
(c ,F), (c ,F), (c ,S), (c ,F), (d ,e), (d ,S) and (e,S)
1 2 3 4 1 4 αx∗ β(x∗)3+β √r β
are interval-valued and must all have equal probability in
− = + =popt.
any refining Markov chain. Assign the variable x to the 4 N 4
probability of these transitions. The interval to which these Thus,ifwechoosesomeclosedintervalwhichcontainsx∗ but
transition probabilities are restricted (i.e. the range of x) not 0 and 1 to be the range of x, then the gadget described
is to be specified later. Consequently, the remaining tran- thusfarwillhaveǫ-knownstructureandmaximumreachability
sitions (c ,d ),(d ,F),(e,F),(c ,S), (c ,F),(c ,d ),(d ,F) probabilityfroma to S givenby √r scaled by a constantand
1 1 1 2 3 4 4 4
are also interval-valued, with probability 1 x. offset by another constant.
−
Now,supposewewishtodecidewhether√r + +√r x in their appropriate intervals such that P(x ,...,x ) τ
1 m i 1 n
··· ≥ ≥
k for given positive integers r ,...,r and k. Construct a if and only if there exists a refining Markov chain such that
1 m
gadgetasaboveforeachr .Theconstantsα,N,M areshared P(v ։S) (τ β)/N.
i 0
≥ −
across the gadgets, as are the sinks S,F, but each gadget has
VI. APPROXIMATE CASE
its own constant β in place of β, and its own copy of each
i
non-sinkvertex.Theedgeequalityconstraintsarethesameas In this section, we focus on the approximate reachability
abovewithineachgadget,andtherearenoequalityconstraints problem for AIMCs. To obtain our upper bound, we will use
across gadgets.Assign a variable x to those edgesin the i-th a result from [Cha12].
i
gadget which in the description above were labelled x, and
Definition 8. If M =(V,δ ) and M =(V,δ ) are Markov
1 1 2 2
choose a rangefor x as described abovefor x. Finally,add a
i chains with the same vertex set, then their absolute distance
newinitialvertexv ,withmequiprobableoutgoingtransitions
0 is
to the a-vertices of the gadgets.
In this AIMC, the probability of v ։ S is given by the dist (M ,M )= max δ (u,v) δ (u,v) .
0 A 1 2 1 2
u,v∈V {| − |}
multivariate polynomial
Lemma 9. (Appears in [Cha12].) Let M = (V,δ ) and
1 m αx β x3+β 1 1
i− i i i, M2 =(V,δ2)bestructurallyequivalentMarkovchains,where
mXi=1 4 nδ =(u,|Vv)| anǫd.Lfoertaalllsoud,v ∈dVis,tw(eMha,vMe ei)thaenrdδfi1x(utw,vo)v=ert0icoesr
whose maximum value on [0,1]m is 1 ≥ ≤ A 1 2
s,t V. Then
∈
m
m1 (cid:18)√Nri + β4i(cid:19). PM1(s։t) PM2(s։t) 1+ d 2n 1.
Xi=1 − ≤(cid:18) ǫ d(cid:19) −
(cid:12) (cid:12) −
(cid:12) (cid:12)
Therefore, √r + +√r k if and only if there exists a We will also need the following well-known inequality:
1 m
··· ≥
refining Markov chain of this AIMC with
Lemma 10. For all x 1 and r [0,1], we have
m ≥− ∈
k 1 β
P(v ։S) + i, (1+x)r 1+rx.
0 ≥ mN m 4 ≤
Xi=1
Now we proceed to prove our upper bound.
so the reduction is complete.
Theorem 11. The approximate reachability problem for
Remark 7. It is easy to see that if we are given an acyclic AIMCs with ǫ-known structure is in NP.
AIMC with the interval-valued edges labelled with variables,
thereachabilityprobabilitiesfromallverticestoasingletarget Proof. Let be the given AIMC and let ǫ > 0 be a
M
vertex are multivariate polynomials and can be computed lower bound on all non-zero transitions across all M [ ].
∈ M
Suppose we are solving the maximisation version of the
symbollically with a backwards breadth-first search from the
target. Then optimising reachability probabilities reduces to problem: we are given vertices s,t and a rational ε > 0, we
optimisingthe valueof a polynomialovergivenrangesfor its mustaccept if PM(s։t)>τ+ε/2 for some M [ ] and
variables. we must reject if PM(s։t)<τ ε/2 for all M∈ M[ ].
− ∈ M
Let n be the number of vertices and let
It is interesting to observe that a reduction holds in the
other direction as well. Suppose we wish to decide whether d:=ǫ 1 (1+ε)−1/2n .
there exist values of x1 I1,...,xn In such that (cid:16) − (cid:17)
∈ ∈
P(x ,...,x ) τ for a given multivariate polynomial P, For each interval-valued transition, split its interval into at
1 n
≥
intervals I ,...,I [0,1] and τ Q. Notice that P most 1/d intervals of length at most d each. For example,
1 n
⊆ ∈
can easily be written in the form P(x ,...,x ) = β + [l,r] partitions into [l,l +d),[l +d,l +2d),...,[l +kd,r],
1 n
N m α Q (x ,...,x ), where N > 0, α ,...,α where k is the largest natural number such that l+kd r.
n(0o,nP1-)eim=a1pretyicporniosdtua1ncttsofsutecrnhmtshdatraPwnmi=fr1oαmi ≤nj=11,{xe1ajc,h(1Q−ixmijs)}∈a, ChMallit⊆he[eMnd]pboeintthsedseefitnoifnMg tahreksoevscuhbaiinntserrveafilnsignrgidMposiunctsh.≤tLheatt
andβ is a (possiblynegative)constantterSm.For example,the the probabilities of all interval-valued transitions are chosen
monomial 2x x x has a negative coefficient, so rewrite it from among the grid points. Observe that for all M [ ],
1 2 3 1
as 2(1 x−)x x +2(1 x )x +2(1 x ) 2. Do this there exists M such that dist (M ,M ) d.∈ M
1 2 3 2 3 3 2 A 1 2
− − − − ∈hMi ≤
to all monomials with a negative coefficient, then choose an Our algorithm showing membership in NP will be the
appropriately large N to obtain the desired form. following. We will choose M nondeterministically
Now it is easy to construct an AIMC with two sinks S,F and compute p := PM(s ։ t)∈ushinMgiGaussian elimination.
and a designated initial vertex v where the probability of Then if p τ ε/2, we will accept, and otherwise we will
0
v ։S is m α Q . We use a chain to represent each Q , reject. ≥ −
0 i=1 i i i
and then brPanch from v0 into the first vertices of the chains To complete the proof, we need to argue two points. First,
with distribution given by the α . There exist values of the that is at most exponentially large in the size of the
i
hMi
β 1 x x
b1 c1 − d1 e
β
1/4
b2 1 x c2 x
− 1 x
β −
1 x x
1/4 −β 1 −
a 1 F 1 x S
−
1/4 α 1 1 x
1
1 − −x −x
β
1/4 b3 1− x c3 x
α
b4 c4 d4
β 1 x
−
Figure2. Gadgetforreduction fromsquare-rootsumproblem toAIMCreachability.
input, so that M can indeed be guessed in nondeterministic way that for all M [ ], there is some M with
1 2
polynomial time. Second, that if for all M we have dist (M ,M ) d.∈In Mparticular, if PM2(s ։t∈) <hMτi ε/2
A 1 2
PM(s։t)<τ ε/2,thenitis safeto rejec∈t,thhMatiis, thereis for all M ≤ , then certainly PM1(s ։ t) < τ +−ε/2
2
no M′ with PM′−(s։t) τ +ε/2. (Note that the procedure for all M ∈[hM],iso it is safe to reject. This completes the
1
≥ ∈ M
is obviously correct when it accepts.) proof.
Tothefirstpoint,weapplyLemma10withx= ε/(ε+1)
−
and r =1/2n:
REFERENCES
1/2n
ε 1 ε
(1+ε)−1/2n = 1 1
(cid:18) − 1+ε(cid:19) ≤ − 2n1+ε [ABKPM09] EricAllender,PeterBürgisser,JohanKjeldgaard-Pedersen,and
PeterBroMiltersen. Onthecomplexityofnumericalanalysis.
and hence, SIAMJournalonComputing, 38(5):1987–2006, 2009.
[Bel57] Richard Bellman. A Markovian decision process. Technical
1 1 1+ε 1 1 report, DTICDocument,1957.
d−1 =ǫ−1 2n = 2n 1+ .
1 (1+ε)−1/2n ≤ ǫ ε ǫ (cid:18) ε(cid:19) [BLW13] MichaelBenedikt,RastislavLenhardt,andJamesWorrell. LTL
− modelcheckingofintervalMarkovchains. InToolsandAlgo-
This upper bound is a polynomial in n, 1/ε and 1/ǫ, and rithms fortheConstruction andAnalysis ofSystems (TACAS),
pages 32–46.Springer,2013.
hence at most exponential in the length of the input data.
[BSS89] Lenore Blum, Mike Shub, and Steve Smale. On a the-
Therefore, for each interval-valued transition, we can write ory of computation and complexity over the real numbers:
downusing onlypolynomiallymanybits whichgrid pointwe NP-completeness,recursivefunctionsanduniversalmachines.
Bulletin (New Series) of the American Mathematical Society,
wish to use for the probability of that transition. Since the
21(1):1–46, 1989.
numberof transitionsis polynomialin the lengthof the input, [Cha12] Krishnendu Chatterjee. Robustness of structurally equivalent
itfollowsthatanelementof maybespecifiedusingonly concurrent parity games. In Proceedings of the 15th In-
hMi ternational Conference on Foundations of Software Science
polynomially many bits, as required.
and Computational Structures, FOSSACS’12, pages 270–285,
To the second point, consider M1,M2 [ ] such that Berlin, Heidelberg, 2012.Springer-Verlag.
dist (M ,M ) d. Then by Lemma 9, we∈haMve [CHS08] Krishnendu Chatterjee, Tom Henzinger, and Koushik Sen.
A 1 2
≤ Model-checking omega-regular properties of interval Markov
PM1(s։t) PM2(s։t) chains. InRobertoM.Amadio,editor,FoundationsofSoftware
− ScienceandComputationStructure(FoSSaCS),pages302–317,
(cid:12)(cid:12) d 2n (cid:12)(cid:12) March2008.
1+ 1
[CY95] Costas Courcoubetis andMihalis Yannakakis. Thecomplexity
≤(cid:18) ǫ d(cid:19) −
− of probabilistic verification. Journal of the ACM (JACM),
ǫ 2n 42(4):857–907, 1995.
= 1 [DLL+11] BenoîtDelahaye, KimG.Larsen,AxelLegay,MikkelL.Ped-
(cid:18)ǫ(1+ε)−1/2n(cid:19) −
ersen, and Andrzej Wa˛sowski. Decision problems for interval
= ε. Markovchains. InInternational ConferenceonLanguageand
Automata Theory and Applications, pages 274–285. Springer,
Inotherwords,changingthetransitionprobabilitiesbyatmost 2011.
[EY09] Kousha Etessami and Mihalis Yannakakis. Recursive Markov
d does not alter the reachability probability from s to t by
chains,stochasticgrammars,andmonotonesystemsofnonlin-
more than ε. However, recall that we chose in such a earequations. JournaloftheACM(JACM),56(1):1, 2009.
hMi
[GGJ76] M. R. Garey, R. L. Graham, and D. S. Johnson. Some NP-
complete geometric problems. In Proceedings of the Eighth
AnnualACMSymposiumonTheoryofComputing,STOC’76,
pages 10–22,NewYork,NY,USA,1976.ACM.
[HJ94] HansHanssonandBengtJonsson. Alogicforreasoningabout
time and reliability. Formal aspects of computing, 6(5):512–
535,1994.
[JL91] Bengt Jonssonand Kim Guldstrand Larsen. Specification and
refinement of probabilistic processes. In Logic in Computer
Science, 1991. LICS’91., Proceedings of Sixth Annual IEEE
Symposium on,pages 266–277.IEEE,1991.
[O’R81] Joseph O’Rourke. Advanced problem 6369. Amer. Math.
Monthly, 88(10):769, 1981.
[Put14] Martin L. Puterman. Markov decision processes: discrete
stochastic dynamic programming. JohnWiley&Sons,2014.
[Ren92] JamesRenegar.Onthecomputationalcomplexityandgeometry
of the first-order theory of the reals. Part I: Introduction. Pre-
liminaries. The geometry of semi-algebraic sets. The decision
problem for the existential theory of the reals. Journal of
Symbolic Computation, 13(3):255–299,1992.
[RKNP04] Jan J. M. M. Rutten, Marta Kwiatkowska, Gethin Norman,
and David Parker. Mathematical techniques for analyzing
concurrent andprobabilistic systems. American Mathematical
Soc.,2004.
[SŠ11] Marcus Schaefer and Daniel Štefankovicˇ. Fixed points, Nash
equilibria, and the existential theory of the reals. Theory of
Computing Systems,pages1–22,2011.
[SVA06] Koushik Sen, Mahesh Viswanathan, and Gul Agha. Model-
checking Markov chains in the presence of uncertainties. In
Holger Hermanns and Jens Palsberg, editors, TACAS, pages
394–410, 2006.
[Tar51] Alfred Tarski. A decision method for elementary algebra and
geometry. 1951.
[Tiw92] PrasoonTiwari. Aproblem thatiseasiertosolveontheunit-
cost algebraic RAM. Journal of Complexity, 8(4):393 – 397,
1992.