An Automaton Learning Approach to Solving Safety Games over Infinite Graphs

0.57 MB·

by  Daniel Neider

An Automaton Learning Approach to Solving
Safety Games over Infinite Graphs
Daniel Neider Ufuk Topcu
Department of Electrical Engineering Department of Electrical and Systems Engineering
University of California at Los Angeles, USA University of Pennsylvania, USA
Abstract—We propose a method to construct finite-state re- We focuson games with safety specifications,which already
active controllers for systems whose interactions with their capture practically interesting properties (e.g., safety and
6 adversarial environment are modeled by infinite-duration two- bounded-horizon reachability). However, games over infinite
1 player games over (possibly) infinite graphs. The proposed
graphs require special attention on the representation and
0 method targets safety games with infinitely many states or with
2 such a large number of states that it would be impractical— manipulation of the underlying graph structure. Hence, one of
if not impossible—for conventional synthesis techniques that our main contributions is a symbolic representation of safety
n
work on the entire state space. We resort to constructing finite- games, called rational safety games, that follows the idea of
a
state controllers for such systems through an automata learning
J regular model checking [5] in that it represent sets of vertices
approach, utilizing a symbolic representation of the underlying
by regular languages and edges by so-called rational relations.
7 game that is based on finite automata. Throughout the learning
process, the learner maintains an approximation of the winning A straightforward approach to solve (rational) safety games

] region (represented as a finite automaton) and refines it using is computing a winning set for the controlled system (i.e., a
L
different types ofcounterexamples provided by theteacher until safe subset of the vertices in which the system can force to
F asatisfactorycontrollercanbederived(ifoneexists).Wepresent
remain). Once a winning set is computed, a strategy for the
. a symbolic representation of safety games (inspired by regular
s system is determined by choosing its moves (in each of its
c model checking), propose implementations of the learner and
[ teacher, and evaluate their performance on examples motivated turns) to stay inside the set, which is possible regardless of the
by robotic motion planning in dynamic environments. moves of the environment. We use winning sets as a proxy for
1
an actual controller, and the objective of the learning task is
v I. INTRODUCTION
the construction of a winning set. In fact, learning a winning
0
6 We propose an automata learning-based method to construct setratherthanacontrollerresultsinmorepermissivestrategies
6 reactive controllers subject to safety specifications. We model (and potentially smaller solutions) as the moves of the system
1 the interaction between a controlled system and its possibly do not need to be fixed during the learning process.
0
adversarial environment as a two-player game over a graph [1]. We develop a framework for learning winning sets for
.
1 We consider games over infinite graphs. In this setting, the rational safety games and particular implementations of a
0 conventional techniques for reactive controller synthesis (e.g., teacher and learner. The actual learning works iteratively. In
6
fixed-point computations) are not applicable anymore. There- eachiteration,thelearnerconjecturesawinningset,represented
1
fore, we resort to a learning-based approach for constructing as a deterministic finite automaton. The teacher performs
:
v finite-state reactive controllers for the controlled system. The a number of checks and returns, based on whether the
i
X learning takes place in a setting akin to counterexample-guided conjecture passes the checks, a counterexample. Following
inductive synthesis (CEGIS) [2] between a teacher, who has the ICE learning framework [6] and partially deviating from
r
a knowledge about the safety game in question, and a learner, the classical learning frameworks for regular languages [7],
whose objective is to identify a controller using information [8], the counterexample may be one of the following four
disclosed by the teacher in response to (incorrect) conjectures. types: positive, negative, existential implication and universal
Anaturalcontextfortheproposedmethodisoneinwhichthe implication counterexamples. Based on the response from the
interaction between the controlled system and its environment teacher, the learner updates his conjecture. If the conjecture
is so complex that it can be represented only by graphs with passes all checks (i.e., the teacher returns no counterexample),
infinitely many vertices (e.g., motion planning over unbounded the learning process terminates with the desired controller.
grid worlds) or “practically infinitely many" states (i.e., the A learning-based approach offers several advantages: First,
number of possible configurations is so large that the game even though the underlying game may be prohibitively large,
becomesimpracticalforconventionaltechniques).Additionally, the reactive controller necessary to realize the specifications
in situations where a complete description of the game is not often has a compact representation in practice; for example,
available in a format amenable to existing game solvers [3], depending on the given task specification in a robotic motion
[4], there may still exist human experts (or automated oracles, planning scenario, only a small subset of all possible rich
as in Section IV) who have sufficient insight into how the interactions between the robot and its dynamic environment
controlled system should behave and can act as teacher. overapossiblylargeworkspaceisoftenrelevant.Second,since
learning-basedapproachesusuallyidentify“small"solutions(as strategyf iscalledwinningifanyplayv v ...thatisplayed
σ 0 1
they typically produce intermediate conjectures of increasing accordingtothestrategy(i.e.,thatsatisfiesv =f (v ...v )
i+1 σ 0 i
size), their runtime mainly depends on the size of the solution for all i ∈ N and v ∈ V ) is winning for Player σ. A
i σ
rather than the size of the underlying game. Third, learning- winningstrategyforPlayer0straightforwardlytranslatesintoa
basedapproachesreducethegapbetweenhumandesignersand controller satisfying the given safety specifications and, hence,
construction of reactive controllers by hiding the complexity werestrictourselvestocomputewinningstrategiesforPlayer0.
of the underlying game from the learner. ComputingawinningstrategyforPlayer0isusuallyreduced
Finally,wedemonstratetheuseofouroveralllearning-based to finding a so-called winning set.
framework empirically on a series of examples motivated by Definition 1 (Winning set): For a safety game G=(A,I,F)
robotic motion planning in dynamic environments. overthearenaA=(V ,V ,E),awinningsetisasetW ⊆V
0 1
satisfying (1) I ⊆ W, (2) W ⊆ F, (3) E({v})∩W (cid:54)= ∅ for
RelatedWork: Gamesoverinfinitegraphshavebeenstudied
all v ∈W ∩V (existential closedness), and (4) E({v})⊆W
in the past, predominantly in the case of games over pushdown 0
for all v ∈W ∩V (universal closedness).
graphs [9]. The games we consider here, however, are played 1
By computing a winning set, one immediately obtains a
overaricherclassofgraphsandrequiredifferenttechniquesto
strategy for Player 0: starting in an initial vertex, Player 0
be solved. Also, a constraint-based approach to solving games
simply moves to a successor vertex inside W whenever it is
over infinite graphs has recently been proposed [10].
his turn. A straightforward induction over the length of plays
Learning-based techniques for games over infinite graphs
proves that every play that is played according to this strategy
have already been studied in the context of reachability
stays inside F, no matter how Player 1 plays, and, hence, is
games [11]; in fact, our symbolic representation of safety
won by Player 0 (since I ⊆W ⊆F). A winning set is what
games is a generalization of the representation proposed there.
we want to compute—or, more precisely, learn.
In the context of safety games, recent work [12] has already
Games over infinite arenas require a symbolic representation
demonstratedtheabilityoflearning-basedapproachestoextract
in order to work with them algorithmically. We follow the
small reactive controllers from a priori constructed controllers
idea of regular model checking [5], an approach in verification,
with possibly large number of states. In this work, we by-pass
and represent sets of vertices by regular languages and edges
theaprioriconstructionofpossiblylargereactivecontrollersby
by so-called rational relations. Before we can introduce our
learning (an appropriate representation of) a controller directly.
symbolic representation of safety games, however, we need to
II. RATIONALSAFETYGAMES recap basic concepts and notations of automata theory.
Thissectionrecapsinfinite-duration,two-playersafetygames b) Basics of Automata Theory: An alphabet Σ is a
as well as basic concepts of automata theory and introduces nonempty,finiteset,whoseelementsarecalledsymbols.Aword
rational safety games. over the alphabet Σ is a sequence u = a1...an of symbols
a) Safety Games: We consider safety games (i.e., infinite ai ∈ Σ for i ∈ {1,...,n}; the empty sequence is called
duration two-person games on graphs) as popularized by empty word and denoted by ε. Given two words u=a1...am
McNaughton [1]. A safety game is played on an arena and v = b1...bn, the concatenation of u and v is the word
A = (V0,V1,E) consisting of two nonempty, disjoint sets u·v =uv =a1...amb1...bn. The set of all words over the
V ,V of vertices (we denote their union by V) and a directed alphabet Σ is denoted by Σ∗, and a subset L⊆Σ∗ is called a
0 1
edge relation E ⊆V ×V. In contrast to the classical (finite) language. The set of prefixes of a language L⊆Σ∗ is the set
setting, we allow V and V to be countable sets. As shorthand Pref(L)={u∈Σ∗ |∃v ∈Σ∗: uv ∈L}.
0 1
notation, we write the successors of a set X ⊆V of vertices A nondeterministic finite automaton (NFA) is a tuple A=
as E(X)={y |∃x∈X: (x,y)∈E}. (Q,Σ,q0,∆,F)consistingofanonempty,finitesetQofstates,
We consider safety games with initial vertices, which are an input alphabet Σ, an initial state q ∈ Q, a transition
0
defined as triples G = (A,F,I) consisting of an arena A = relation∆⊆Q×Σ×Q,andasetF ⊆Qoffinalstates.Arun
(V ,V ,E), a set F ⊆V of safe vertices, and a set I ⊆F of of an NFA A on a word u=a ...a is a sequence of states
0 1 1 n
initial vertices. Such safety games are played by two players, q ,...,q such that (q ,a ,q )∈∆ for i∈{1,...,n}. We
0 n i−1 i i
u
named Player 0 and Player 1, as follows: A token is placed on denote this run by A: q −→ q . An NFA A accepts a word
0 n
some initial vertex v ∈I and, in each turn, the player owning u∈Σ∗ if A: q −→u q with q ∈F. The set L(A)={u∈Σ∗ |
0 0
u
the current vertex moves the token to a successor vertex of A: q −→q,q ∈F} is called language of A. A language L is
0
his choice. This process of moving the token is repeated ad said to be regular if there exists an NFA A with L(A)=L.
infinitum, thereby forming an infinite sequence of vertices, Finally, NFA denotes the set of all NFAs over Σ.
Σ
which is called a play. Formally, a play is an infinite sequence A deterministic finite automaton (DFA) is an NFA in which
π = v v ... ∈ Vω that satisfies v ∈ I and (v ,v ) ∈ E (p,a,q) ∈ ∆, (p,a,q(cid:48)) ∈ ∆ implies q = q(cid:48). We replace the
0 1 0 i i+1
for all i∈N. The set F defines the winning condition of the transition relation ∆ with a transition function δ: Q×Σ→Q.
game in the sense that a play v v ... is winning for Player 0 We define rational relations by resorting to transducers. A
0 1
if v ∈F for all i∈N—otherwise it is winning for Player 1. transducer is an NFA T =(Q,Σˆ,q ,∆,F) over the alphabet
i 0
A strategy for Player σ, σ ∈ {0,1}, is a mapping Σˆ = (Σ∪{ε})×(Γ∪{ε})—Σ and Γ are both alphabets—
f : V∗V →V, which prescribes how to continue playing. A that processes pairs (u,v) ∈ Σ∗×Γ∗ of words. The run of
σ σ
a transducer T on a pair (u,v) is a sequence q ,...,q of
0 n
states such that (qi−1,(ai,bi),qi)∈∆ for all i∈{1,...,n}, 0 1 2 4 5 ...
u = a ...a , and v = b ...b ; note that u and v do not
1 n 1 n
need to be of equal length since any a or b can be ε. A pair
i i (a)Arobotmovingonaone-dimensionaldiscretegrid.Thefigure
(u,v) is said to be accepted by T if there exists a run of T showsthesettingfork=2.
on (u,v) that starts in the initial state and ends in a final state.
As an acceptor of pairs of words, a transducer T defines a (s,0) (e,1) (s,2) (e,3)
relation, namely the relation consisting of exactly the pairs
accepted by T, which we denote by R(T). Finally, a relation
R ⊆ Σ∗ ×Γ∗ is called rational if there exists a transducer (e,0) (s,1) (e,2) (s,3) ...
T with R(T) = R. (This definition of rational relations is
(b) The safety game G2. Player 0 vertices are drawn as ellipses and and
simplified from that in [13] but sufficient for our purpose.)
Player1verticesaredrawnassquares.ShadedverticesbelongtoF.
Our learning framework relies on the two well-known facts.
Fig.1. Illustrationofthesafetygamediscussedintheintroductoryexample.
Lemma 1: Let R ⊆ Σ∗ × Γ∗ be a rational rela-
tion and X ⊆ Σ∗ a regular set. Then, (1) the relation A formalization as safety game is straightforward. Player 0
R−1 = {(y,x) | (x,y) ∈ R} is again rational, and a corresponds to the system and Player 1 corresponds to the
transducer defining this set can be constructed in linear time; environment. The arena A=(V ,V ,E) consists of vertices
0 1
and (2) the set R(X) = {y ∈ Γ∗ | ∃x ∈ X: (x,y) ∈ R}, V ={s}×N and V ={e}×N—s, respectively e, indicates
0 1
called the image of X under R, is again regular, and an NFA the player moving next—as well as the edge relation E =
accepting this set can be constructed effectively. (cid:8)(cid:0)(s,i),(e,i+1)(cid:1) | i ∈ N(cid:9)∪(cid:8)(cid:0)(e,i+1),(s,i)(cid:1) | i ∈ N(cid:9).
c) Rational Safety Games: A rational safety game is a The safety game itself is the triple G =(A,F,I) with F =
k
symbolic representation of a safety game in terms of regular {s,e}×{i ∈ N | i ≥ k} and I = {s}×{i ∈ N | i ≥ k}.
languages and rational relations. Figure 1b sketches the game G for the case k =2.
k
Definition2:ArationalarenaoverthealphabetΣisanarena WenowturnG intoarationalsafetygame.Tothisend,we
k
A = (V ,V ,E) where V ,V ⊆ Σ∗ are regular languages label each vertex uniquely with a finite word. In our example,
Σ 0 1 0 1
and E ⊆V ×V is a rational relation. we choose Σ = {s,e,l} and associate the vertex (x,i) ∈
The definition of rational safety games is now immediate. {s,e}×N with the word xli where li is the encoding of i in
Definition 3: A rational safety game over the alphabet Σ is unary. We represent the sets V and V by the following NFAs:
0 1
a safety game G =(A ,F,I) where A is a rational arena
Σ Σ Σ s e
over Σ and F,I ⊆Σ∗ are regular languages. AV0: l AV1: l
Intheremainder,weassumeregularlanguagestobegivenas
Moreover, we represent the edges by the following transducer:
NFAs and rational relations as transducers. In addition, we use
(l,l) (l,l)
these notions interchangeably when referring to rational arenas
and rational safety games; for instance, we write a rational (ε,l) (s,e) (e,s) (l,ε)
T :
area A = (V ,V ,E) as A = (A ,A ,T ) given that E
Σ 0 1 Σ V0 V1 E
L(AV0)=V0, L(AV1)=V1, and R(TE)=E. Finally, the NFA k−1 states
Let us illustrate rational safety games through an example.
Example 1: Consider a simple example motivated by motion AF: s,e l ... l l l
planning, sketched in Figure 1a, in which a robot moves on
an infinite, discrete one-dimensional grid that is “bounded on represents the set F; similarly, I is represented by a copy of
the left”. The robot can move left or right to an adjacent cell AF in which the transition labeled with e is omitted.
(providedthatithasnotreachededgeofthegrid)oritcanstay It is worth mentioning that rational arenas not only subsume
at its current position. The grid is partitioned into a safe and finitearenasbutalsoarichclassofofinfinitearenas,including
an unsafe area, the former being shown shaded in Figure 1a. such encoding computations of Turing machines. Hence, the
The safe area is parameterized by an integer k ∈N\{0} and problem of determining the winner of a rational safety game
consists of all position greater than or equal to k. The robot is undecidable, and any algorithm for computing a winning
starts somewhere inside the safe area. set can at best be a semi-algorithm (i.e., an algorithm that, on
Therobot’smovementisgovernedbytwoadversarialplayers, termination, gives the correct answer but does not guarantee
called system and environment; the system can move the robot to halt). The algorithm we design in this paper is of this kind
to the right or keep it at its current position, whereas the and guarantees to learn a winning set if one exists. To ease
environment can move the robot to the left (if the edge has not description, we always assume that a winning set set exists.
been reached) or keep it at its current position. The players
move the robot in alternation, and the system moves first. The III. THELEARNINGFRAMEWORK
system’s objective is to stay within the safe area, whereas the Our learning framework is an extension of the ICE frame-
environment wants to move the robot out of it. Note that the work proposed by Garg et. al. [6], which deals with learning
system can win, irrespective of k, by always moving right. loop invariants from positive and negative data as well as
implications. The learning takes place between a teacher, who Suppose that the learner conjectures the DFA C with
0
has (explicit or implicit) knowledge about the rational safety L(C ) = ∅. As C fails Check 1 (it passes all other checks),
0 0
game in question, and a learner, whose objective is to learn the teacher returns a positive counterexample, say u=sll∈I.
a DFA accepting a winning set, but who is agnostic to the Next, suppose the learner conjectures the DFA C with
1
game. We assume that the teacher announces the alphabet of L(C )={sln |n≥2}, which passes all checks but Check 3
1
the game before the actual learning starts. (as the players alternate but L(C ) does not contain a vertex
1
The learning proceeds in a CEGIS-style loop [2]. In every of the environment). The teacher replies with an existential im-
iteration, the learner conjectures a DFA, let us call it C, and plication counterexample, say (sll,A) with L(A)={ell,elll}.
the teacher checks whether L(C) is a winning set—this kind In the next round, let us assume that the learner conjectures
of query is often called equivalence or correctness query. the DFA C2 with L(C2) = {sln | n ≥ 2}∪{elm | m ≥ 3}.
Although the teacher does not know a winning set (the This conjecture passes all checks (i.e., L(C2) is a winning set),
overall objective is to learn one after all), he can resort to the teacher replies “yes”, and the learning ends.
Conditions (1)–(4) of Definition 1 in order to decide whether It is important to note that classical learning frameworks
L(C) is a winning set. If L(C) satisfies Conditions (1)–(4) for regular languages that involve learning from positive and
(i.e., L(C) is a winning set), then the teacher replies “yes” and negative data only, such as Gold’s passive learning [7] or
the learning ends. If this is not the case, the teacher returns Angluin’s active learning [8], are insufficient in our setting. If
a counterexample witnessing the violation of one of these the learner provides a conjecture C that violates Condition (3)
conditions, and the learning continues with the next iteration. or (4) of Definition 1, the teacher is stuck. For instance, if
The definition below fixes the protocol between the teacher C does not satisfy Condition (4), the teacher does not know
and the learner and defines counterexamples. whether to exclude u or to include E({u}). Returning an
Definition 4 (Teacher for rational safety games): Let G = implication counterexample, however, resolves this problem in
Σ
(A ,F,I) be a rational safety game over the rational arena that it communicates exactly why the conjecture is incorrect
Σ
A =(V ,V ,E).ConfrontedwithaDFAC,ateacherforG and, hence, allows the learner to make progress.1
Σ 0 1 Σ
replies as follows:
IV. AGENERICTEACHER
1) If I (cid:54)⊆ L(C), then the teacher returns a positive coun-
Wenowpresentagenericteacherthat,takingarationalsafety
terexample u∈I\L(C).
game as input, answers queries according to Definition 4. For
2) If L(C) (cid:54)⊆ F, then the teacher returns a negative
the remainder of this section, fix a rational safety game G =
Σ
counterexample u∈L(C)\F.
(A ,A ,A ) over the rational arena A = (A ,A ,T ),
3) Ifthereexistsu∈L(C)∩V suchthatE({u})∩L(C)=∅, Σ F I Σ V0 V1 E
0 and let C be a DFA conjectured by the learner.
then the teacher picks such a u and returns an existential
To answer a query, the teacher performs Checks 1 to 4
implication counterexample (u,A)∈Σ∗×NFA where
Σ of Definition 4 as described below. If the conjecture passes
L(A)=E({u}).
all checks, the teacher returns “yes”; otherwise, he returns a
4) If there exists u∈L(C)∩V1 such that E({u})(cid:54)⊆L(C), corresponding counterexample, as described next.
then the teacher picks such a u and returns a universal Check 1 (initial vertices): The teacher computes an NFA
implication counterexample (u,A)∈Σ∗×NFAΣ where B with L(B) = L(AI)\L(C). If L(B) (cid:54)= ∅, he returns a
L(A)=E({u}). positive counterexample u∈L(B).
If C passes all four checks, the teacher replies “yes”. The Check 2 (safe vertices): The teacher computes an NFA
order in which the teacher performs these checks is arbitrary. B with L(B) = L(C)\L(AF). If L(B) (cid:54)= ∅, he returns a
It is easy to see that the language of a conjecture is indeed negative counterexample u∈L(B).
a winning set if the teacher replies “yes” (since it satisfies Check3(existentialclosure): Tocheckexistentialclosure,
all conditions of Definition 1). The meaning of a positive the teacher successively computes three NFAs:
counterexampleisthatanyconjectureneedstoacceptsit,butit 1) AnNFAB withL(B )=R(T )−1(L(C));thelanguage
1 1 E
wasrejected.Similarly,anegativecounterexampleindicatesthat L(B ) contains all vertices that have a successor in L(C).
1
anyconjecturehastorejectitbutitwasaccepted.Anexistential 2) An NFA B with L(B )=L(A )\L(B ); the language
2 2 V0 1
implication counterexample (u,A) means that any conjecture L(B ) contains all vertices of Player 0 that have no
2
accepting u has to accept at least one v ∈ L(A), which successor in L(C).
was violated by the current conjecture. Finally, a universal 3) An NFA B with L(B )= L(C)∩L(B ); the language
3 3 2
implication counterexample (u,A) means that any conjecture L(B ) contains all vertices of Player 0 that belong to
3
accepting u needs to accept all v ∈L(A). At this point, it is L(C) and have no successor in L(C).
important to note that Definition 4 is sound (in particular, both Every u∈L(B ) is a witness that C is not existentially closed.
3
types of implication counterexamples are well-defined due to Hence, if L(B )(cid:54)=∅, the teacher picks an arbitrary u∈L(B )
3 3
Lemma 1 Part 2) and every counterexample is a finite object.
Letusillustratethislearningframeworkthroughanexample. 1Garget.al.[6]arguecomprehensivelywhyimplicationsneededinarobust
invariant learning framework. Their arguments also apply to our setting as
Example 2: We revisit the setting of Example 1 for the case
oneobtainsasettingsimilartoGarget.al.’sbyconsideringasolitarygame
k =2 and describe how the learner learns a winning set. withPlayer1astheonlyplayer.
and returns the existential implication counterexample (u,A) Algorithm 1: A learner for rational safety games
where L(A)=R(T )({u}).
E 1 Initialize an empty sample S =(Pos,Neg,Ex,Uni) with
Check 4 (universal closure): To check universal closure, Pos =∅, Neg =∅, Ex =∅, and Uni =∅;
the teacher, again, computes three NFAs: 2 repeat
(cid:0) (cid:1)
1) An NFA B1 with L(B1) = L(AV0)∪L(AV1) \L(C); 3 Construct a minimal DFA AS consistent with S;
the language L(B1) contains all vertices not in L(C). 4 Submit AS to an equivalence query;
2) An NFA B2 with L(B2) = R(TE)−1(L(B1)); the lan- 5 if the teacher returns a counterexample then
guage L(B2) contains all vertices that have a successor 6 Add the counterexample to S;
not belonging to L(C). 7 end
3) An NFA B3 with L(B3)=L(AV1)∩L(C)∩L(B2); the 8 until the teacher replies “yes” to an equivalence query;
language L(B3) contains all vertices of Player 1 that are 9 return AS;
in L(C) and have at least one successor not in L(C).
Every u∈L(B ) is a witness that C is not universally closed. winning set exists and the counterexamples returned by the
3
Hence, if L(B )(cid:54)=∅, the teacher picks an arbitrary u∈L(B ) teacher always form contradiction-free samples.2
3 3
and returns the universal implication counterexample (u,A) After having constructed a minimal consistent DFA, the
where L(A)=R(T )({u}). learnerconjecturesittotheteacher.Iftheteacherreplies“yes”,
E
All checks can be performed using standard methods of thelearningterminates.Iftheteacherreturnsacounterexample,
automata theory, including product constructions, projections, ontheotherhand,thelearneraddsittotheappropriatesetinS
determinizingautomata,andemptinesschecks(seeLemma1). and iterates. This procedure is sketched as Algorithm 1. Note
that, by definition of the teacher, a conjecture is guaranteed to
V. ALEARNERFORRATIONALSAFETYGAMES accept a wining set once the learning terminates.
It is left to describe how the learner actually constructs a
We design our learner with two key features: (1) the learner
minimal DFA that is consistent with the current sample. How-
always conjectures a DFA consistent with the counterexamples
ever, this task, known as passive learning, is computationally
received so far (we make this precise shortly), and (2) the
hard (i.e., the corresponding decision problem is NP-complete)
learner always conjectures a minimal consistent DFA (i.e., a
already in the absence of implications [7]. Our strategy to
DFA with the least number of states among all DFAs that are
approach this hurdle is to translate the original problem into a
consistent with the received counterexamples). The first design
sequence of satisfiability problems of formulas in propositional
goal prevents the learner from making the same mistake twice,
Boolean logic and use highly optimized constraint solvers as
while the second design goal facilitates convergence of the
a practically effective means to solve the resulting formulas
overall learning (assuming that a winning set exists).
(note that a translation into a logical formulation is a popular
To meet these goals, our learner stores counterexamples in
and effective strategy). More precisely, our learner creates
a data structure, which we call sample. Formally, a sample is and solves propositional Boolean formulas ϕS, for increasing
n
a four-tuple S = (Pos,Neg,Ex,Uni) consisting of a finite values of n∈N, n≥1, with the following two properties:
set Pos ⊂ Σ∗ of positive words, a finite set Neg ⊂ Σ∗ of
1) The formula ϕS is satisfiable if and only if there exists a
negative words, a finite set Ex ⊂ Σ∗×NFA of existential n
Σ DFA with n states that is consistent with S.
implications, and a finite set Uni ⊂Σ∗×NFA of universal
Σ 2) A model M of ϕS (i.e., a satisfying assignment of
implications. We encourage the reader to think of a sample as n
the variables in ϕS) contains sufficient information to
a finite approximation of the safety game learned thus far. n
construct a DFA, denoted by A , that has n states and
M
In every iteration, our learner constructs a minimal DFA
is consistent with S.
consistentwiththecurrentsample.ADFABiscalledconsistent
If ϕS is satisfiable, then Property 2 enables us to construct
with a sample S =(Pos,Neg,Ex,Uni) if n
a consistent DFA from a model. However, if the formula is
1) Pos ⊆L(B);
unsatisfiable, then the parameter n has been chosen too small
2) Neg ∩L(B)=∅;
and the learner increments it (e.g., by one or using a binary
3) u∈L(B)impliesL(B)∩L(A)(cid:54)=∅foreach(u,A)∈Ex;
search). This procedure is summarized as Algorithm 2. We
4) u∈L(B) implies L(A)⊆L(B) for each (u,A)∈Uni.
show its correctness shortly in Section V-B.
Constructing a DFA that is consistent with a sample is The key idea of the formula ϕS is to encode a DFA with n
n
possible only if the sample does not contain contradictory statesbymeansofBooleanvariablesandtoposeconstraintson
information. Contradictions can arise in two ways: first, Pos those variables. Our encoding relies on a simple observation:
and Neg are not disjoint; second, the (alternating) transitive for every DFA there exists an isomorphic (hence, equivalent)
closure of the implications in Ex and Uni contains a pair DFA over the state set Q = {0,...,n−1} with initial state
(u,v) with u∈Pos and v ∈Neg. This observation justifies to
introduce the notion of contradiction-free samples: a sample 2In fact, checking for contradictions equips the learner with a means to
detect that the game is won by Player 1. However, since determining the
S is called contradiction-free if a DFA that is consistent with
winnerofarationalsafetygameisundecidable,anysampleobtainedduring
S exists. Since we assume that Player 0 wins from set I, a thelearningmightbecontradiction-freedespitethefactthatPlayer1wins.
Algorithm 2: Computing a minimal consistent DFA. ι=(u,A)∈Uni aformulaϕιn thatassertsL(A)⊆L(AM)if
u∈L(A ). The formulas ϕUni is then the finite conjunction
Input: A contradiction-free sample S (cid:86) ϕMι.Fortheremaindern,letusfixauniversalimplication
Output: A minimal DFA that is consistent with S ι∈Uni n
ι∈Uni, say ι=(u,A) with A=(Q ,Σ,qA,∆ ,F ), and
1 n←0; let Ante(Uni)={u|(u,A)∈Uni}Abe the0set oAf allAwords
2 repeat occurring as antecedent of a universal implication.
3 n←n+1; As a preparatory step, we introduce auxiliary Boolean vari-
4 Construct and solve ϕSn; ablesthattracktherunsofA onwordsofPref(Ante(Uni))
5 until ϕSn is satisfiable, say with model M; in order to detect when AM accepts the antecedent of a
M
6 return AM; universal implication. More precisely, we introduce variables
x where u∈Pref(Ante(Uni)) and q ∈Q, which have the
u,q
q0 = 0; moreover, given that Q and q0 are fixed, any DFA meaning that xu,q is set to true if AM: q0 −→u q (i.e., AM
with n states is uniquely determined by its transitions and final reaches state q on reading u):
x (3)
states. Therefore, we can fix the state set of the prospective ε,q0
(cid:94) (cid:94)
DFA as Q = {0,...,n−1} and the initial state as q0 = 0; ¬xu,q∨¬xu,q(cid:48) (4)
the alphabet Σ is announced by the teacher. u∈Pref(Ante(Uni)) q(cid:54)=q(cid:48)∈Q
Our encoding of transitions and final states follows an idea (cid:94) (cid:94)
(x ∧d )→x (5)
u,p p,a,q ua,q
from [14] (independently due to [15]). We introduce Boolean
ua∈Pref(Ante(Uni)) p,q∈Q
variables d and f where p,q ∈ Q and a ∈ Σ, which
p,a,q q Formula (3) asserts that x is set to true since any run
have the following meaning: setting dp,a,q to true means that starts in the initial state qε,.q0Formula (4) enforces that for
the transition δ(p,a)=q exists in the prospective DFA, and 0
every u ∈ Pref(Ante(Uni)) there exists at most one q ∈
setting f to true means that q is a final state.
q Q such that x is set to true (in fact, the conjuction of
Tomakesurethatthevariablesd encodeadeterministic u,q
p,a,q Formulas (2)–(5) implies that there exists a unique such state).
transition function, we impose two constraints:
Finally, Formula (5) prescribes how the run of A on a word
M
(cid:94) (cid:94) (cid:94)
¬dp,a,q∨¬dp,a,q(cid:48) (1) u ∈ Pref(Ante(Uni)) proceeds: if AM reaches state p on
reading u (i.e., x is set to true) and there exists a transition
p∈Q a∈Σ q,q(cid:48)∈Q,q(cid:54)=q(cid:48) u,p
(cid:94) (cid:94) (cid:95) from p to state q on reading the symbol a∈Σ (i.e., d is
d (2) p,a,q
p,a,q set to true), then A reaches state q on reading ua and x
M ua
p∈Q a∈Σ q∈Q
needs to be set to true.
Let ϕDFA be the conjunction of Formulas (1) and (2). Given Wenowdefineϕι.Theformularanges,inadditiontod ,
n n p,a,q
a model M of ϕDFA (we assume a model to be a map from f , and x , over Boolean variables yι where q ∈ Q and
n q u,q q,q(cid:48)
the variables of a formula to the set {true,false}), deriving q(cid:48) ∈ Q , which track runs of A and A . Their precise
A M
the encoded DFA is straightforward, as shown next. meaning is the following: if there exists a word u∈Σ∗ with
Definition 5: Let M be a model of ϕDFA. We define A : q −→u q and A: qA −→u q(cid:48), then yι is set to true:
n M 0 0 q,q(cid:48)
tfhoer tDheFAunAiqMue =q ∈(Q,QΣ,qw0i,thδ,FM)(dby (1)) =δ(p,taru)e;=andq yqι0,q0A (6)
p,a,q (cid:94) (cid:94)
(2) F = {q ∈ Q | M(fq) = true}. (Recall that we fixed (ypι,p(cid:48) ∧dp,a,q)→yqι,q(cid:48) (7)
Q={0,...,n−1} and q0 =0.) p,q∈Q (p(cid:48),a,q(cid:48))∈∆A
To enforce that A is consistent with the given sample Formula (6) enforces yι to be set to true because
S = (Pos,Neg,Ex,MUni), we impose further constraints, A : q −→ε q and A: qq0A,q0A−→ε qA. Formula (7) is similar
M 0 0 0 0
corresponding to the four requirements of consistent DFAs:
to Formula (5) and describes how the runs of A and A
M
• a formula ϕPnos asserting Pos ⊆L(AM); proceed: if there exists a word v such that AM: q0 −→v p and
• a formula ϕNneg asserting Neg ∩L(AM)=∅; A: q0A −→v p(cid:48) (i.e., ypι,p(cid:48) is set to true) and there are transitions
• a formula ϕEnx asserting that u ∈ L(AM) implies (p(cid:48),a,q(cid:48)) ∈ ∆A and δ(p,a) = q in AM, then AM: q0 −v→a q
L(AM)∩L(A)(cid:54)=∅ for each (u,A)∈Ex; and and A: qA −v→a q(cid:48), which requires yι to be set to true.
• a formula ϕUnni asserting that u ∈ L(AM) implies Finally0, the next constraint ensuq,rqe(cid:48)s that whenever AM
L(AM)⊆L(A) for each (u,A)∈Uni. accepts u (i.e., the antecedent is true), then all words that
Then, ϕS := ϕDFA ∧ϕPos ∧ϕNeg ∧ϕEx ∧ϕUni. We here lead to an accepting state in A also lead to an accepting state
n n n n n n
sketch formula ϕUni and refer the reader to Appendix A for a in A (i.e., the consequent is true).
n M
detailed presentation of the remaining formulas. A description (cid:0)(cid:95) x ∧f (cid:1)→(cid:0)(cid:94) (cid:94) yι →f (cid:1) (8)
of ϕPos and ϕNeg can also be found in [14]. u,q q q,q(cid:48) q
n n
q∈Q q∈Q q(cid:48)∈FA
A. The formula ϕUnni Let ϕAnnte(Uni) be the conjunction of Formulas (3), (4), and
We break the construction of ϕUni down into smaller parts. (5) as well as ϕι the conjunction of Formulas (6), (7), and (8).
n n
Roughly speaking, we construct for each universal implication Then,ϕUni isthe(finite)conjunctionϕAnte(Uni)∧(cid:86) ϕι.
n n ι∈Uni n
B. Correctness of the Learner and, hence, the resulting learner is a fast heuristic that is sound
but in general not complete. Another limitation is that it can
We now sketch a correctness proof of the learner—we refer
only handle implication counterexamples of the form (u,A)
the reader to Appendix B for a detailed proof. First, we state
that ϕS has the desired properties. where L(A) is finite. We refer to the learner of Section V as
n
Lemma 2: Let S be a sample, n≥1, and ϕS be as defined SAT learner and the RPNI-based learner as RPNI learner.
n
above. Then, the following statements hold: (1) If M |= ϕS, Our experiments are on a slightly restricted type of games:
n
then A is a DFA with n states that is consistent with S. 1) Edge relations are automatic. Automatic relations are
M
(2) If there exists a DFA that has n states and is consistent definedbytransducersthatdonotpossesstransitionsofthe
with S, then ϕS is satisfiable. form (a,ε) and (ε,b) but rather use a dedicated padding
n
Next, let us show the correctness of Algorithm 2. symbol to balance the length of their input-words.3
Theorem1:Givenacontradictionfree-sampleS,Algorithm2 2) Each vertex of an arena has a finite (but not necessarily
returns a minimal DFA (in terms of the number of states) that bounded) number of outgoing edges.
is consistent with S. If a minimal consistent DFA has k states, Restriction 1 simplifies the implementation of the teacher.
then Algorithm 2 terminates after k iterations. Restriction 2 is due to the limitation of the RPNI learner.
Proof: Given a sample S, suppose that there exists a We use two benchmark suits: the first suite serves to demon-
DFA that has k states and is consistent with S. Then, ϕS is strate the feasibility of our techniques for various examples,
n
satisfiable for all n≥k (see Lemma 2). Moreover, if M is a predominantly taken from the area of motion planning; the
modelofϕSn,thenAM isaDFAwithnstatesthatisconsistent secondsuiteservestoassesstheperformanceofourtechniques
with S. Since Algorithm 2 increases the parameter n by one in when confronted with games of increasing “complexity”. All
every iteration (starting with n=1), the algorithm eventually games were given as finite automata, and we employed the
finds the smallest value for which ϕS is satisfiable (after k teacher described in Section IV. We conducted all experiments
n
iterations) and, hence, a consistent DFA of minimal size. on an Intel Core i7-4510U CPU (running Microsoft Windows
Finally, we can prove the correctness of our learner. 8.1)withamemorylimitof4GiBandaruntimelimitof300s.
Theorem 2: Given a teacher, Algorithm 1, equipped with
A. Examples
Algorithm 2 to construct conjectures, terminates and returns a
(minimal) DFA accepting a winning set if one exists. We consider the following examples.
Proof: Theorem 2 follows from three observations about Diagonal game: A robot moves on an infinite, discrete two-
the learner: (1) The learner never conjectures the same DFA dimensional grid world from one cell to an adjacent cell.
twice (due to Theorem 1 and the fact that counterexamples are Player 0 controls the robot’s vertical movement, whereas
added to the sample). (2) The conjectures grow monotonically Player 1 controls the horizontal movement. Both players
insize(duetominimalityofconjectures)withincreasingn,and move the robot in alternation, and Player 0’s objective is to
(3) adding counterexamples to a sample does not rule out any stay inside a margin of two cells around the diagonal.
solution (as every DFA accepting a winning set is consistent Box game: AversionofthediagonalgameinwhichPlayer0’s
with any sample produced during the learning). Now, suppose objective is to stay within a horizontal stripe of width three.
a DFA accepting a winning set exists, say with k states. Due Solitary box game: A version of the box game in which
to Observations 1 and 2, the learner eventually conjectures a Player 0 is the only player and has control over both the
DFA with k states and, moreover, cannot conjecture a larger horizontal and the vertical movement.
DFA (due to Observation 3 and the minimality of conjectures). Evasion game: Two robots move in alternation on an infinite,
Hence, the learner eventually conjectures a DFA with k states two-dimensional grid. Each robot is controlled by a player.
that accepts a winning set, and the learning terminates. Player 0’s objective is to avoid collision with Player 1’s robot.
Follow game: A version of the evasion game in which
VI. EXPERIMENTS
Player 0’s objective is to keep his robot within a distance of
In order to demonstrate the feasibility of our learning
two cells (in the Manhattan distance) from Player 1’s robot.
approach, we implemented a Java prototype using the BRICS Program-repair game: A finitely-branching version of the
automatonlibrary[16]andMicrosoft’sZ3[17]constraintsolver.
program-repair game described by Beyene et al. [10].
The source code, including the games used in the experiments,
Table I lists the overall time taken by each of the two
is available at http://preview.tinyurl.com/n7a7byj.
learners to learn a winning set (including the time taken by the
In addition to the learner of Section V, we implemented a
teacher)aswellasfurtherstatisticsofthelearningprocess.The
learner based on the popular RPNI algorithm [18], which is
secondcolumn|G|correspondstosumofstatesofallautomata
a polynomial time algorithm for learning DFAs from positive
constitutingagame(sizeofthegame),whichservesasmeasure
and negative words. For this learner, we modified the RPNI
for the complexity of a game. The remaining columns list the
algorithm such that it constructs a consistent DFA from
number of iterations, the number of states of the learned DFA,
existential and universal implications in addition to positive
and the cardinality of each set of the final sample.
and negative words (a detailed presentation can be found in
Appendix C). In contrast to Algorithm 2, our modified version
3Automaticrelationsconstituteapropersubsetofrationalrelations,butare
of RPNI cannot guarantee to find smallest consistent DFAs stillexpressiveenoughtoencodecomputationsofTuringmachines.
TABLEI
RESULTSOFTHEFIRSTBENCHMARKSUITE
SATlearner RPNIlearner
Game |G| Timeins Iter. Size |Pos| |Neg| |Ex| |Uni| Timeins Iter. Size |Pos| |Neg| |Ex| |Uni|
Diagonal 29 1.352 62 4 1 55 2 3 1.000 77 6 1 54 10 11
Box 25 0.516 32 4 1 30 0 0 0.188 15 5 1 10 1 2
SolitaryBox 22 4.289 81 6 1 77 2 0 0.156 16 6 1 13 1 0
Follow 53 165.670 294 7 2 269 10 12 timeout(>300s)
Evasion 56 140.888 255 7 2 232 11 9 2.316 142 12 1 115 14 11
Program-repair 41 1.948 62 3 2 55 4 0 0.438 31 4 1 20 9 0
103 105 applicability to practically interesting problem instances.
SSAATTlleeaarrnneerr
102 RRPPNNIIlleeaarrnneerr VII. CONCLUSION
s
G|in 101 |G| 103 We developed an automata learning method to construct
|me finite-state reactive controllers for systems whose interactions
ti 100 with their environment are modeled by infinite-state games.
101
We focused on the practically interesting family of safety
10−1 games, utilized a symbolic representation of the underlying
101 102 103 104 105
k(cid:48) game, developed specific implementations of the learner and
the teacher, and demonstrated the feasibility of the method on
Fig.2. Resultsofthescalabilitybenchmark.
a set of problems motivated by robotic motion planning.
AsTableIshows,theSATlearnercomputedthewinningsets
forallgames,whereastheRPNIlearnercomputedthewinning REFERENCES
sets for all but the Follow game. Since the RPNI learner does
[1] R. McNaughton, “Infinite games played on finite graphs,” Ann. Pure
not compute minimal consistent DFAs, we expected that it is
Appl.Logic,vol.65,no.2,pp.149–184,1993.
on average faster than the SAT learner, which turned out to be [2] S.Itzhaky,S.Gulwani,N.Immerman,andM.Sagiv,“Asimpleinductive
the case. However, the RPNI learner fails to terminate within synthesismethodologyanditsapplications,”inOOPSLA2010. ACM,
2010,pp.36–46.
the time limit on the Follow game, and the large number of
[3] R.Ehlers,V.Raman,andC.Finucane,“SlugsGR(1)synthesizer,”2014,
iterations seem to indicate that the learner in fact diverges. availableathttps://github.com/LTLMoP/slugs/.
Finally, it is important to note that the teacher replied [4] A.Bohy,V.Bruyère,E.Filiot,N.Jin,andJ.-F.Raskin,“Acacia+,atool
forltlsynthesis,”inCAV,2012,pp.652–657.
implication counterexamples in all but one experiment. This
[5] A. Bouajjani, B. Jonsson, M. Nilsson, and T. Touili, “Regular model
observation highlights that classical learning algorithms, which checking,” in CAV 2000, ser. LNCS, vol. 1855. Springer, 2000, pp.
learn from positive and negative words only, are insufficient to 403–418.
[6] P. Garg, C. Löding, P. Madhusudan, and D. Neider, “ICE: A robust
learn winning sets (since the learning would be stuck at that
frameworkforlearninginvariants,”inCAV2014,ser.LNCS,vol.8559.
point) and one has to move to a richer learning framework. Springer,2014,pp.69–87.
[7] E.M.Gold,“Complexityofautomatonidentificationfromgivendata,”
InformationandControl,vol.37,no.3,pp.302–320,1978.
B. Scalability Benchmarks
[8] D.Angluin,“Learningregularsetsfromqueriesandcounterexamples,”
To assess the scalability of our technique when confronted Inf.Comput.,vol.75,no.2,pp.87–106,1987.
[9] O.Kupferman,N.Piterman,andM.Y.Vardi,“Anautomata-theoretic
with inputs of increasing size, we modified the game of
approachtoinfinite-statesystems,”inTimeforVerification,Essaysin
Example 1 such that the safe region is now determined by two Memory of Amir Pnueli, ser. LNCS, vol. 6200. Springer, 2010, pp.
parameters, namely k and k(cid:48), and contains all positions in the 202–259.
[10] T. A. Beyene, S. Chaudhuri, C. Popeea, and A. Rybalchenko, “A
interval [k,k(cid:48)] (we assume k <k(cid:48) and fix k =1). In this new
constraint-basedapproachtosolvinggamesoninfinitegraphs,”inPOPL
setting, the number of states of the automaton AF increases 2014. ACM,2014,pp.221–234.
when k(cid:48) increases as the automaton needs to count in unary [11] D.Neider,“Reachabilitygamesonautomaticgraphs,”inCIAA2010,
ser.LNCS,vol.6482. Springer,2010,pp.222–230.
to check the position of the robot.
[12] ——, “Small strategies for safety games,” in ATVA 2011, ser. LNCS,
Figure 2 depicts the overall time taken to learn a winning vol.6996. Springer,2011,pp.306–320.
set, depending on the parameter k(cid:48). To put the runtimes into [13] A.BlumensathandE.Grädel,“Finitepresentationsofinfinitestructures:
Automataandinterpretations,”TheoryComput.Syst.,vol.37,no.6,pp.
perspective, it also shows the size of the games.
641–674,2004.
On the scalability benchmark suite, the RPNI learner was [14] D. Neider and N. Jansen, “Regular model checking using solver
about one order of magnitude faster than the SAT learner and technologies and automata learning,” in NFM 2013, ser. LNCS, vol.
7871. Springer,2013,pp.16–31.
can computed a winning set for games up to a combined size
[15] M.HeuleandS.Verwer,“ExactDFAidentificationusingSATsolvers,”
of 50000. The SAT learner, on the other hand, computed a inICGI2010,ser.LNCS,vol.6339. Springer,2010,pp.66–79.
winning set for games up to a combined size of 10000 but did [16] A. Møller, “dk.brics.automaton – finite-state automata and regular
expressionsforJava,”2010,http://www.brics.dk/automaton/.
not terminate for game with k(cid:48) = 50000. While a thorough
[17] L.M.deMouraandN.Bjørner,“Z3:anefficientSMTsolver,”inTACAS
assessment remains as part of future work, our results promise 2008,ser.LNCS,vol.4963. Springer,2008,pp.337–340.
[18] J. Oncina and P. Garcia, “Inferring regular languages in polynomial
updatetime,”inPatternRecognition&ImageAnalysis,1992,pp.49–61.
APPENDIXA list of variables fq for p,q ∈ Q and a ∈ Σ. Given a model
CONSTRUCTINGCONSISTENTDFAS M of ϕDFA, deriving the encoded DFA is straightforward, as
n
USINGCONSTRAINTSOLVERS shown next.
Definition 6: Let M |= ϕDFA(d,f). We define the DFA
The key building block of our learner is an algorithm that, n
A =(Q,Σ,q ,δ,F) by
given a sample S, produces a smallest DFA that is consistent M 0
with S. Recall that the learner translates this problem into • δ(p,a)=q for the unique q ∈Q with M(dp,a,q)=true;
a series of satisfiability problem of propositional Boolean and
formulas ϕS and uses a constraint solver to check their • F ={q ∈Q|M(fq)=true}.
n
satisfiability. (Recall that we fixed Q={0,...,n−1} and q =0.)
0
In the following, we describe in detail how the formula ϕS To produce a DFA that is consistent with a sample, we add
n
is constructed. For the sake of a self-contained presentation, further constraints:
we repeat parts of Section V; as a beneficial side-effect, this • a formula ϕPnos asserting Pos ⊆L(AM);
repetition allows us to provide further explanations of the • a formula ϕNneg asserting Neg ∩L(AM)=∅;
formulas presented in Section V. Moreover, to facilitate a • a formula ϕEnx asserting for each (u,A) ∈ Ex that u ∈
more concise and accessible description, we define ϕSn slightly L(AM) implies L(AM)∩L(A)(cid:54)=∅; and
different. In particular, we introduce a formula ϕWn , which • a formula ϕUnni asserting for each (u,A)∈Ex that u∈
tracks the run of AM on words occurring in the sample (in L(AM) implies L(AM)⊆L(A).
Pos, Neg, and as antecedent of an implication). In contrast Moreover, we add an auxiliary formula ϕW, which we discuss
to Section V (where we defined the formula ϕUni to track n
n shortly. Then,
the run of A on the set Ante(Uni)) this approach results
M
in more concise and easier to understand formulas since (a ϕS :=ϕDFA∧ϕW ∧ϕPos ∧ϕNeg ∧ϕEx ∧ϕUni
n n n n n n n
prefix of) a word can occur more than once in a sample. As a
is the desired formula.
consequence, however, the formula ϕUni has to be changed in
n The pivotal idea of these formulas is to impose constraints
comparison to Section V.
on the variables d and f , which, in turn, determine the
p,a,q q
Recappingthemainideasandencodingofstatesandtransitions DFA AM. Having this in mind, it is easier to describe the
effects of these constraints by referring to M rather then to the
The key idea of the formula ϕS is to encode a DFA with n
n variables themselves. However, we thereby implicitly assume
statesbymeansofBooleanvariablesandtoposeconstraintson
that the formula is satisfiable and that the valuation M is a
those variables in order to obtain a DFA that is consistent with
model.
the given sample. Our encoding relies on a simple observation:
if we fix the alphabet, the set of states and the initial state, THEFORMULAϕW
n
then any DFA with n states is uniquely determined (up to
To ensure that the prospective automaton A is consistent
M
isomorphism) by its transitions and final states. Hence, we can
with the given sample, we need a mechanism to determine
without loss of generality fix the state set of the prospective
whether A accepts or rejects the words occurring in the
M
DFA to be Q = {0,...,n−1} and the initial state to be
sample. The idea is to track the run of A on all prefixes of
M
q =0; the alphabet Σ is determined by the given game.
0 the set
To encode the transitions and the final states, we introduce
Boolean variables d and f where p,q ∈ Q and a ∈ Σ, W =Pos ∪Neg ∪Ante(Ex)∪Ante(Uni),
p,a,q q
which have the following meaning: assigning true to d
p,a,q
which contains all positive and negative words as well as all
means that the transition δ(p,a)=q exists in the prospective
words that occur as antecedent of an existential or universal
DFA, and assigning true to f means that q is a final state.
q
implication.TheideaistointroduceauxiliaryBooleanvariables
To make sure that the variables d indeed encode a
p,a,q
x where u ∈ Pref(W) and q ∈ Q; the intended meaning
deterministic transition function, we impose the following u,q
of these variables is that if the prospective DFA A reaches
constraints. M
state q on reading the word u, then x is set to true. The
u,q
(cid:94) (cid:94) (cid:94)
¬d ∨¬d (9) following constraints enforce this.
p,a,q p,a,q(cid:48)
p∈Q a∈Σ q,q(cid:48)∈Q,q(cid:54)=q(cid:48)
x (11)
(cid:94) (cid:94) (cid:95) ε,q0
dp,a,q (10) (cid:94) (cid:94)
¬x ∨¬x (12)
p∈Q a∈Σ q∈Q u,q u,q(cid:48)
u∈Pref(W) q(cid:54)=q(cid:48)∈Q
Formula (9) and (10) are the same as Formula (1) and (2) (cid:94) (cid:94)
(x ∧d )→x (13)
of Section V, respectively: Formula (9) enforces that d u,p p,a,q ua,q
p,a,q
encode a deterministic function, while Formula (10) asserts ua∈Pref(W) p,q∈Q
that the function is total. Since any run starts in the initial state q , Formula (11) asserts
0
Let ϕDFA(d,f) be the conjunction of Formulas (9) and (10) that x is set to true. Formula (12) enforces that for every
n ε,q0
where d denotes the list of variables d and f denotes the u ∈ Pref(W) there exists at most one q ∈ Q such that
p,a,q