NASA Technical Reports Server (NTRS) 20020038834: Unifying Model-Based and Reactive Programming within a Model-Based Executive

8 Pages·0.78 MB·English

by  NASA Technical Reports Server (NTRS)

Unifying Model-based and Reactive Programming
within a Model-based Executive
Brian C. Williams and Vineet Gupta t
NASA Ames Research Center, MS 269-2
Moffett Field, CA 94035 USA
E-mail: {williams, vgupta}@ptolemy, arc.nasa.gov
Abstract coding that supports efficient state estimation, moni-
toring and control generation (Reactive Burton).
Real-time, model-based, deduction has recently
Reactive MPL achieves expressivity at both the soft-
emerged as a vital component in AI's tool box for
ware and hardware levels by merging key ideas from
developing highly autonomous reactive systems. Yet
synchronous programming languages, qualitative mod-
one of the current hurdles towards developing model-
based reactive systems is the number ofmethods simul- eling and Markov decision processes. Synchronous pro-
taneously employed, and their corresponding melange gramming offers a class of languages (Halbwachs 1993)
of programming and modeling languages. This paper developed for writing control programs for reactive sys-
offers an important step towards unification. We in- tems (Harel & Pnueli 1985; Berry 1989) -- logical
troduce RMPL, a rich modeling language that com- concurrency, preemption and executable specifications.
bines probabilistic, constraint-based modeling with re- Qualitative modeling and Markov decision processes to-
active programming constructs, while offering asimple
gether offer a rich language for describing continuous
semantics in terms of hidden state Markov processes.
process and uncertainty.
We introduce probabilistic, hierarchical constraint au-
Reactive Burton achieves efficient execution through
tomata (PHCA), which allow Markov processes to be
expressed in a compact representation that preserves a careful generalization of state enumeration algorithms
the modularity of RMPL programs. Finally, a model- that are successfully employed by the Sherlock(de Kleer
based executive, called Reactive Burton is described Williams 1989) and Livingstone(Williams & Nayak
that exploits this compact encoding to perform efficent 1996) systems on simpler modeling languages.
simulation, belief state update and control sequence We start with a sketch of R. Burton, set in the con-
generation. text of other work on robotic execution and reactive
programming. The first half of the paper then in-
Introduction troduces hierarchical constraint automata, their deter-
ministic execution, and their expression using Reactive
Highly autonomous systems, such as NASA's Deep
MPL. The direct mapping from RMPL combinators to
Space One spacecraft(Muscettola et aI. 1999) and
HCA, coupled with HCA's hierarchical representation
Rover prototypes, are being deployed that leverage
avoids the state explosion problem that frequently oc-
many of the fruits of AI's work on automated reasoning
curs while compiling large reactive programs.
- planning and scheduling, task decomposition execu-
The second half of the paper turns to model-based ex-
tion, model-based reasoning and constraint satisfaction.
ecution under uncertainty. First we generalize HCAs to
Yet a likely show stopper to widely deploying this level
a factored representation of partially observable Markov
of autonomy is the myriad of AI modeling languages
decision processes (POMDPs) with limited rewards.
employed, coupled to the programming and specifica-
We then develop RBurton's stochastic monitoring and
tion languages used to implement and verify the real-
execution capabilities, while leveraging off the com-
time system.
pact encoding offered by probabilistic HCA. Finally, we
This paper concentrates on the part of this chal-
demonstrate RMPL on a simplified version of a navi-
lenge that lies at the reactive layer - robotic execu-
gation maneuver performed within the Remote Agent
tion, model-based monitoring and reactive program-
Autonomous Spacecraft Experiment. The paper con-
ruing. Key to this challenge is the development of a uni- cludes with an additional discussion of related work.
fied language that can express a rich set of mixed hard-
ware and software behaviors (Reactive Model-based The Reactive Burton Executive
Programming Language - RMPL), a compact encod-
ing of the underlying Markov process (hierarchical con- The robotic execution task consists of controlling a
straint automata - HCA), and an executive for this en- physical plant according to a stream of high-level com-
mands (goals), in the face of unexpected behavior from
tCaelum Research Corporation. the system. To accomplish this the executive controls
somevariablesoftheplant,andsensetshevaluesof such as fluid flows. Fourth, automata are arranged in a
somesensortsodetermintehehiddenstateoftheplant.
hierarchy - the state of an automaton may itself be an
AschematiocfReactivBeurtonisshownbelowT. he
automaton, which is invoked when marked by its par-
physicatlestbedbseingusedtodemonstraatespectosf ent. This enables the initiation and termination of more
RBurton'scapabilitieisncludeadeepspaceprobeen-
complex concurrent and sequential behaviors. Finally,
routetoanasteroida,Russiarnovera, deepspaceinter-
each transition may have multiple targets, allowing an
ferometera,ndachemicapllantthatgeneratersocket
automaton to be in several states simultaneously, This
fuelfromtheatmosphere. enables a compact representation for recursive behav-
iors like "always" and "do until".
RBurton
Executlve_ _x_rnal These attributes are a synthesis of representations
from several areas of computation. The first attribute
comes from the area of Markov processes, and is essen-
tial for tasks like stochastic control or failure analysis
and repair. The second and third attributes are preva-
State t I _°ntrOl I lent in areas like digital systems and qualitative mod-
eling. The fourth and fifth are prevalent in the field of
synchronous programming, and form the basis for re-
tObservables 1Controls active languages like Esterel(Berry & Gonthier 1992),
[ Plant I
Lustre(Halbwachs, Caspi, & Pilaud 1991), Signal(Guer-
nicet al. 1991) and State Charts(Harel 1987). Together
R. Burton consists of two main components. The they allow complex systems to be modeled that involve
state estimation module determines the current most software, digital hardware and continuous processes.
likely states of the plant from observed behavior us- Hierarchic constraint automata (HCA) incorporate
ing a plant model. This generalizes mode identification each of these attributes. An HCA models physical pro-
(MI), (Williams & Nayak 1996). The key difference is cesses with changing interactions by enabling and dis-
the expressiveness of the modeling languages employed. abling constraints within a constraint store (e.g., a valve
Reactive MPL allows a rich set of embedded software opening causes fuel to flow to an engine). Transitions
behaviors to be modeled, hence RBurton's state esti- between successive states are then conditioned on con-
mator offers a powerful tool for monitoring mixed soft- straints entailed by that store (e.g., the presence or ab-
ware/hardware systems. sence of acceleration).
RBurton's control sequencer executes a program for A constraint system (D, _) is a set of tokens D,
controlling the plant that is also specified using RMPL. closed under conjunction, together with an entailment
Actions are conditioned on external goals and proper- relation __C D × D. The relation _ satisfies the stan-
ties of the plant's current most likely state. Given mul- dard rules for conjunction. *
tiple feasible options, RBurton selects the course of ac- R. Burton, uses propositional state logic as its con-
tion that maximizes immediate reward. As a control straint system. In state logic each proposition is an as-
language RMPL offers the expressiveness of reactive signment xi = vii, where variable xi ranges over a finite
languages like Esterel(Berry & Gonthier 1992), along domain ©(x_). Propositions are composed into formula
with many of the goal-directed task decomposition and using the standard logical connectives - and (A), or (V)
monitoring capabilities supported by robotic execution and not (-_). If a variable can take on multiple values,
languages like RAPS(Firby 1995), TCA(Simmons 1994) then x_ = vii is replaced with vij E x_.
and ESL(Gat 1996). A deterministic, hierarchical, constraint automaton
What is particularly key is the fact that the plant S is specified as a tuple (E, O, H, Cp, Tp/, where:
model used for reasoning and the control program used
• E is a set of states, partitioned into primitive states
for execution are written in the same language.
Ep and composite states Ec. Each composite state
Hierarchic Constraint Automata denotes a hierarchical, constraint automaton.
• 0 C E is a set of start states.
RMPL programs may be viewed as specifications of
POMDPs, that is probabilistic automata with partial • H is a set of variables with each xi E II ranging over a
observability and rewards. While POMDPs offer a nat- finite domain Z)[xi]. C[H] denotes the set of all finite
ural way of thinking about reactive systems, as a direct domain constraints over H.
encoding they are notoriously intractable. To develop
• Cp : EB -_ C[II], associates with each primitive state
an expressive, yet compact encoding we introduce five
si a finite domain constraint Cp(s_) that holds when-
key attributes. First, transitions are probabilistic, with
ever s_ is marked.
associated costs. Second, the POMDP is factored into
a set of concurrently operating automata. Third, each •The standard rules for conjunction are 1) a _ a (iden-
state is labeled with a constraint that holds whenever tity); 2) aA b_ a and aAb_ b (A elimination); 3) a _ b
the automaton marks that state. This allows an ef- and bAc _ d impliesaAc _ d (cut); and 4) a _ b and
ficient, intentional encoding of co-temporal processes, a _ c implies a _ bAc (A introduction).
• 7"p:Ep×
C[1-I]--_ 2E associates with each primitive Alternatively Step provides a deterministic version of
state si a transition function Tp(si). Each Tp(si) : the control sequencer for RBurton, by placing appro-
C[H] --+ 2E specifies a set of states to be marked at priate restrictions on the control HCA. Constraints at-
tached to primitive states on this HCA are restricted
time t ÷ I, given assignments to FIat time t.
to control assignments, while transition labels are con-
ditioned on the external goals and the estimated cur-
Simulating Deterministic HCA
rent state. The set of active constraints collected from
First some preliminaries. Given automaton A, gp(A)
marked states during step 2of the algorithm is then the
denotes a function that returns the relevant constraints
set of control actions to be output to the plant.
that are associated with any primitive state contained
in A or one of its descendants. Formally, Cp(A) =
A Simple Example
Cp [.J UB6_. _p(B). Similarly, Tp(A) returns the rel- We illustrate HCA with a simple automaton, c rep-
evant transition function associated with any of these
resents a constraint, start states of an automaton are
primitive states -- Tp(A) = Tp U (-JB_Eo Tp(B). marked with arrows, ar/d all transitions are labeled. For
A full marking of an automaton is a subset of states convenience we use c to denote the label _ c, and _ to
of an automaton, together with the start states of denote the label _- c. Circles represent primitive states,
any composite states in the marking. This is com- while rectangles represent composite states.
puted recursively from an initial set of states M using
JVIF(M) = M U U{.AdF(O(S)) iS E M, s composite}.
Given a current marking M on an automaton ,4, the
function Step(A,M) computes a new marking corre-
sponding to the automaton transitioning one time step.
Step(A, M)::
J
1. M1 := {s 6 M Is primitive}
2. c :=
3. M2 := [-JseM1 Tp(s, C)
4. return MF(M2)
Step 1 throws away any composite marked states, The automaton has two start states, both of which
they are uninteresting as they lack associated con- are composite. Every transition is labeled _ d, hence
straints or transitions. Step 2computes the conjunction all transitions are disabled and the automaton is pre-
of the constraints implied by all the primitive states in empted whenever d becomes true. The first state has
one primitive state, which asserts the constraint c. If
M. Step 3 computes for each primitive state the set
of states it transitions to after one time step. In step d does not hold, then it goes back to itself -- thus it
4, applying :k4F to the union of these states marks the repeatedly asserts c until d becomes true. The second
start states of any composite state. The result is the automaton has a primitive start state. Once again, at
full marking for the next time step. anytime if d becomes true, the entire automaton will
A trajectory of an automaton A is a finite or infi- immediately terminate. Otherwise it waits until a be-
comes true, and then goes to its second state, which is
nite sequence of markings mo,ml,..., such that mo
composite. This automaton has one start state, which
is the initial marking, and for each i > 0, mi+l =
Step(A, mi). The initial marking is MF_O). it repeats at every time instant until d holds. In ad-
Elaborating on step 3, we represent the transition dition, it starts another automaton, which checks if e
function for each primitive state 7-p(s) as a set of pairs holds, and if true generates b in the next state. Thus,
the behavior of the overall automaton is as follows: it
(/i, si), where si 6 E, and li is a set of labels of the form
c or _ c, for some c 6 C[II]. This is the traditional starts asserting c at every time instant. If a becomes
true, then at every instant thereafter it checks if e is
representation of transitions, as a labeled arc in a graph.
true, and asserts b in the succeeding instant. Through-
If the automaton is in state s, then at the next instant
out it watches for d to become true, and if so halts. An
it will go to all states si whose label li is entailed by
constraints C that are associated with currently marked RMPL program that produces an equivalent automaton
is:
primitive states, as computed in the second step of the
algorithm, li is said to be entailed by C, written C _ li, do
if V _ c 6 li.C _ c, and for each _= c E Ii.C _ c. It (always c,
is straightforward to translate this representation into when a donext always if e thennext b)
our formal representation: 7-p(s, C) = {si IC _ li}. watching d
Two properties of these transitions are distinctive:
Reactive MPL: Primitive Combinators
Transitions are conditional on what can be deduced,
not just what is explicitly assigned, and transitions are We now present the syntax for the reactive model-based
enabled based on lack of information. programming language. Our preferred approach is to
Step provides a deterministic simulator for the plant, introduce a minimum set of primitives, used to con-
when applied to an HCA that specifies a plant model. struct programs -- each primitive that we add to the
languagiesdrivenbyadesiredfeatureW. ethendefine
We also allow generalized sequences for if ... then
ontopoftheseprimitivesavarietyofprogramcombi-
and unless ... then, terminated with thennext.
nators,suchasthoseusedinthesimpleexampleth, at
makethelanguaguesableT,heprimitivesaredrivenby
theneedtowritereactivecontrolsoftwarein thelan- A,B. This is the parallel composition of two au-
guagea,swellastomodepl hysicaslystemsT.owrite tomata, and is the basic construct for introducing con-
currency. The composite automaton has two start
reactivecontrolprogramswerequirecombinatorfsor
states, given by the two automata for A and B.
preemptionc,onditionablranchinganditeration.For
modelinhgardwarew,erequireconstructfsorrepresent- A,B
ingco-temporainlteractionasnduncertaineffectsF. i-
nallyweneedlogicacloncurrenctoybeableto
compose
models and programs together.
As we introduce each primitive we show how to con-
struct its corresponding automata. In these definitions
lower case letters, like c, denote constraints, while upper
always A. This program starts a new copy of A at
case letters, like A and B, denote automata. The term each instant of time -- this is the only iteration con-
"theory" refers to the set of all constraints associated struct needed. The automaton is produced by marking
with marked primitive states at some time point. A as a start state and by introducing an additional new
start state. This state has the responsibility of initiat-
c. This program asserts that constraint c is true at ing A during every time step after the first. A tran-
the initial instant of time. This construct is used to sition back to itself ensures that this state is always
represent co-temporal interactions, such as a qualita- marked. A second transition to A puts a new mark on
tive constraint between fluid flow and pressure. The the start state of A at every next step, each time invok-
automaton for it is: ing a virtual copy of A. The ability of an automaton
to have multiple states marked simultaneously is key
to this novel encoding, which avoids requiring explicit
copies of A.
always A
if c thennext A. This program starts behaving like
A in the next instant if the current theory entails c.
This is the basic conditional branch construct. Given
the automaton for A, we construct an automaton for Adding Uncertainty to RMPL
ifc thennext A by adding a new start state, and going The presentation has concentrated thus far on an ex-
from this state to A if c is entailed. pressive language and an algorithm for deterministi-
cally executing hierarchical constraint automata. This
if c thennext A
can be used to simulate the plant or to generate deter-
ministic plant control sequences. Uncertainty requires
closing the controller's loop. The plant's observables
are used to predict its internal state, and to determine
when it deviates from the intended effect. Uncertain
effects are modeled by introducing transition proba-
unless c thennext A. This program executes A in bilities, turning the plant into a partially observable
the next instant if the current theory does not entail Markov process. The efficient estimation of these pro-
c. The automaton for this is similar to the automa-
cesses for complex systems is notoriously difficult.
ton for if c thennext A. This is the basic construct
An efficient estimate of the plant's possible states (its
for building preemption constructs -- it is the only one belief state) is enabled through the compact encoding of
that introduces conditions _ c. This introduces non- the plant's model in terms of hierarchical constraint au-
monotonicity, however since these non-monotonic con- tomata. This estimate is used to guide the evaluation of
ditions hold only in the next instant, the logic is strati- the control program at each time tick. To express prob-
fied and monotonic in each state. This avoids the kinds abilistic knowledge into Reactive MPL we introduce the
of causal paradoxes possible in languages like Esterel. probabilistic combinator choose :
unless c thennext A
choose [A with p,B with q]. This combinator re-
duces to A with probability p, to B with probability q,
and so on. In order to ensure that the current theory
does not depend upon the probabilistic choices made
inthecurrentstate,wemakethefollowingrestriction
tions "]-pJ(si), while the probability on the OR branch,
-- allassertionosfconstraintisn
A and B must be in terminating on this AND node, denotes P(TpJ(si)).
the scope of a next. This restriction ensures that no
constraints are associated with the start states of A and RBurton: State Estimation
B (technically the attached constraint is "true"), and
To implement belief state update recall that a prob-
thus the probabilities are associated only with transi- abilistic HCA encodes a POMDP. A POMDP can be
tions. The corresponding automaton is encoded with a
described as a tuple (E,/t4, 60,PT, PO, T_). E, .M and
single probabilistic start transition, which allows us to
O denote finite sets of feasible states si, control actions
choose between A and B.
#i, and observations oi. The state transition function,
choose [A with p, B with q] PT[si(t),pi(t) _ si(t+l)] denotes the probability that
si(t+l) is the next state, given current state s,(t) and
control action #i (t) at time t. The observation func-
tion, P(.9[si (t) _-+oi(t)] denotes the probability that oi(t)
isobserved, given state s_(t) at time t. The rewardfune-
B tion :R(si (t)) specifies the immediate reward for taking
each control action given state si(t) at time t.
RBurton incrementally updates the plant belief state,
conditioned on each control action sent and each obser-
To incorporate probabitistic transitions into HCA we vation received, respectively:
change the definition of Tp. Recall for deterministic
.tOo) .(:)]
HCA that Tp(s_) denotes a single transition function. O'('t+l)[si] _ P[si(t+l) I vvo ,...,,.,v, , ...
For probabilistic HCA Tp(si) denotes a distribution ,...,
a(t+l.)[si ] _ p[s_/_+l) l_(o) (t+l) (0)
over transition functions TpJ(si), whose probabilities
P(TpJ(si)) sum to 1. Exploiting the Markov property, the belief state at time
t + 1is computed from the belief state and control ac-
Tp(si) is encoded as a probabilistic, AND-OR
tions at time t and observations at t + 1 using the stan-
tree. This supports a simple transformation of nested
dard equations:
choose combinators to probabilistic HCA. Each leaf
of this tree is labeled with a set of one or more target
states in E, which the automaton transitions to in the a('t+')[si] = _ a(t')[sj]PT[S,,#i _-_sj]
next time tick.
j=l
The branches ai --+ b_j of a probabilistic OR node Po[si _ ok]
ai represent a distribution over a disjoint set of alter-
a(t+")[s/] = a('t+')[s']E_=, a('t+')[sJ]Po[sJ _ o_]
natives, and are labeled with conditional probabilities
P[bii Iai]. The probability of branches emanating from To calculate PT recall that a transition T is com-
each a, sum to unity. posed of a set of primitive transitions, one for each
The branches of a deterministic AND node represent marked primitive state. Assuming conditional inde-
an inclusive set of choices. Each branch is labeled by a pendence of primitive transition probabilities, given the
set of conditions lij of the form _ ¢ or _=¢, where ¢ is current marking, the combined probability of each set
any formula in propositional state logic over variables is the product of the primitive transition probabilities
II. Every branch is taken whose conditions are satisfied of the set. This is analogous to the various indepen-
by the current state (i.e., P[bij [ai,lii] = 1). dence of failure assumptions exploited by systems like
GDE(de Kleer & Williams 1987), Sherlock(de Kleer
& Williams 1989) and Livingstone(Williams & Nayak
1996). However unlike these earlier systems, multiple
sets of transitions may go to the same target mark-
ing. This is a consequence of the fact that in an HCA
primitive states have multiple next states. Hence the
A CB C A D B D
transition probabilities for all transitions going to the
Each AND-OR tree is compiled into a two level tree same target must be summed according to the above
(shown above), with the root node being a probabilistic equation for cr(.t+l)[si].
OR, and its children being deterministic ANDs. Compi- Given PT' the belief update algorithm for a(.t+l)[si]
lation is performed using distributivity, as shown below, is a modified version of the Step algorithm presented
and commutativity. This allows adjacent AND nodes earlier. This new version of Step returns a set of mark-
to be merged, by taking conjunctions of labels, and ad- ings, each with its own probability. Step 3a builds the
jacent OR nodes to be merged, by taking products of sets of possible primitive transitions. Step 3b computes
probabilities. the combined next state marking and transition prob-
This two level tree is a direct encoding of Tp(si). ability of each set. Step 3c sums the probability of all
Each AND node represents one of the transition func- composite transitions with the same target:
Stepp(A,
M):: Control sequence generation again uses a variation of
1. M1 := {s E M ls primitive} Step. For step 3 of this algorithm a best first enumer-
2. c := ation algorithm is given the sets of enabled transitions
3a. M2a := I-[,_M1 Tp(s,C) from each primitive state that is marked in the most
3b. M2b := {(:V[F(Ui=I Si),17Ii=1pi) likely current marking. During the enumeration it must
[ ((SI,pl),..., (Smpn)} E M2a} rule out any sets of transitions that lead to an incon-
3c. M2 := {(S, _(S,p)eM2bP) [ (S,_) E M2b} sistent (conflicting) control assignment. It then returns
the set of transitions that maximize combined reward.
4. return M2
The best first enumeration algorithms developed for This is analogous in RAPS(Firby 1995) to selecting ap-
Sherlock and Livingstone, are directly used by RBurton plicable methods based on priority numbers.
to generate the composite transitions in step 3a and b
in order from most to least likely. However, since the Extending RMPL: Definable operators
correspondence between transitions and next states is Given the basic operators defined earlier, we can define
many to one, there is no guarantee that the belief states a variety of common language constructs, making the
are enumerated in decreasing order. task of programming in reactive MPL considerably eas-
Instead we assume that most of the probability den- ier. Common constructs in RMPL include recursion,
sity resides in the few leading candidate transition sets. conditional execution, next, sequencing and iteration.
Hence a best first enumeration of the few leading tran- In this section we concentrate on those constructs nec-
sition sets will quickly lead to a reasonable approxima- essary to support the DS1 navigation example.
tion. We enumerate transitions in decreasing order until
most of the probability density space is covered (e.g.,
Recursion and procedure definitions. Given a
95%), and then perform step 3c to merge the results.
declaration P :: A[P], where A may contain oc-
Computing _rft÷l*)[si] requires Po[si (t) _-_oi(t)].
currences of procedure name P, we replace it by
PO is computed using the standard approach in model-
always ifp then AlP(cid:0)P]. At each time tick this looks
based reasoning, first introduced within the GDE sys-
to see if p is asserted (corresponding to p being in-
tem. For each variable assignment in each new obser-
voked), and if so starts A.
vation, RBurton uses the model, current state and pre-
vious observation to predict or refute this assignment,
giving it probability 1 or 0 respectively. If no predic- next A. This is simply if true thennext A. We
tion is made, then a prior distribution on observables is can also define if c thennext A elsenext B as
assumed (e.g., 1/n for n possible values). if c thennext A, unless c thennext B.
RBurton: Greedy Sequencing if c then A. This construct has the effect of start-
A full decision theoretic executive that maximizes ex- ing A at the time instant in which c becomes true. It
can be defined in terms of the other combinators as fol-
pected reward using HCA is well beyond the scope of
this paper. However, RBurton makes the simplest use lows, where the expression to the left of the equality is
of immediate reward and belief state, resulting in a sim- replaced with the expression on the right:
ple form of task decomposition execution. In particu- ifc then d = c--+ d
lar, RBurton maximizes immediate reward under the
if c then if d thennext A = if c A d thennext A
assumption that the most likely estimated state is cor-
if c then always A =
rect. We further assume that rewards are additive. The
if c then A, if c thennext always A
hierarchical automaton provides a way of structuring
if c then (A, B) =
tasks, subtasks and solution methods.
if c then A, if c then B
Recall that the asserted constraints c of a control
if c then choose [A with p,B with q] =
program are restricted to plant control assignments. In choose [if c then A with p, if c then B with ql
addition, to support selection of methods for tasks, we
replace the probabilistic combinator choose with an
analogous combinator based on reward: A; B. This does sequential composition of A and B.
It keeps doing A until A is finished. Then it starts B.
It can be written in terms of the other constructs by
choosereward [A with p,B with q]. This combi-
detecting the termination of A by a proposition, and
nator reduces to A with reward p, to B with reward q,
using that to trigger B. RMPL detects the termina-
and so on. choosereward has restrictions analogous
tion of A by a case analysis of the structure of A (see
to choose that associate rewards only to expressions
(Fromherz, Gupta, & Saraswat 1997) for details).
containing next .
The AND-OR Tree formed by nested applications of
choosereward is analogous to choose . The tree is do A watching c. This is a weak preemption oper-
reduced in a similar manner, except that rewards are ator. It executes A, but if c becomes true in any time
added while probabilities are multiplied. instant, it terminates execution of ,4 in the next instant.
Theautomatofnorthisisderivedfromtheautomaton
do loop next
for
A by adding the label _ c on all transitions in A. {MICASreset; TurnMicasOn; MICAStakePicture}
watching MICASdone,
when MICASdone do {
suspend A on c reactivate on d. This is like the
StorePicture,
"Control - Z, fg" pair of Unix -- it suspends the pro- do {
cess when cbecomes true, and restarts it from the same when CorruptPicture donext SnapStore(n+l)
point when d becomes true. } watching PictureError V StoreOk
}
when c donext A. This starts A at the instant after
the first one in which c becomes true. It is a temporally
extended version of if c thennext A.
MICAS :: always (
choose {
{
when c do A. This temporally extends if c then A.
Its automaton is similar to the automaton for if MICASon then {
if TurnMicasOff thennext MICASoff
if c then A, except for the fact that there is a tran-
elsenext MICASon,
sition from the start state to itself labeled _ c.
if MICAStakePicture thennext MICASdone
},
DS1 Optical Navigation Example if MICASoff then
if TurnMicasOn thennext MICASon
To make RMPL's capability concrete we model the au-
tonavigation system of the spacecraft Deep Space 1. elsenext MICASoff,
if MICASfail then
This system is used on the spacecraft once a week to
if MicasReset thennext MICASoff
do small course corrections. It works by talcing pictures
elsenext MICASfail
of three asteroids, and by using the difference between
} with 0.99,
their actual locations from their projected locations to next MICASfail with 0.01
determine the course error. This is then used by an- }
other system to determine a new course. The following
is a greatly simplified version of tile program. MICAS
is a hardware model for the miniaturized camera, Au- Discussion and Related Work
toNav is the top-level control program, and TakePicture
and SnapStore are subroutines, the second including a The RMPL compiler is written in C, and generates hi-
repair procedure. erarchical constraint automata as its target. This sup-
ports all primitive combinators and a variety of defined
AutoNav 0 :: {
combinators. RBurton is written in Lisp, and builds
TurnMicasOn,
upon the best-first enumeration code at the heart of
if IPSon thennext SwitchIPSStandBy,
do the Livingstone system. The optical navigation sce-
when IPSstandby A MICASon donext { nario and other simple but expressive examples have
TakePicture(1); been encoded. In addition the language is sufficiently
TakePicture(2); expressive and compact to support the full DS1 space-
TakePicture(3); craft models developed for Livingstone. RBurton's be-
{
havior is equivalent to Livingstone for those examples.
TurnMicasOff, Current working includes modeling for a Mars rover and
OpticalNavigation0
JPL's Space Interferometer Mission.
}
Turning to related work, Reactive MPL synthe-
} watching PictureError V OpticalNavError,
sizes ideas underlying constraint-based modeling, syn-
when OpticalNavError donext AutoNav(),
when PictureError donext AutoNavFailed chronous programming languages and POMDPs. Syn-
} chronous programming languages (Halbwachs 1993;
Berry & Gonthier 1992; Halbwachs, Caspi, & Pilaud
TakePicture(n) :: { 1991; Guernic et al. 1991; Harel 1987; Saraswat, Ja-
do { gadeesan, & Gupta 1996) were developed for writing
TurnToTarget(n), control code for reactive systems. They are based on
when Turndone do SnapStore(0) the Perfect Synchrony Hypothesis -- a program reacts
} watching PictureError
instantaneously to its inputs. Synchronous program-
}
ming languages exhibit logical concurrency, orthogonal
preemption, multiform time and determinacy, which
SaapStore(n) ::{
if (n=3) then PictureError, Berry has convincingly argued are necessary character-
next { istics for reactive programming. Reactive MPL is a
MICAStakePicture; synchronous language, and satisfies all these character-
if MICASfail then istics.
Inaddition,ReactiveMPLis distinguishebdythe
Berry, G. 1989. Real-time programming: General
adoptionofMDPsasitsunderlyingmodel,itstreat-
purpose or special-purpose languages. In Ritter, G.,
mentofpartialobservabilitaynditsextensivueseof ed., Information Processing 89, 11 - 17. Elsevier.
contraint modeling to observe hidden state. This pro- de Kleer, J., and Williams, B. C. 1987. Diagnosing
vides a rich language for continuous process, failure, multiple faults. Artificial Intelligence 32(1):97-130.
uncertainty and repair.
de Kleer, J., and Williams, B. C. 1989. Diagnosis
In the Esterel work, Berry emphasizes executable
with behavioral modes. In Proceedings of IJCAI-89,
specifications -- "What you prove is what you execute"
1324-1330.
-- this is to eliminate the gap between the specifications
about which we prove properties, and the programs that Firby, R. J. 1995. The RAP language manual. Ani-
are supposed to implement them. We carry this one mate Agent Project Working Note AAP-6, University
of Chicago.
step further, by doing our reasoning on executable pro-
grams directly, in real time. Fromherz, M.; Gupta, V.; and Saraswat, V. 1997. cc -
As previously discussed, RMPL and RBurton over- A generic framework for domain specific languges. In
lap substantially with AI robotic execution languages POPL Workshop on Domain Specific Languages.
RAPS, ESL and TCA. For example, method selection, Gat, E. 1996. Esh A language for supporting ro-
monitoring, preemption and concurrent execution are bust plan execution in embedded autonomous agents.
core elements of these languages, shared with RMPL. In Proceedings of the 1996 AAA[ Fall Symposium on
One key difference is that RMPL's constructs fully Plan Execution.
cover synchronous programing, hence moving towards Guernic, P. L.; Borgne, M. L.; Gauthier, T.; and
a unification of the executive with the underlying
Maire, C. L. 1991. Programming real time applica-
real-time language. In addition RBurton's deduc- tions with SIGNAL. In Proceedings of the IEEE (1991)
tive monitoring capability handles a rich set of soft- 1321-1336.
ware/hardware models that go well beyond those han-
Halbwachs, N.; Caspi, P.; and Pilaud, D. 1991. The
dled by systems like Livingstone. This moves execution
synchronous programming language LUSTRE. In Pro-
languages towards a unification with model-based, de-
ceedings of the IEEE (1991) 1305-1320.
ductive monitoring.
Finally, note that hierarchical state diagrams, like Halbwachs, N. 1993. Synchronous programming of
State Charts(Harel 1987), are becoming common tools reactive systems. Series in Engineering and Computer
for system engineers to write real-time specifications. Science. Kluwer Academic.
These specifications are naturally expressed within Harel, D., and Pnueli, A. 1985. Logics and Models
RMPL, due to RMPL's simple correspondence with hi- of Concurrent Systems, volume 13. NATO Advanced
erarchical constraint automata, which are closely re- Study Institute. chapter On the development of reac-
lated to state charts. Together this offers a four way tive systems, 471-498.
unification between synchronous programming, robotic Harel, D. 1987. Statecharts: A visual approach to
execution, model-based autonomy and real-time speci- complex systems. Science of Computer Programming
fication, - a significant step towards our original goal. 8:231 - 274.
Nevertheless substantial work remains. Many execu-
Muscettola, N.; Nayak, P. P.; Pell, B.; and Williams,
tion and control capabilities key to highly autonomous
B.C. 1999. The new millennium remote agent: To
systems fall well outside the scope of RMPL and RBur-
boldly go where no ai system has gone before. Artificial
ton. For example, RMPL has no construct for express-
Intelligence 100.
ing metric time. Hence RBurton cannot execute or
monitor temporal plans without the aid of an executive Muscettola, N. 1994. HSTS: Integrating planning and
scheduling. In Fox, M., and Zweben, M., eds., Intelli-
like RAPS or Remote Agent's Exec. In addition, out-
side of monitoring, RBurton does not employ any de- gent Scheduling. Morgan Kaufmann.
duction or planning during control sequence generation. Saraswat, V. A.; Jagadeesan, R.; and Gupta, V. 1996.
Unifying the kinds of sequence generation capabilties Timed Default Concurrent Constraint Programming.
that are the hallmark of systems like HSTS(Muscettola J of Symbolic Computation 22(5-6):475-520.
1994) and Burton(Williams & Nayak 1997), requires Simmons, R. 1994. Structured control for autonomous
significant research. robots. IEEE Transactions on Robotics and Automa-
tion 10(1).
References
Williams, B. C., and Nayak, P. P. 1996. A model-
Benveniste, A., and Berry, G., eds. 1991. Another based approach to reactive self-configuring systems.
Look at Real-time Systems, volume 79:9. In Proceedings of AAAI-96, 971-978.
Berry, G., and Gonthier, G. 1992. The ESTEREL pro- Williams, B. C., and Nayak, P. P. 1997. A reactive
planner for a model-based executive. In Proceedings of
gramming language: Design, semantics and implemen-
IJCAI-97.
tation. Science o] Computer Programming 19(2):87 -
152.