Optimal Control of the SugarScape ABM

19 Pages·2014·2.95 MB·English

Optimal Control of the
SugarScape ABM

Scott Christley

Department of Surgery

University of Chicago

SwarmFest 2014

Scott Christley, SugarScape Optimal Control
Control Engineering

•  Control is about designing and building systems that achieve
desired behavior.

•  Most control problems fall into one of two categories

–  Move a system from current state to desired state

–  Keep a system operating near a desired state

•  These are formalized as control objectives.

•  Optimal control seeks to achieve control objectives by posing
and solving an optimization problem.

–  Moving to desired state in minimum time/minimum fuel/etc

–  Minimizing the discrepancy between current and desired state

•  A model of the system is crucial for design and analysis.

Modified from Ben Fitzpatrick (tempest-tech.com)
A Block Diagram of Model-Based Control

u = control input y = observed output
“Real” System
(“Plant”)
y’ =
approximated
output
“Approximate”
System (“Model”)
Control Algorithm
(“Controller”)
•  The “model” is typically a mathematical object well-suited to control.

•  The “controller” is a means to make the “plant” behave as desired.

Modified from Ben Fitzpatrick (tempest-tech.com)
Control Modeling and ABMs

•  We will be thinking about the “plant” as the agent-based model.

•  Abstract away the real system, and the “model” becomes an
approximation of the ABM.

u = control input y = observed output
“Real” System (“Plant”)
ABM
“Approximate” System
(“Model”)
Control Algorithm
(“Controller”)
Modified from Ben Fitzpatrick (tempest-tech.com)
SugarScape ABM

•  Netlogo wealth distribution model

•  Agents

–  Eat sugar in environment

–  Metabolize (lose) sugar each time step

–  Vision (and movement) to patches with higher sugar

–  Heterogenous: metabolism rate, vision

–  Die when sugar <= 0

•  Environment

–  Sugar regrows eat time step

–  Spatial heterogeneity for sugar growth

SwarmFest 2014

Scott Christley, SugarScape Optimal Control
SugarScape ABM

•  Demonstrate wealth inequality over time

•  Implement a tax policy to achieve a desired outcome?

SwarmFest 2014

Scott Christley, SugarScape Optimal Control
Optimal Control of SugarScape

1.  PDE (partial differential equation) approximation
of ABM.

2.  Define control objective function.

3.  Derive the optimal control characterization for
the PDE and objective function.

4.  Numerically solve the PDE and associated
optimal control problem.

5.  Transform and apply the optimal control to
SugarScape ABM.

6.  Evaluate performance of outcome.

SwarmFest 2014

Scott Christley, SugarScape Optimal Control
PDE Approximation

•  Challenging: moving from a discrete model (which can have
discontinuities) to a continuous model.

–  Agent movement: agents can “jump” to a far away spatial location
without visiting the intermediate space (teleportation).

–  This essentially applies to all agent attributes (metabolism, sugar)

•  Translate agent and environment behaviors into
mathematical equations.

•  Open question on whether this translation can be performed
in general.

•  SugarScape is simple model, we ad-hoc constructed a PDE.

SwarmFest 2014

Scott Christley, SugarScape Optimal Control
1 Control Problem
N (x, s, t) denotes the density of individuals in class i = 1, 2, 3, 4 at location x = (x , x ) in
i 1 2
⌦ = (0, L) (0, L) and with sugar quantity s 0.
⇥ PDE Appro�ximation

The PDE for N = N (x, s, t) is expressed as
i i
2 2
2
@N @ N @N @
i i i
a(x) + b (x) + (R (x, s, t)N ) = d (s)N (1)
ij i i i i
@t � @x2 @x @s �
j
j
j=1 j=1
X X
where R (x, s, t) = S(x) m u (x, t)s.
i i i
� �
Change in agent Sugar: eating
Advection:
NOTE it is easier to leave in the first order terms above form when calculating adjoint operator.
population (N) Diffusion: and metabolism,
directed agent
over time

random agent and also control

ALSO NOTE that it would be a more cmoovrermeecntt diffusion representation, if the diffusion term was
movement (a is
towards higher
rate parameter)

sugar (b is rate
2
parameter)

[(a(x)N ) ] .
i x x
j j
•  3D PDE, 2 space dimensions and 1 sugar dimension.

j=1
X
•  We initially made rough guess estimates for a and b parameters.

The adjoint variables are calculated using equation (1).
•  We calibrated the PDE simulation output against average ABM
@
After using the definition of R and applying the product rule to evaluate the term with ,
i
@s
simulation output to find the best parameters.

equation (??) can be written as
2 2
2
@N @ N @N @
SwiarmFest 2014

i i Scott Christley, SugarScape Optimal Control

a(x) + b (x) + (S(x) m u (x, t)s) N = u N d (s)N (2)
ij i i i i i i i
@t � @x2 @x � � @s �
j
j
j=1 j=1
X X
Boundary conditions for the PDE were chosen to reflect a wrap-around spatial domain in the
vertical direction and no flux movement in the horizontal direction. Also, when sugar reaches
zero, individuals die. At the initial time, there is an initial distribution of individuals given by
¯
N (x , x , s).
i 1 2
The boundary conditions are
N (x , 0, s, t) = N (x , L, s, t) (3)
i 1 i 1
@N (x , 0, s, t) @N (x , L, s, t)
i 1 i 1
= (4)
@x @x
2 2
@N (0, x , s, t)
i 2
= 0 (5)
@x
1
@N (L, x , s, t)
i 2
= 0 (6)
@x
1
N (x , x , 0, t) = 0 (7)
i 1 2
¯
N (x , x , s, 0) = N (x , x , s) (8)
i 1 2 i 1 2
ASSUME
a(x , 0) = a(x , L)
1 1
b (x , 0) = ab (x , L)
2 1 2 1
1
1 PDE model
PDE Approximation

Agents in the Sugarscape model are aggregated into one of four classes: (1) high vision - high
metabolism, (2) high vision - low metabolism, (3) low vision - high metabolism, (4) low vision -
lowmetabolism.
WeletN (x,s,t)denotesthedensityofindividualsinclassi = 1,2,3,4atlocationx = (x ,x )
•  Original SiugarScape environment consists of two 1“h2 ills”
in⌦ = (0,48) (0,48)andwithsugarquantity0 s s¯. ThePDEforN = N (x,s,t)isexpressed
i i
⇥  
as
of sugar.

@N 2 @2N 2 @N @
i i i
a (x) + b (x) + (R (x,s,t)N ) = 0 (1)
@t � i @x2 ij @x @s i i
j=1 j j=1 j
•  We simplified the sXugar eXnvironment into a gradient with 4
where R (x,s,t) = S(x) m u (x,s,t)s. The function S(x) describes the amount of environ-
i i i
� �
mental sugar given to any agent inhabiting location x = (x ,x ) in ⌦. The sugar at location x is
regions of increasing sugar.

1 2
never depleted; S(x) remains constant for all t. The function u (x,s,t) is the proportion of sugar
i
removedfromanagentinclassiwithsugarsatlocationxandtimet.
•  This gBoiuvndeasry cuonsd itaio nps frorethdeiPcDEtawberelceho saengteo rneflte ctma woratpi-aoronun,d fspraotiaml do mleainfitn tthoe right
vertical direction and no flux movement in the horizontal direction. Boundary conditions on the
towas-vradriasbl egwrereeachtoesern tsoureflgecat dre.a

th when sugar reaches 0 and restricted sugar growth when
sugar reaches a selected upper bound, s¯. At the initial time, there is an initial distribution of
individualsgivenbyN¯ (x ,x ,s). ThelocationoftheBCats = 0ors = s¯dependsonthesignof
i 1 2
•  Agent vision and sugar environment à advection motion
R.
in PDE.

48
Region 1 Region 2 Region 3 Region 4
x2
S(x) =1 S(x) =2 S(x) =3 S(x) =4
0
0 12 24 36 48
x1
SwarmFest 2014

Scott Christley, SugarScape Optimal Control

Figure1: Pictureofspatialdomain⌦. FunctionS(x)representsamountofsugaratlocationx ⌦.
2
1