Exploring Model Predictive Control to Generate Optimal Control Policies for HRI Dynamical Systems

1.6 MB·

by  Steven Jens Jorgensen

Exploring Model Predictive Control to Generate Optimal
Control Policies for HRI Dynamical Systems
Steven Jens Jorgensen Orion Campbell Travis Llado
[e-mail] [e-mail] [e-mail]
Donghyun Kim Junhyeok Ahn Luis Sentis
[e-mail] [e-mail] [e-mail]
7 DepartmentofMechanicalEngineering
1 TheUniversityofTexasatAustin
0 Austin,TX78712
2

n ABSTRACT behavioral influences. It was recently used to model the
a
walking exercise behavior of humans as a linear dynamical
J WemodelHuman-Robot-Interaction(HRI)scenariosaslin-
ear dynamical systems and use Model Predictive Control system [14]. Among many other states that interplay with
3
each other, their model included a measure of self-efficacy
1 (MPC) with mixed integer constraints to generate human-
which increases as a result of exercise, thereby increasing
aware control policies. We motivate the approach by pre-
the exercising behavior further. Subsequently, the authors
] senting two scenarios. The first involves an assistive robot
C that aims to maximize productivity while minimizing the also showed that a policy for behavior intervention can be
generated usingModel PredictiveControl [13].
H human’s workload, and the second involves a listening hu-
In the same way, we ask similar questions: Can HRI sce-
manoidrobotthatmanagesitseyecontactbehaviortomax-
. nariosbemodeledasdynamicalsystems? Ifso,canthetools
s imize“connection”andminimize social“awkwardness”with
c ofcontroltheorygenerateusefulpolicies? Previousworkon
the human during the interaction. Our simulation results
[ modeling HRI scenarios and generating appropriate behav-
show that the robot generates useful behaviors as it finds
iors include creating belief models of the robot and human
1 controlpoliciestominimizethespecifiedcostfunction. Fur-
[4,5],probabilisticanticipatoryactionselection[10],collab-
v ther,weimplementthesecondscenarioonahumanoidrobot
orative agent planning [21], motion planning for navigation
9 andtesttheeyecontactscenariowith48humanparticipants
to maximize human comfort [22], fluent-turn taking using
3 todemonstrateandevaluatethedesiredcontrollerbehavior.
timed petri-nets [6], utilizing POMDPs for modeling cogni-
8 Thehumanoidgenerated25%moreeyecontactwhenitwas
tion of an autonomous service robot [20], and many others.
3 told to maximize connection over when it was told to max-
0 imize awkwardness. However, despite showing the desired Forallscenariostherobot’scognitivemodeloftheworldand
. behavior,therewasnostatisticaldifferencebetweenthepar- thehumanwasnecessarytogenerateappropriateactionsto
1
address thetask at hand.
0 ticipant’s perceived connection with thehumanoid.
Here, we frame the cognitive modeling problem based on
7
intuitivemechanicalanalogies. Thistechniqueleveragesthe
1 Keywords
power of feedback optimal control to generate useful inter-
:
v HRI; Model Predictive Control; Cognitive Modeling; Pro- active behaviors. In particular, this paper explores how
Xi ductivity;EyeContact Model Predictive Control (MPC) with mixed-integer con-
straints can be used to solve HRI scenarios modeled with
r linear dynamics. In doing so, this methodology is an ap-
a 1. INTRODUCTION
proach towards creating cognitive feedback controllers.
ThestudyofHRIpromisestocreateintelligentrobotsca- As thename suggests, MPC contains a model of the sys-
pable of becoming socially aware personal assistants. How- tem and can simulate how its control policies can affect
ever, robots and assistive devices are not yet able to elicit the model in the future [17]. Since MPC can “see” a fi-
thefeeling of beingunderstood in a human-likeway. nite horizon into the future using its model of the world,
OurworkisinspiredfromstudyingSocialCognitiveThe- it can identify locally what the best control policies are to
ory (SCT) [3] which claims that human behavior is based minimize some objective function. In some HRI scenarios,
on the dynamic interplay of personal, environmental, and a predictive controller provides significant advantages over
traditional PID controllers.
To describe the usefulness of MPC for extracting useful
controlpoliciesinHRI,weconsidertwoscenarios. First,we
consider an assistive robot that helps a human accomplish
his work by bringing the human the necessary deliverables
fromaninventorystation(Fig.1). Theassistiverobotmust
(1a)ensureithasenoughbatterytoremainoperational,(1b)
continuebeingproductivebydeliveringworktothehuman,
ACMISBN. and (1c) ensure that the human is never overworked. Sec-
DOI:
H
R
Figure 2: HRI Scenario II: A humanoid listens to a
participant’s monologue about their weekly routine. Using
the MPC controller, the humanoid manages its eye contact
behaviorto either maximize connection or awkwardness.
Figure 1: HRI ScenarioI: Anassistiverobotmustbring
deliverablesfromtheinventorystation(I.S.)tothehuman’s
most planningalgorithms, thisrequiresspecifyinga desired
work station area (W.S.). A mindful robot will ensure that
goalstatethatmaynotbereachable. Ontheotherhand,an
thehuman is neveroverworked.
MPC formulation performs an optimization routine to find
the best control policy to minimize an objective function
ond,weconsideralisteninghumanoidrobotwhichmanages overthe given time horizon.
eye contact behavior (Fig. 2) to (2a) maximize connection Propereyecontactduration,whichcanaffectrelationship
and (2b) minimize awkwardness with thehuman. andinterpersonalaffinity,wasstudiedin[2]. Gazetypecat-
Note that the two HRI scenarios contain a number of if- egorizationsandcomputationalunderstandingofsocialgaze
then statements which activate or deactivate boolean vari- was explored in [24]. The amount of precision and deliber-
ables to specify if certain control inputs are available for atedelaysoftheeyecontactbetweenrobotandhumanwere
the robot. For instance, the robot can only move if it has studied in [11] and [1]. In addition, manipulating the eye
enough battery power, or gaze contributes to awkwardness contactfrequencybetweenarobotandhumantomakebet-
if eyecontactismaintainedbeyondathresholdvalue. Such ter connection was studied and implemented in [15]. Here
if-thenconstraintsmustbeincorporatedintheoptimization weproposeanewmodelforcontrollingeyecontactbehavior
routinemathematically viareformulatingthestatementsas using MPC.
inequality constraints. To incorporate theseconstraints, we
use Mixed Integer MPC, which is an optimization routine 3. TECHNICAL APPROACH
thatminimizessomeobjectivefunctionsubjecttobothreal-
valued and integer constraints. This optimization frame- 3.1 HRIas Linear DynamicalSystems
work is called Mixed-IntegerProgramming (MIP) [23].
Totakeadvantageofthetechniquesfoundinthecontrols
Interestingly, while our objective function only contains
community,wemodelHRIscenariosaslineardynamicalsys-
batterylevels,productivity,andhumanworkloadinthefirst
tems which havetheform
scenario,andconnectionandawkwardnessinthesecond,the
dx
robotisabletofindthecorrectcontrolpoliciestosatisfythe =x˙ =Ax+Bu, (1)
dt
end objective.
Todemonstratetheapproach,simulationswereperformed where A ∈An×n and B ∈ Rn×mdescribe the state changes
using a Python interface called CVXPY [7], a convex opti- duetothenworldstates,x∈Rn,andmcontrolinputs,u∈
mizationlibrarywithmixed-integerprogrammingcapability Rm respectively. In this paper, we discretize the dynamics
withanacademiclicenseoftheGurobi[9]solver. Ourcodeis by ∆t
available at https://github.com/hrianon/mpc hri. Further-
x(k+1)=(A∆t+I)x(k)+Bu(k)∆t, (2)
more, the second scenario was implemented on a humanoid
robotandtestedwith48humanparticipantstodemonstrate where I ∈ Rn×n is identity and x(k) and u(k) denote the
and evaluate thedesired eyecontact behavior. state and input at time step k.
3.2 PolicyGeneration viaMPC
2. RELATED WORKS
MIP was previously used for planning spacecraft trajec- Givensomedesiredrobotbehavioryref ∈Rny withnybe-
haviors, e.g. we want the robot to be 100% productiveand
tories [18] and using integer constraints to model obstacle
minimizethehumanworkloadto25%,astandardquadratic
avoidance. MIP has also been used for generating optimal
cost function is used to quantify how well the decision vec-
paths for manipulators [8]. Martin et al. [14] used a fluid-
tank analogy and a corresponding linear dynamic model to tors, u0,u1,...,up−1 , bring a state output y to yref over a
finitehorizonptimesteps. Thecostfunctionisthendefined
characterizehumanmentalstatesthatinfluencedailywalk-
as
ing behavior. With a simplified version of the model, they
p
controlled the system using Hybrid Model Predictive Con-
trolwithintegerandbooleanconstrainstoachieveadesired J = (y(k+i)−yref)TQy(y(k+i)−yref), (3)
goal [13]. Xi=1
In addition to previously mentioned works on modeling where Qy ∈ Rny×ny is a matrix describing the quadratic
HRIand generating behaviors[4,5,10, 21, 22, 6, 20], kino- weights of thedesired behavior.
dynamicplanningwithRRTcanalsobeusedtosolvesearch The mixed-integer MPC problem can now be formulated
problems with dynamic constraints [12]. However, as with as follows. In general, the MPC problem attempts to min-
Inventory Station
imize a cost function J subject to dynamic constraints and
inequality constraints: Charger
argmin J (4) Productivity Deliverables
{[u(k+i)]pi=−01,[δ(k+i)]pi=−01} Belief
Battery Level
s.t.
x(k+i)=(A∆t+I)x(k)+Bu(k)∆t,
y(k+i)=Cx(k+i), Human
Productivity Work Load
E δ(k)≤E +E x(k)+E u(k)+E z(k),
1 2 3 4 5
where δ(k) ∈ {0,1}nbool are nbool boolean variables used
in the problem, z(k) ∈ Rno are no floating variables, and
E , are matrices used to compactly specify the con-
{1,...,5}
straints of the problem. However, to be very clear about
how constraints are specified in the MPC problem, we will
Figure 3: Fluid Analogy for Scenario I.
describe each inequality constraint used for each scenario.
4. ASSISTANT ROBOTSCENARIO
to the human, but feels less productive whenever it takes
We consider a scenario in which a robot assists a human work, uwpo, away from the human and whenever the robot
withhisworkbydeliveringthenecessarymaterials,tools,or perceive high levels of human workload, −βl/τp. As with
paperwork needed by the human (deliverables) to perform batterylevels,therobot’sperceptionofitsownproductivity
usefulwork(Fig.1). Subjecttocertainconstraintsdescribed decreases by−1/τp with time.
in Sec. 4.2.1 The robot’s primary role is to bring work to The robot’s capacity to carry deliverables is modeled by
thehumananditsproductivityismeasureddirectlybyhow the state R . Whenever the robot performs a deliverable
d
much work is delivered to the human. However, every time pick up action (uipu and uwpu), this state increases. Simi-
the robot drops off work to the human, the human’s work- larly, when therobot performs a deliverable drop off action
load increases, so a mindful robot will be cautious of the (u and u ) thisstate decreases.
ido wdo
human’sworkload. Finally, the robot has a model of the human’s workload,
4.1 Linear DynamicModel Hl. Namely, deliverables dropped off to the human’s work
stationisanalogoustoincreasingthehumanworkload. The
4.1.1 WorldStateandActions human, working at his own pace, reduces his workload by
−1/τ .
l
Weuseafluid-flowanalogytodescribethelineardynam-
Thus,theworld stateevolveswiththefollowing AandB
ics of the scenario (Fig. 3) which enables easy visualization
matrices
of how the different states of the world are affected by the
robot’s actions and other world states. We model the state
of theworld as a vectorxR ∈R5, 0 0 0 0 0
0 −1/τ 0 0 0
xR =[Rx, Rb, Rd, Rp, Hl]T, (5) A= 0 0 b 0 0 0 , (7)
whereRx,Rb,Rd,Rp,andHl,aretherobot’sx-coordinate  0 βb/τp −βl/τp −1/τp 0 
position, battery levels, amount of deliverables being car-  0 0 0 0 −1/τl 
ried,self-perceivedproductivity,andperceivedhumanwork-
load respectively. In general, the system has control inputs
u∈R6 defined as  γ0m γ0c 00 00 00 00 
B= 0 0 γi −γi −γw −γw , (8)
uR =[umove, ucharge, uipu, uido, uwpu, uwdo]T. (6)  0 0 0 0 −γp γp 
 0 0 0 0 −γ γ 
 l l 
Morespecifically,umoveletstherobotmoveleftandright,
ucharge specifies how therobot’s batteries change, uipu and where the γ variables are constants that describe how the
uwpu denote the act of picking up deliverables from the in- state changes duetotheinput u.
ventory station and the human’s work station respectively, ThecurrentstatetransitionmatrixthatdescribestheHRI
and u and u denote the act of dropping off deliver- scenariodoesnotincorporatecertainconstraintsoftheprob-
ido wdo
ablestotheinventorystationandthehuman’sworkstation lem. For example, the robot may only pick up deliverables
respectively. whenever it is near the inventory station or when it is near
the human work station. The next section describes how
4.1.2 StateTransitionMatrix
suchconstraintsareautomaticallyincorporatedintheMPC
Referring to Fig. (3) , at every time step, the robot’s problem statement.
battery decreases by −1/τ and is further decreased as it
b 4.2 MPC Formulation
moves(|umove|>0). Thebatterycanbechargedbyucharge
when therobot is neartheinventorystation. To maximize battery and productivity,and minimize hu-
The robot feels productive whenever it has high battery man workload, we define the observation vector y ∈ R3 at
levels, βB/τp, and whenever it drops off deliverables, uwdo time step k+1 to bey(k+1)=Cx(k+1)
where C is canbeexpressedasamixed-integerconstraintusingthefol-
lowing inequality constraints.
0 1 0 0 0
C = 0 0 0 1 0 , (9) (Rx(k)−lis)≤(1−δis(k))Rxmax, (17)
 0 0 0 0 1  (Rx(k)−lis)≥δis(k)(−Rxmax), (18)
We set yref = [1,1,0.25]T which tells the robot to aim (Rx(k)−lws)≤δws(k)Rxmax, (19)
for 100% battery level, 100% productivity, and a human (Rx(k)−lws)≥(1−δws(k))(−Rxmax). (20)
workload of 25%. The cost function weights are set to
be Qy = diag{wb, wp, wl} with wb, wp, and wl denote Having location constraints, we can now constrain the
the weights for the battery, robot productivity, and human robot to only pick up and drop off deliverables whenever
workload respectively. it is near theinventorystation or thehumanwork station:
4.2.1 ScenarioIConstraints 0≤uipu(k)≤δis, (21)
The HRI scenario presented contains a number of con- 0≤uwpu(k)≤δws, (22)
straints and the optimization routine must be restricted to 0≤uido(k)≤δis, (23)
a set of allowable actions depending on the world states. 0≤uwdo(k)≤δws, (24)
The robot has thefollowing constraints:
Next, to specify capacity constraints 4 and 5, we simply
1. It can moveonly if it has enough battery. statethattherobotcannottakeactionsthatwillcauseitto
exceed its carrying capacity of 100% or to drop off deliver-
2. Itsbatteriesarechargedonlyifitisneartheinventory ables when it doesn’t haveany.
station.
0≤R ≤1.0 (25)
d
3. It can pick up and drop off deliverables only if it is
To specify constraint 6, we introduce two boolean vari-
near eitherstation.
ables, δmp and δmn which are true if the robot exerts a
4. It cannot pick up deliverables beyondits capacity. positive effort and negative effort respectively. This is nec-
essary because taking absolute values does not satisfy the
5. Itcannotdropoffdeliverablesifithasnodeliverables. Disciplined Convex Programming (DCP) [7] ruleset.
6. It loses more battery as it moves. δmp =1⇔umove >0,δmn =1⇔umove <0. (26)
These are again specified as inequality constraints
Such if-then constraints must be converted to inequality
constraintsinordertoframetheproblemasamixed-integer (1−δmp(k))ummionve<umove(k)<δmp(k)ummaovxe, (27)
MPC problem. To specify constraint 1, we introduce a
booleanvariable,δbat thatindicatesiftherobothasenough δmn(k)ummionve <umove(k)<(1−δmn(k))ummaovxe. (28)
battery to move. Namely, it is only true if and only if the Finally,toencodetheoverallchangeinbatteryduetorobot
robot’s batteryis abovea threshold,bthresh. movement and charging, we specify
δbat =1⇔Rb(k)≥bthresh (10) ucharge=δcharge−(δmove)γmovepenalty (29)
Thefollowinginequalitiesexpressthisif-thenconstraintfrom whereδcharge=δis,andδmove =(δmp+δmn). Additionally,
Eq. 10 as a mixed-integerconstraint. to be consistent with the constants’ effects as specified in
R (k)−b ≤δ (k)(Rmax−Rmin), (11) Fig. 3 and Eq. 8, γmovepenalty = γpenalty/γc. That is, the
b thresh bat b b
battery is charged wheneverit is neartheinventorystation
Rb(k)−bthresh ≥(1−δbat(k))(Rbmin−Rbmax), (12) (constraint 2) and the battery is decreased whenever the
robot moves(constraint 6).
where Rmax and Rmin are upper and lower bounds of the
b b
batterylevel. Next,wespecifythattherobotcanonlymove
5. EYE CONTACTSCENARIO
if it has enough battery
ThesecondHRIscenarioexaminedinthispaperexplores
umove(k)≥δbat(k)ummionve, (13) the dynamics of a one-on-one interaction between a human
umove(k)≤δbat(k)ummaovxe, (14) and a robot. The interaction characteristics we consider in
this model are the connection (or intimacy) between the
whereumin andumax specifythemaximummovementef-
move move humanandtherobotandtheawkwardness oftheexchange.
fort therobot can useat every time step.
Next, to specify constraints 2, and 3, we introduce two 5.1 Linear DynamicModel
booleans,δis andδws toindicatewhethertherobotisatthe
inventory station or the human work station. The desired 5.1.1 WorldStateandActions
location constraint is described as
Similar to Section 4, we use a fluid-flow analogy to de-
δis =1⇔Rx ≤lis (15) ssctaritbeeofthteheliinnetaerradctyinonamisicasvoefcttohrexsc∈enRa2r,io (Fig. 4). The
δws =1⇔Rx ≥lws (16)
where lis specifies the location of the inventorystation and x=[xc, xa]T, (30)
lws specifiesthelocation of thehuman’swork station. Sim- where xc is the“human-robot connection”or“intimacy”
ilar to the battery level constraint, the location constraints and xa is the“awkwardness”of theinteraction.
γ γ 0 −γ 0
B= 00 01 00 . (35)
Robot Eye Contact Connection Awkwardness (cid:20) 0 0 γ12 γ13 γ14 (cid:21)
Duration
5.2 MPC Formulation
With y ∈ R2 being our observation variable, C ∈ R2×2
flush to set
an identity matrix relating y to x and yref being the de-
sired interaction state, we use the same standard quadratic
Figure 4: Fluid Analogy for Scenario II cost function, J in Eq. 3, to quantify how well the input
bringsytoyref overafinitehorizonptimesteps,andwhere
Qy =diag{wc, wa}withwcandwadenotethecostfunction
In this simple model, there are a variety of inputs that weights for the connection and awkwardness of the interac-
influence the state of the interaction, several of which can tion.
beeitherdirectlyorindirectlycontrolled bytherobot. The Wealsoassumedthatduringthepredictionhorizonp,the
input u∈B5 is expressed as a boolean vector: human’s gaze u would not change. Despite the fact that
1
this assumption is probably false more often than not, we
u = robot looking at person
0 believethatifthecontrolhorizonisshortenough,thepolicy
u = person looking at robot
 1  will recover and can still produceusablebehavior.
u= u2 = gaze not reciprocated (u0 6=u1) . (31)
 u3 = robot staring at person (d>z)  5.2.1 ScenarioIIConstraints
 u = robot switching gaze (ut 6=ut−1) 
 4 0 0  TheconstraintsinthisscenarioensurethattheMPCfor-
The auxiliary variable d measures the duration of con- mulation properlytracksthedynamicsoftheinteraction as
secutive u input activations (in other words, it tracks how previously described. The constraints are grouped as fol-
0
long the robot has been looking at the person). The star- lows:
ing threshold z is a linear function of the connection state
1. Enforce thediscrete-time state equation.
variablesothatasconnectionincreases,sodoestheamount
of time that the robot can look at the human before the 2. Computethestaringgazethresholdaccordingtoequa-
gaze would be classified as staring. These variables change tion 33.
in discrete timeaccording tothefollowing equations:
3. Increment or reset the duration variable according to
d(t+∆t)=u0(t)(d(t)+∆t), (32) equation 32.
z(t+∆t)=mzxc(t)+bz, (33) 4. Set thevalueof thehumangaze inputu1.
where ∆t is the timestep and mz and bz are positive con- 5. Set thevalueof thenon-reciprocated gaze input u2.
stants modulating the relationship between connection and
6. Set thevalueof thestaring gaze input u .
the staring threshold such that if d > z, then the robot is 3
considered to bestaring at theperson. 7. Set thevalueof theswitching gaze input u .
4
5.1.2 StateTransitionMatrix Constraints 1 and 2 are simply equality constraints en-
Followingthesamemethodastheprevioussection,wede- forcing equations2 and 33, respectively.
scribe how thestate changes as a function of current states Constraint3setsthetrackingdurationaccordingtoequa-
and inputs. At every time step, the connection and awk- tion 32, which requiresthefollowing inequality constraints:
wardness decay by−1/τc and −1/τa respectively. u0(k)dmin ≤d(k+1)≤u0(k)dmax (36)
In each time step,gaze from therobot tothehumanand
from the human to the robot contributes to the connection (d(k)+∆t)−d(k+1)≤(1−u0(k))(dmax−dmin) (37)
statebyγ00 andbyγ01 respectively. Non-reciprocatedgaze (d(k)+∆t)−d(k+1)≥(1−u0(k))(dmin−dmax) (38)
and staring contribute to awkwardness by γ and γ re-
12 13 Constraint 4 simply holds human gaze constant over the
spectively. Additionally,robot staring cancels thecontribu-
prediction horizon via
tion of therobot’s gaze toward theconnection. Finally, the
switching gaze state detracts from or contributes to awk- u (k)=uinit. (39)
1 1
wardness (depending on the goal) by a factor of γ . This
14
Constraint 5 can be expressed as an equality constraint
last input is used to discourage a policy that would cause
the robot to rapidly switch between looking at the person u (k)=u (k)(1−uinit)+(1−u (k))uinit. (40)
2 0 1 0 1
and looking away by slightly penalizing switching between
thetwo states. The multiplication of two variables normally would violate
Thus, the world state evolves according to the same dis- the rules of Disciplined Convex Programming (DCP); how-
cretizedstateequationasthemodelintheprevioussection, ever, it is permissible in this case due to the fact that u1 is
except where theA matrix is held constant duringtheprediction horizon.
Inordertoset thevalueofthestaringgazeinputu ,two
3
inequality constraints are necessary.
A= −1/τc 0 , (34)
(cid:20) 0 −1/τa (cid:21) d(k)−z(k)≤u3(k)(dmax−zmin) (41)
and theB matrix is d(k)−z(k)≥(1−u3(k))(dmin−zmax) (42)
battery workload workload existential battery workload
management tracking relief crisis management tracking
(a) 0% human workload at t= 0 (b) 30% Robot battery & 90% human workload at t= 0
Figure 5: Assistant Robot Simulation Results: For both (a) and (b), the robot worries more about the human’s
workload more than its own battery levels and productivity. Note that upu =uipu+uwpu and udo =uido+uwdo to indicate
the total deliverable pick up and drop off actions respectively. Also, the W.S. and I.S. are located at lws = 9 and lis = 1
respectively. In(a),therobotinitially dropsoffthedeliverablesitiscarryingtogivethehumanworkandproceedstocharge
its own batteries while slowly droppingoff more work to thehuman. In (b), therobot notices that thehuman is overworked
andproceedstoremoveworkfromthehumanatthecostoftherobot’sownproductivityuntilthehuman’sworkloadbecomes
manageable. Therobot also chargesitslow batterylevels toremain operational. Then,therobot proceedstoslowly drop off
work to thehumanat a manageable rate, which also makes therobot’s perception of its own productivityto rise again.
Thefinalsetofconstraints(7)requirestheintroductionof charges its battery to remain operational. Then, the robot
twobooleanauxiliaryvariablesδ andδ whichcapturegaze returns to a behavior which ensures the human has a man-
0 1
turning on or off respectively. We then set u by summing ageable workload (25−30%).
4
δ and δ .
0 1
6.2 Eye ContactSimulation
u (k)=δ (k)+δ (k) (43)
4 0 1
To test the eye contact model, we generated a random
δ (k+1)≤u (k+1) (44)
0 0 artificial humangazepatternbasedonthegeometric distri-
u0(k+1)−u0(k)≤δ0(k+1)≤1−u0(k) (45) butionsuchthattheprobabilityoftheperson switchinghis
δ (k+1)≤u (k) (46) or her gaze in a single timestep was set to ∆t·20% with
1 0
∆t = 0.33 seconds. We set the prediction horizon for the
u (k)−u (k+1)≤δ (k+1)≤1−u (k+1) (47)
0 0 1 0 MPC to be 8 steps (2.66 seconds) and the control horizon
(thenumberofcontrolstepstobeappliedbeforerecomput-
6. SIMULATIONRESULTS
ing theoptimal control policy) tobe 3 steps(1.0 second).
6.1 Predictive AssistantSimulation Twodifferentinteractiongoalstateswereusedforvalida-
tionandtuning. Inthefirsttestcase,thestatedgoalwasto
We provide two test cases to the robot. For both cases, bringconnectionto80%,andtokeepsocialawkwardnessas
the optimization routine is set to maximize robot battery close to 0% as possible (yref =[80,0]T). For this test case,
andproductivity,andtargeta25%humanworkload(yref = both setpoints were given equal weight by setting wc = 1
[1;1;0.25]). Thefollowingweightswb=1,wp =1,andwl = and wa =1 (Qy =diag{1,1}).
10 wereused. That is, therobot cares moreabout ensuring Forthesecondtestcase, theconnectionwasignored,and
the human is never overworked over its own battery and theawkwardnesssetpointwasincreasedto80%. Thus,yref =
productivitylevels. Thesimulationparametersareavailable [0,80]T and Qy =diag{0,1}.
in thelinked code repository. Fig 6(a) shows that when the robot aimed to create a
Inthefirstcase(Fig.5a),therobotstartsbetweenthein- connection,therobotwouldmatchtheeyecontactbehavior
ventorystationandthehumanworkstationandthehuman ofthehumanbyreciprocatinggaze. Additionally,therobot
startsoutwithnoworkload. Therobotfirstdropsoffitsde- would initiate eye contact if the human had not looked for
liverablestothehumanandproceedstochargeitsbatteries. long periods of timein an attempt to build-upconnection.
Then it moves back and forth to bring just enough deliver- Fig 6(b) shows that when the robot aimed to maximize
ables toensurethatthehumanhasamanageable workload awkwardness,therobotgeneratednon-reciprocatedgazebe-
(always at 25%). havior: the robot looked away when the human made eye
In the second case (Fig. 5b), all the parameters and ini- contact and looked at the human when the human looks
tial conditions are the same except that the human starts away. Additionally, the robot attempted to maximize awk-
out with 90% workload and the robot starts out with 30% wardnessbystaringatthepersonwithoutdisconnectingeye
battery. Despitehavinglowbattery,therobotrushestothe contact.
human and removes the workload from the human. This
causes the robot’s productivity to become negative as per
7. EYE CONTACTEXPERIMENT
thedefinitionofrobot productivity. Therobot understands
that the human being overworked is the more important To evaluate our eye contact MPC model, we test it on
issue. When the human’s workload is at 40%, the robot humanparticipants. Ourhypothesisisthefollowing: “Ifwe
reciprocated gaze staring non-reciprocated intentional
gaze initiation avoidance gaze staring
(b) High Awkwardness
Figure 6: Eye Contact Behavior Simulation Results. In (a), the robot is attempting to generate connection with
the human while minimizing awkwardness. The robot does this by reciprocating gaze, and if the person looks away for an
extended period, the robot will attempt to initiate gaze in order to bring the connection back up again, as shown between
85 and 90 seconds in this simulation. Finally, the robot will continue to hold gaze until just before it becomes awkwardly
long as shown at 97 seconds, at which point the robot glances away to release the tension held in mutual gaze. In (b) the
robotismaximizingawkwardnessvianon-reciprocatedgaze: therobotlooksawaywhenthepersonlooksatit,andvice-versa.
Additionally,if given thechance, therobot will stare, as shown in thesimulation plot beyond97 seconds.
aim for connection building behavior with our model, then temptingtomaximizeawkwardness. Intermsofthepartici-
participantswillconsidertheinteractiontobemorenatural pantsperception,Fig.7bshowsthattherewerenostatistical
than if we aim for awkward behaviors.” differencesbetweenfeelingmoreconnection,comfort,enjoy-
ment,orperceivedinterestwhentherobotbehavedineither
7.1 Human-Study Setup case. Acorrelationtestforeachsurveyquestionshowedthat
the R2 correlation values were {0.002,0.003,0.012,−0.003}
Participants were told to stand at a specified location in
for questions q ∈ {Q1,Q2,Q3,Q4} (See Fig 7b). Due to
front of the humanoid and to describe their weekly routine
these results, we must accept the null hypothesis that our
for 60-90 seconds. Participants were only told that the hu-
MPC model did not produce significant effects on the par-
manoid is trying to learn how to interact better with hu-
ticipants for both robot behaviortargets.
mans. Attheendoftheirmonologue,participantsfilledout
Interestingly,despitethesimilarity of themultiplechoice
a short survey asking the level of enjoyment, comfort, and
responses, the 48 participants’ written feedback were more
connectiontheyfeltwiththerobotaswellashowmuchthey
neutral in the connection eliciting behavior (4 positive, 7
believed the robot was interested in their story. For every
neutral, 6 negative, and 7 non-responders) and more nega-
participant,therobot’sbehaviorwassettoeithermaximize
tiveintheawkwardelicitingbehavior(2positive,6neutral,
connection or maximize awkwardness.
10 negative, and 6 non-responders).
While we implemented off-the-shelf face tracking, it was
not reliable for detecting subtle eye contact changes. We
thereforeutilizedtheWizard-of-Oztechnique[19]forwizard- 8. DISCUSSIONS ANDCONCLUSIONS
recognitionofeyecontactthroughasmallwebcammounted
Inthesecond scenario, while therobot demonstrated the
on the humanoid’s shoulder. The operator would press the
desired behavior, participantsdid not feel more or less con-
space barkeyeverytimethehumanmadeeyecontact with
nectionorawkwardness. Thisresultcouldbeduetoanum-
therobot. At a 3Hzcontrol loop rate, this acted as thehu-
berofimplementationfactorssuchasthelackofeyelidcon-
maneyecontact,u ,inputtotheMPCdescribedinEq.39.
1 trol, the unmodeled effects of head pose, wizard error, the
The MPC outputs whether or not the robot should look
lag introduced by the control loop speed, and overly dis-
at the human u ∈ {0,1}. When u = 0, the robot moves
0 0 tracting look-away behaviors. Also, the model was tuned
the head in a random direction outside a cone around the
basedonintuition. Withtheseexperimentalresults,human
human’sface, and otherwise therobot looks at thestarting
feedback could be used to better tune the model. Finally,
position to make“eye contact”with the human. Unfortu-
many participants felt that the robot should be respond-
nately, during in-lab testing, the humanoid’s eyelid motors
ing totheir stories via backchanneling[16] head movement,
burnedout, which may haveaffected our results.
voice,andearmotions,whichindicatesthatoureyecontact
model is not descriptive enough to elicit deep connection
7.2 Results
with humans.
In the experiment, the robot produced the same types of Potential future work includes testing the assistive robot
behaviorsdescribedinthesimulationsectionforeachbehav- MPC model as well as further improvements on modeling
ior target, except that it was in response to human input. human-robot connection dynamics. Still, by taking inspi-
Intermsofoveralleyecontactbehavior,asFig.7ashows, ration from SCT models found in the exercise behavior in-
therobotmaintainedeyecontact∼75%oftheoverallinter- tervention community, this paper explored the possibility
actiontimewhenelicitingconnectionversus∼50%whenat- of treating HRIas controllable dynamical systems in which
(a) Total eye contact duration (b) Survey Response Transactions on Robotics, 23(5):952–961, 2007.
[11] M.Imai,T.Kanda,T.Ono,H.Ishiguro,andK.Mase.
Robot mediated round table: Analysisof theeffect of
robot’s gaze. Proceedings - IEEE International
Workshop on Robot and Human Interactive
Communication,pages 411–416, 2002.
[12] S.M. LaValle and J. J. Kuffner.Randomized
Robot Eye Contact Human Eye Contact
(Fraction of total experiment time) KinodynamicPlanning. International Journal of
Robotics Research, 20(5):378–400, 2001.
Figure7: ExperimentalResults: In(a),whiletherobot [13] A.Martin, D. E. Rivera,and E. B. Hekler.A Decision
gave25%moreeyecontacttotheparticipants,participants Framework for an AdaptiveBehavioral Intervention
maintained similar eye contact behaviors with the robot. for Physical ActivityUsing HybridModel Predictive
In (b), while no statistical significance is found, raw sur- Control. In American Control Conference(ACC),
vey scores indicate that participants’ responses were more pages 3576–3581, 2016.
favorable with an“awkward”robot. [14] A.Martin, D. E. Rivera,W. T. Riley, E. B. Hekler,
M. P. Buman,M. A. Adams,and A. C. King. A
DynamicalSystemsModel of Social Cognitive Theory.
state-of-the-arttechniquesfromthecontrolscommunitycan
InAmerican Control Conference(ACC), pages
beleveraged. Furthermore,theexperimentshowedthatthe
2407–2412, 2014.
control policies are deployableto real robotic systems.
[15] B. Mutlu,J. Forlizzi, and J. Hodgins. A Storytelling
Robot : Modeling and Evaluation of Human-likeGaze
9. ACKNOWLEDGEMENTS
Behavior. 6th IEEE-RAS International Conference on
This work was partially funded by a NASA Space Tech- Humanoid Robots, pages 518–523, 2006.
nology Research Fellowship, grant number NNX15AQ42H [16] C. Rich,B. Ponsler, A.Holroyd, and C. L. Sidner.
and an Officeof NavalResearch grant. Recognizing engagement in human-robotinteraction.
Human-Robot Interaction (HRI), 2010 5th
10. REFERENCES
ACM/IEEE International Conference on, pages
[1] H.Admoni,A. Dragan, S.Srinivasa, and 375–382, 2010.
B. Scassellati. Deliberate DelaysDuring [17] A.Richardsand J. How. Mixed-integerprogramming
Robot-to-HumanHandoversImproveCompliance for control. Proceedings of the 2005, American Control
With Gaze Communication. International Conference Conference, 2005., pages 2676–2683, 2005.
on Human-Robot Interaction, pages 49–56, 2014. [18] A.Richards, T. Schouwenaars, J. P. How, and
[2] M. Argyleand J. Dean.Eye-Contact, Distance and E. Feron.Spacecraft Trajectory Planning with
Affiliation. Sociometry, 28(3):284–304, 1965. AvoidanceConstraints UsingMixed-IntegerLinear
[3] A.Bandura. Social Foundations of Thought and Programming. Journal of Guidance, Control, and
Action: A Social Cognitive Theory. Prentice Hall, 1 Dynamics, 25(4):755–764, 2002.
edition, 1985. [19] L. D.Riek.Wizard of Oz Studiesin HRI:A
[4] C. Breazeal, M. Berlin, A. Brooks, J. Gray,and A. L. SystematicReview and NewReporting Guidelines.
Thomaz. Usingperspectivetaking tolearn from Journal of Human-Robot Interaction, 1(1):119–136,
ambiguous demonstrations. Robotics and Autonomous 2012.
Systems, 54(5):385–393, 2006. [20] S.R.Schmidt-Rohr,M. L¨osch, and R. Dillmann.
[5] C. Breazeal, J. Gray, and M. Berlin. An Embodied Humanand robot behaviormodeling for probabilistic
Cognition Approach to MindreadingSkills for Socially cognition of an autonomousservice robot. Proceedings
Intelligent Robots. The International Journal of of the 17th IEEE International Symposium on Robot
Robotics Research, 28(5):656–680, 2009. and Human Interactive Communication, RO-MAN,
[6] C. Chao and A.Thomaz. Timed Petri nets for fluent pages 635–640, 2008.
turn-takingovermultimodal interaction resources in [21] J. Shah,J. Wiken,B. Williams, and C. Breazeal.
human-robotcollaboration. The International Journal Improvedhuman-robotteam performance using
of Robotics Research, page 0278364915627291, 2016. Chaski, a Human-InspiredPlan Execution System.
[7] S.Diamond and S.Boyd. CVXPY:A Proceedings of the 6th international conference on
Python-embeddedmodeling language for convex Human-robot interaction, pages 29–36, 2011.
optimization. Journal of Machine Learning Research, [22] E. A.Sisbot, K. F.Marin-Urias, R.Alami, and
17(83):1–5, 2016. T. Sim´eon. A humanaware mobile robot motion
[8] H.Ding, G. Reißig, D.Groß, and O. Stursberg. planner.IEEE Transactions on Robotics,
Mixed-IntegerProgramming for OptimalPath 23(5):874–883, 2007.
Planning of Robotic Manipulators. Int. Conference on [23] J. Smith and Z. Taskin. A Tutorial Guideto Mixed
Automation Science and Engineering, pages 133–138, IntegerProgramming Models and Solution
2011. Techniques.Optimization in Medicine and Biology,
[9] I.Gurobi Optimization. Gurobi optimizer reference pages 1–23, 2008.
manual, 2015. [24] V.Srinivasan and R. R.Murphy.A survey of Social
[10] G. Hoffman and C. Breazeal. Cost-Based Anticipatory Gaze. ACM/IEEE International Conference on
Action Selection for Human-RobotFluency.IEEE Human-Robot Interaction, pages 253–254, 2011.