A Game-Theoretic Formulation of Multi-Agent Resource Allocation

8 Pages·0.356 MB·English

by  Bredin J. Maheswaran R.T., Imer C., Basar T., Kotz D., Rus D.

A Game-TheoreticFormulation of Multi-Agent Resource Allocation
DartmouthCollegeDepartmentofComputerScience
TechnicalReportPCS-TR99-360
Jonathan Bredin†,Rajiv T. Maheswaran‡, C¸agriImer‡,
Tamer Bas¸ar‡,DavidKotz†, and DanielaRus†
Abstract fromhosts. Wederiveabiddingstrategythat,givenperfect
information, computes an agent’s bids to minimize its ex-
This paper considers resource allocation in a network ecution time of a sequence of tasks under a fixed budget
with mobile agents competing for computational priority. constraint.
We formulatethis problem as amulti-agentgame withthe Theaddeddimensionofagents’choiceofexecution lo-
players being agents purchasing service from a common cation,however,exposeshoststoadditionalcongestionand
server. Weshowthatthere exists acomputableNash equi- volatility. Weproposethatagentsuseasystemthatcreates
librium when agents have perfect information into the fu- incentivesforhoststoparticipate,providesagentsfeedback
ture. Wesimulateanetworkof hosts andagentsusing our aboutthecostsoftheiractions,andallowsagentsofhetero-
strategytoshowthatourresource-allocationmechanismef- geneousprioritiestooperatesimultaneously.
fectivelyprioritizesagentsaccordingtotheirendowments.
We construct a resource-allocation policy where hosts
take bids from agents for prioritized access to computa-
tional resources (CPU time). The priority of access to a
1 Introduction resourceanagentreceivesisproportionaltoitsbidrelative
tothesumofallcurrentbidsatthehost. Hostscollectrev-
enuesfromeachagentatarateequaltotheagents’bids.
Mobile-agent systems allow user programs to au-
tonomously relocate from one host to another. An agent Weapplythispolicytoamobile-agentsystemwithsev-
may jump to one site to filter a database, jump to another eral different types of resources distributed throughout a
site to access a camera, and to a third to process the re- network. Each agent has a sequence of resources to con-
sults of the previous two hops before returning the results sume (an itinerary) and an endowment of electronic cur-
totheuser. Foreach ofthethree hops,theremaybealter- rency to be used to purchase resource access to complete
nativesites to access compatible resourcesand this choice itsitinerary.
of execution location subjects hosts to greater congestion We formulatethe hosts’ resource-allocationproblem as
volatility. a game with the players being agents competing for a re-
Codemobilityisasoftwarearchitecturalfeaturethathas sourcefromacommonserver. Weshowhowtocomputethe
many benefits [12]. User programs can reduce the effect unique positive Nash equilibrium explicitly under perfect
ofnetworklatencybyupdatingtheirexecution states tobe information when there are two or more players. Starting
closer to their data. Mobility is a flexible abstraction that from this simple mechanism and an assumption of perfect
can speed software development and deployment. Addi- knowledge, we develop an optimal agent bidding strategy
tionally,mobilityprovidesanextralayeroffaulttolerance. thatplansanagent’sexpenditureovermulti-taskitineraries.
To regulate mobile agents and provide resourceowners Our bidding strategy minimizes execution time while pre-
withgreater incentiveto host agents,we presentamarket- serving a prespecifiedbudget constraint. We complete our
basedsysteminwhichagentsbidforcomputationalpriority workbypresenting asimulationofmobile agentscompet-
ingforcomputationalaccessinanetworkofheterogeneous
†DepartmentofComputerScience,DartmouthCollege,6211Sudikoff hosts.
Lab,Hanover,NH03755
‡CoordinatedScienceLaboratory,UniversityofIllinois,1308W.Main Thepaper isorganizedasfollows. InSection2, wede-
Street,Urbana,IL61801 scribethesystemmodel.InSection3,wederivetheoptimal
1
Available at URL ftp://ftp.cs.dartmouth.edu/TR/TR99-360.ps.Z
bidforasingleagent’sfirstjobgivenloadstatistics.InSec-
tion4,weshowtheexistenceofaNashequilibriumthatcan qi(ui +θ−i)
ei = k k k . (3)
becomputedbytheserver,whenallagentssubmitbidfunc- k ci
k
tionsofa givenform. InSection5,weshowthattheNash
equilibrium obtained in the previous section is unique. In 3 SingleAgent Optimization
Section6, we simulate a networkwhereagents submitthe
optimalpoliciestoserversthatallocateresourcesaccording
The problem facing the i-th agent is how to choose its
to the resulting Nash equilibrium. The results are further
bids, ui K . Computationally,thiscanbeformulatedas
discussedinSection7,andsomerelatedworkisdescribed { k}k=1
anoptimizationproblemtominimizethetotaltimeofcom-
inSection8. pleting its task, T = K ti, such that the budget con-
i k=1 k
straintispreserved, E = tiui I . Atthis point,
2 SystemModel weassumethattheagei(cid:80)ntspenkd=s1atkacko≤nstaintrateui atthe
providerforthek-thtypeof(cid:80)job,andalsothatθ−iiskknown,
k
Thesystemmodelfollowsthatpresentedin[5]. Wecon- independent from ui and remains constant throughout the
k
sider a network graph whereagents are generatedat some time that server is being utilized. Inthis section, since we
subsetofnodes. Theseagentsaregivenataskofcompleting are dealing with the i-th agent only, we drop the i super-
a setofjobs ofdifferenttypesin agivensequence bypur- scripts and subscripts in all our variables except θ−i. We
k
chasing resources from service providers located through- retain the notion of θ−i to differentiatethe sum of bids at
k
outthenetwork. AnagentbeginswithanendowmentofIi thehostsubmittedbycompetingagentsfromθk,thesumof
dollarstospendtocompleteitstaskandwishestominimize allbidsincludingthei-thagent’s,tosimplifyourderivation
thetotaltimetakentocompleteasequenceofjobsgivenits ofthebiddingprocedure.
budgetconstraint. WeassumethatthereareK typesofser- Wehavethefollowingoptimizationproblem:
viceand that eachagent onlyneedsto completea job ofa
K K
particular type at most once. The agent’s task can be rep-
min ti s.t. ei I. (4)
resented as the sequence qi K , where qi is the size of k k ≤
thek-th typeofjob for th{eik-}thk=a1gent, and qki = 0implies (cid:88)k=1 (cid:88)k=1
k
that the agent’s task does not include completing a job of We solve this problem using Lagrangian methods, and
typek. Weassume that thereareseveral serviceproviders definetheLagrangian:
for each type of service, and the capacity of the provider
K K
chosenbythei-thagenttocompleteitsjoboftypekiscik. = ti +λ ei I . (5)
Wemakemanyoftheassumptionsforthesakeofnotational L k (cid:32) k− (cid:33)
k=1 k=1
simplicityandthefollowinganalysiscaneasilybeextended (cid:88) (cid:88)
tomoregeneralcasesofmulti-jobtasks. Substituting for tik and eik into equation (5) and taking
Theserviceproviderswishto have asmuchoftheir re- partialderivativeswithrespecttouk:
sources utilized as possible, and thus provide their entire
capacity at no cost if only one agent isrequesting service.
∂ q θ−i q θ−i
If more than one agent is currently requesting service, the L = − k k +λ k =0 λ= k . (6)
∂u c u2 c ⇒ u2
capacity is partitioned as follows. The i-th agent receives k k k k k
serviceatrateproportionalto itsbidrelative tothesum of
Note that θ−i > 0 implies λ > 0 for all but the trivial
allbids, k
casewhenonlyoneagentbids.Thuswehavethefollowing
relationshipbetweenanytwobids,j andk:
ui
vi =ci k , (1)
k k(cid:32)uik+θk−i(cid:33) θ−i
u =u k (7)
whereui istheamount(indollarspersecond)thatthei-th k j(cid:115)θ−i
k j
agent bids for service, the provider receives bids totaling
Incorporatingtheinequalityconstraint,weget
θ from the set of agents, , and θ−i = uj.
k Jk k j∈Jk,j(cid:54)=i k
iT-hthusa,giefntthteosceormvipcleetreatietsijsobcoonfsttyanpte,kthies:tim(cid:80)e taken by the λ∂L =λ K qk(uk+θk−i) I =0. (8)
∂λ (cid:32) ck − (cid:33)
k=1
qi(ui +θ−i) (cid:88)
ti = k k k (2) Since λ > 0, it follows that the inequality constraint
k cikuik mustbesatisfiedwithequality.Substitutingfor{uk}Kk=2in
andtheexpensesare: termsofu usingtherelationshipinequation(7),wehave,
1
2
ui =max 0,f (θ−i) N . (15)
q q θ−i q { 1 { i 1 }}i=1
1(u +θ−i)+ k k u + kθ−i I =0 (9)
c1 1 1 (cid:88)k(cid:54)=1ck(cid:115)θ1−i 1 (cid:88)k(cid:54)=1ck k − ageAntNcaasnhgeaqinuilainbriaudmvansotalugteiobnyisunailsaetetroafllybidchsawnghienrge nitos
Solvingthepreviousequationforu ,weget bid. One possibility of reaching the Nash equilibrium is
1
adecentralizedalgorithmwhereeachagentmakesaninitial
I qkθ−i q1θ−i bidandthenupdatesitsbidatpresettimeintervals,t,using
u = − k(cid:54)=1 ck k − c1 1 (10)
1 theiteration
q1(cid:80)+ qk θk−i
c1 k(cid:54)=1 ck(cid:114)θ1−i u (t+1)=f (θ−i(t)), (16)
whichyieldsthebidforthe(cid:80)firstjobforthei-thagentgiven i i 1
theloadsoftheserversforthejobsinitssequence. whereu (t)andθ−i(t)denotethei-thagent’sbidandsum
i 1
of competing bids, respectively, at time t. Unfortunately,
4 ExistenceofNashEquilibriumforMultiple this algorithm rarely converges to a Nash equilibrium and
AgentsattheSame Provider issuboptimalduetotheinconsistencyoftheinitialguesses
andsubsequentiterations.
Instead,we focusonacentralizedmethodtoobtainop-
Considernow,withoutlossofgenerality,aproviderwith
timal bids. Theagentssubmit bid functionsin theform of
capacity of resource type 1 and N agents desiring service
equation (11) and theserver produces theoptimal bids for
are currently at the provider. We assume that the agents
have some estimate of future loads, θ−i, so that they are eachagent.Toformulatethisnewmethod,wetranslateeach
k agent’sbid functiondomainfrom θ−i to a single common
abletoestimatehowmuchoftheirmoneytospendforthis 1
domain,θ . Oncewechangethedomain,weshowthatthe
currentjob.Anestimatecanbeobtainedbyaugmentingthe 1
spaceoverθ iscontinuousundermostconditionsandthat
system with “advertising” agents that periodically update 1
wecanapproximatetheremaininginstanceswithourexist-
andmakeavailablethevaluesoftotalbidsatvariousservers
throughout the network. A simple estimate of θ−i would ingframework.
k Usingu-spacehasa deficiency inthat itsdimensionin-
be themeansof theloadsofalltheservers oftypek jobs.
creases with the number of agents. To reduce our search
Givena loadestimateforfuturejobs,theoptimalbidfrom
space,weiterateoveracommondomainforallagents,θ -
equation(10)is: 1
space, where θ := θ−i +ui. Modifying the policies in
1 1 1
αi βiθ−i equation (15), we get the following implicit relations be-
ui1 =fi(θ1−i):= βi−+ γi1 (11) tweenui1andθ1:
√θ−i
1
where ui =max 0,f (θ ui) N . (17)
{ 1 { i 1− 1 }}i=1
Fromthis,wecanobtainanexplicitfunctiong (θ ):θ
i 1 1
αi := I − qcikiθk−i (12) uFii1g.uFreig2urede1miollnussttrraatteesshthoewsfhia(pθe−ofui1g)(sθhi)ft.s aOsuθt1sidvaerite→hse.
k(cid:54)=1 k i 1
(cid:88) rangeθ (0,αi/βi),g (θ )takesthevalueof0. Wenow
qi 1 ∈ i 1
βi := 1 (13) deriveg(θ1).
ci1 Substitutingθ1 −ui1 for θ1−i in (11),in therange,θ1 ∈
qi (0,αi/βi),wehavethefollowing:
γi := k θ−i (14)
ci k
(cid:88)k(cid:54)=1 k(cid:113) ui = αi−βi(θ1−ui1) (18)
Intuitively,αirepresentstheestimateofthemoneyavail- 1 βi+ γi
ableforthecurrentjob,andifthatislessthanzero,theagent √θ1−ui1
cannotaffordtopurchaseserviceunderthecurrentstateof which leads to a quadratic equation in ui. Dropping the i
1
thenetwork. Werequirethatthebidsbenon-negative. We superscript,wehave:
haveβi >0,θ−i > 0,andγi 0withequalityonlyifthe
k ≥
agenthasonejob.Ifαi 0,f willreturnanegativevalue.
i
≤
Thustheagentwillonlysubmitabidifαi >0. γ2u2+(α βθ )2u (α βθ )2θ =0 (19)
1 − 1 1− − 1 1
Attheserver,wewouldliketogenerateasetofbidsthat
formsaNashequilibrium[2]withrespecttothepoliciesof Taking the positiveroot of the equation with respect to
theN agents: u ,wehaveu =g(θ )where
1 1 1
3
4 140
3 120
2 q =0 q =2 q =4 1100
) 1 q vs. 1 80
iqf(u,−01 qqh()+11 60
40
−2
20
−3
0
0 20 40 60 80 100 120 140
−−44 −3 −2 −1 0ui 1 2 3 4 q 1 : (0, maxi{a1i/a2i})
Figure3.Sampleplotof N g (θ )versusθ
Figure 1. Behavior of max(0,f (θ ui)) for i=1 i 1 1
i − 1 for16agents
αi/βi =4 (cid:80)
2
Returning to the question of the optimal bids for
q 1 g(q 1) N agents, we seek θ1 and {ui1}Ni=1 to satisfy for all
1.5 agents the definition θ1 = Ni=1ui1 and equation (17).
An equivalent problem is to find a value of θ such
1
that N ui θ = N(cid:80)g (θ ) θ =:h (θ )=0.
1 i=1 1− 1 i=1 i 1 − 1 1 1
We know that ∂h /∂θ = 1 + N 1 > 0
(cid:80) 1 1(cid:80)|θ1=0+ − i=1
if N 2 and thus, h is increasing to the right
1
≥ (cid:80)
0.5 of zero and h (0+) > 0. We also know that
1
h (max αi/βi ) = max αi/βi < 0 for the
1 i i
{ } − { }
non-trivial case where at least two agents have αi > 0.
0
0 1 2 q 3 4 5 Because h is the sum of continuous functions, h is
1 1
1
continuous as well and must be zero for some value of
θ (0,max αi/βi ). We solve for this value byusing
Figure2.Formofgi(θ1) 1 ∈ i{ }
a bisection search of h in the given range. We sketch a
1
sampleof N g (θ )versusθ inFigure3.
i=1 i 1 1
If an agent has only one job left to complete, it can be
(cid:80)
shown that u = θ for θ (0,Ic/q) is its optimal pol-
1 1
g(θ )= (α−βθ1)2 1+ 1+ 4γ2θ1 (20) icy, which is equivalent to h∈aving γ = 0, which violates
1 2γ2 (cid:32)− (cid:115) (α βθ1)2(cid:33) one of the assumptions made earlier. However, by using
−
L’Hoˆpital’srule,weseethat
whenθ (0,αi/βi)andu =0otherwise.
1 1
∈
We see that g is continuous at θ = 0 and θ = α/β.
1 1
Tthhautso,ngθ(θ1) (is0,aαcio/nβtii)n:uous function of θ1. We also note limγ→0+g =limγ→0+ 21(α−8θβ1θγ1)2(α−βθ1)2
1 ∈ 4γ 1+ 4γ2θ1
(α−βθ1)2
=lim (cid:113)θ1
γ→0+
∂∂θg1 = +2β−(α22−γβ2(βαθ−1)βθ1)2+2γ2(α−βθ1)−4βγ2θ1 (21) =θ1 (cid:113)1+(α4−γβ2θθ11)2
2γ2√(α−βθ1)2+4γ2θ1
andwhenθ =0+,wehave Thus, if we require agents with only one job to submit
1
bid functionswithγ > 0, we allowthe agents to approxi-
matetheir optimalsolutions to arbitrary precisionand still
∂g 1 2βα2+2γ2α preservetheassumedstructure,whichyieldsanequilibrium
= 2βα+ − =1. (22)
∂θ1(cid:12)θ1=0+ 2γ2 (cid:20) √α2 (cid:21) solution.
(cid:12)
(cid:12)
(cid:12)
4
5 UniquenessofNashEquilibriumforMulti- onO . Whenn=2,(∂h2/∂θ ) =1andh2(0)=0,
1 1 1 |θ1=0+ 1
pleAgentsatthe SameProvider thus h2(0+) > 0. Also, h2(α2/β2) = h1(α2/β2) < 0
1 1 1
and (∂2h2/∂θ2) < 0 on O . Applying Lemma 1, we get
1 1 2
Let Oi = (0,αi/βi)be indexed such that O1 O2 thatthereisa uniquepoint θ0 s.t. h21(θ0) = 0onO2. But,
··· ⊃ ON (i.e,α1/β1 > a21/β2 > ··· > αN/βN⊃) whe⊃re hw21h(eθr1e)h=2(hθ11)(θ=1)0<on0Oon.O1∩O2c;thusθ0isauniquepoint
N isthenumberofagentsataserver. Letusdefine 1 0 1
Uniquenesscanbe shownusing aninductiveargument.
n Assume that there is a unique point θ < αi/βi on O
hn1(θ1)= gi(θ1)−θ1. (23) where hi1(θ0) = 0. Also assume (∂2h0i1/∂θ12) < 0 on O1i
(cid:88)i=1 andhi1(θ1) < 0onO1∩Oic. Alongwiththecontinuityof
We have already shownthat h1(θ1) = hN1 (θ1) = 0has at hi1,thepreviousresultimpliesthefollowing:
leastonesolutiononO = N O = (0,max αi/βi ) =
∪i=1 i i{ }
O .
1 >0 θ <θ
1 0
Theorem1 hN1 (θ1)=0hasonlyonesolutiononO. hi1(θ1) =0 θ1 =θ0 (28)
<0 θ >θ
 1 0
ToproveTheorem1,wemustfirstprovesomeinitiallem-
mas. InAppendix9,weshowthat(∂2gi/∂θ12) < 0onOi. Rewriting equation (23), we have hi1+1 = hi1 + gi+1.
Fromthedefinitionofhninequation(23)andthedefinition Therearetwocasestoconsider:
1
oftheindices,itcanbeseenthat Case1.If(αi+1/βi+1) θ ,thenequation(28)issatisfied
0
forhi+1becauseg (θ≤)=0forθ θ (αi+1/βi+1)
1 i+1 1 1 ≥ 0 ≥
andg (θ ) 0for θ (αi+1/βi+1). Thus,there isa
∂∂2θg12i <0onOi∀i⇒ ∂∂2θh12n1 <0onOn, (24) uCnaisqeuie+2p1.oin1tIfθ≥0θwhe<reh1i1(+α≤1i(+θ10/)β=i+01)on<O1.(αi/βi), then
0
∂gi =1 i ∂hn1 =n 1, (25) hi1+1(αi+1/βi+1)=hi1(αi+1/βi+1)<0byequation(28).
∂θ1(cid:12)θ1=0+ ∀ ⇒ ∂θ1(cid:12)θ1=0+ − We also know hi1+1(0+) > 0, because hi1+1(0) = 0 and
g(cid:12)(cid:12)(cid:12)i(0)=0∀i⇒ hn1(0(cid:12)(cid:12)(cid:12))=0. (26) O(∂hi1+,1w/∂eθc1a)n|θa1=pp0l+y=Leim>ma0.1Sainncdea(r∂ri2vhei1a+t1t/h∂eθr12e)su<lt0thoant
i+1
Also,hn1 isacontinuousfunctionofθ1. there is a unique point θˆ0 on Oi+1 where hi1+1(θˆ0) = 0.
But since g (θ ) = 0 for θ > (αi+1/βi+1), we have
i+1 1 1
Lemma1 If h(x) is a twice continuously differentiable thatonO O ,hi+1(θ )=hi(θ )<0. Thus,wehave
functionon[r,s],(∂2h/∂x2) < 0on(r,s),h(r) > 0,and 1∩ i+1 1 1 1 1
auniquepointθˆ wherehi+1(θˆ)=0onO .
h(s) < 0, then there exists aunique point x0 (r,s) s.t. 0 1 0 1 ♦
∈
h(x )=0.
0
6 SimulationandResults
Proof. Weprovethislemmabycontradiction.TheInterme-
diate Value Theorem states that there is at least one value
Inthis section, we definean algorithmthat implements
x (r,s) s.t. h(x ) = 0. Because (∂2h/∂x2) < 0 on
0 ∈ 0 theresource-allocationpolicyfromSection4and describe
(r,s),weknowthathisstrictlyconcaveon(r,s),i.e.,
asimulationofthepolicy.
Ahostacceptsbidfunctionsfromallagentspresentany
time an agent arrives to or departs from the site. Agents
ah(x)+(1 a)h(y)<h(ax+(1 a)y) (27)
− − express bids through three coefficients defined in equa-
for a (0,1) and x,y (r,s). Suppose there are two tions (12-14). The host takes all bids to form the bid-
∈ ∈
points that satisfy h(x) = 0, say x ,x (r,s) where responsefunction,g(θ), andusesabisectionsearchtofind
1 2
∈
x <x . Again,usingtheIntermediateValueTheorem,we thebiddinglevelθ =g(θ). Algorithm1sketchesthisoper-
1 2
canshowthat r (r,x ) (r,s)s.t. h(r ) > 0. Then, ation.
0 1 0
∃ ∈ ⊂
we have ah(r ) + (1 a)h(x ) < h(ar + (1 a)x ) We base our simulation on the Swarm simulation sys-
0 2 0 2
− −
which implies ah(r ) < h(ar + (1 a)x ). If tem [13]. Inthe simulation, thereare 100hosts, each pro-
0 0 2
−
a = (x x /x r ) (0,1), then we have videsoneofeightcomputationalservices,andthecapacity
2 1 2 0
− − ∈
ah(r )< h(x ) =0,whichisacontradictionsincea >0. is determined from a truncated Gaussian random variable.
0 1
Thus,therecanbeatmostonepointwhereh(x)=0. Thehostscompriseastochasticallygeneratednetworkgen-
♦
erated from the GT-ITM package [6]. In GT-ITM, a net-
ProofofTheorem1. Whenn=1,(∂h1/∂θ ) =0, workisbuiltfromahierarchicalsystemoftransitdomains
1 1 |θ1=0+
and(∂2h1/∂θ2)<0onO ,whichimpliesthath1(θ )<0 connecting stub domains. The user specifies the number
1 1 1 1 1
5
Algorithm1Allocate Resources for Host k
1: whiletruedo
2: t:=timesincelastarrival/departure
3: forallagentsido
4: deducttgi(θ)fromagenti’sendowment
5: endfor
6: addnewagentorremovedepartingagent
7: forallagentsido
8: query agent i for α,β,and γ (equations (12-14))
tobuildg (θ)
i
9: endfor
10: searchforθ = Ni=1gi(θ)in(0,maxi(αi/βi)) Figure4.Ascreenshotofthesimulator.
11: forallagentsido
12: vki :=ckgi(θ(cid:80))/θ Onceanagentjumpstoasite,itcommitstofinishingthe
13: endfor current task at that site. To complete the current task, the
14: endwhile agentsubmitsabiddingfunctionconformingtotherestric-
tions in Section 5. Whenever an agent arrives or leaves a
host,thehostsearchesforanequilibriumcongestion,θ.
andaveragesizeofdomainsinnodes(hosts)aswellasthe To test the effectiveness of our resource allocation pol-
probabilitythatnodesareconnectedwithinthedomain. icy, we compare the endowment of agents with their per-
formance. Once the network has reached a steady state,
Agents are created at a Poisson arrival rate. Their start
we designate seven percent of the agents injected into
locationisdetermineduniformlyandtheyhaveanexponen-
the system as test agents. Our test agents have identical
tiallydistributednumberofjobsintheiritinerary. Eachjob
itinerariesandstartinghosts,buttheyhavedifferingendow-
size is chosen from an exponential distribution. The final
mentsizes,spanningtwostandarddeviations,σ,aroundthe
parameter describing an agent is its endowment size rela-
meanendowment,µ. Wethenmeasuretheperformanceof
tivetothesumofitsjobsizes. Wegeneratethisparameter
agentscomparedwiththeperformancethattheywouldhave
from a truncated Gaussian random variable with positive
achieved in a network with zero congestion. We compute
mean. Thisparameterreflectstheowninguser’spreference
this idealized measurement by computing a shortest path
thatthetaskcompletesquickly.
through the network where the distance between any two
Oncean agent is injectedinto the network, it must for-
hostsisthesumofthenetworktransfertimesandthetime
mulate a route. It choosesa route incrementally, choosing
an agent would take to complete a job at the host without
ahostforeachtaskaftercompletingtheprevioustask. For
anycompetition,q /c .
the purpose of expenditure planning, however, for all but k k
Figure 5 shows a plot of agent endowment versus per-
thenextimmediatehostchoice,agentsplantovisithostsof
formance relative to the ideal. The figure plots the mean
average capacity c , and average congestion θ , for hosts
k k performance of many agents at various endowment levels,
offeringthecorrespondingservice. Agentsthenchoosethe
as well as the standard deviations. There is a very strong
next site to be the one that minimizes the sum of execu-
relationship between spending and the speed at which an
tion times, assuming no change in bidding level θ−i, and
k agentcompletesitsitinerary. Weshowaχ2linearfitofthe
networktransfertimes for thenext hop. Thus,our routing
performancemeanstoindicatethestrengthoftherelation-
algorithm is greedy and naive. We sketch its operation in
ship.
Algorithm2.
7 Discussionand FutureWork
Algorithm2Choose Next Site for Agent i
1: tmin :=∞; nextHost:=∅ We have presented a simple resource-allocation policy
2: forallhostskofferingservicenextinitinerarydo and an efficientmeans for agents to plantheir expenditure
3: tk := [transferLatency: k from: giventhepolicyandperfectinformation.Becauseagentsre-
currentHost]+c g(θ−i)/(θ−i+g(θ−i))
k k k k formulatetheirexpenditureplansaftercompletingeveryjob
4: iftmin >tk then intheiritineraryandthebiddingstrategiespreserveagents’
5: tmin :=tk; nextHost:=k budget constraints, theoretically agents will always com-
6: endif plete their itineraries. In the implementation of our sim-
7: endfor ulator, however, we use discrete time units and round up
8: returnnextHost when assessing agents’ costs to the host, so occasionally
6
0.4 methods to determine the price of a certain resource in a
0.35 observed givenmarketstructure.Weinvestigatedthederivationofthe
mean
equilibriumpricingschemeinasinglesellermarketwitha
me 0.3 linear fit
Ti numberofbuyers,havingaCobb-Douglastypeutilityfunc-
ual 0.25 tion[4]. Inmorerecentwork,wedeterminepricesinamar-
ct
e/A 0.2 ketstructurein whichtheservers provideapricecurve for
m
Ti 0.15 the resources that they sell, and we obtain the equilibrium
eal 0.1 priceastheresultofautilitymaximizationproblem[15].
d
I We are not the first to research the possibility of using
0.05
market-basedcontrolfor mobile-agent systems. Telescript
0
m-2s m-s m m+s m+2s supported a system where agents carried permits whose
Endowment strength diminished over times and as they moved around
thenetwork[18]. Theweakeningofpermitsapproximates
Figure 5. Endowment versus ideal time rel- currency exchange. TheGeneva Messengersmobile-agent
ative to actual time. We plot the observed system includes support for agents to buy CPU priority,
mean and standard deviations for each en- memory, and network access [17]. Neither system, how-
dowmentlevel. ever, explores howagents shouldplanin market-baseden-
vironments.
POPCORN is a market-based system where users sub-
agentscannotcompletetheiritineraries. Thisoccurrenceis mitJavaprogramstoacentralizedserverthatcontractsthe
lessfrequentwithfinertime-unitgranularity. Furtheragent programs’executionsouttoservers[16]. Userprogramsin
failuresoccurwhen agents withsmall endowments cannot POPCORN,however,arenotautonomouslymobileanddo
completetheiritinerariesbecausehigherprioritycongestion notrelocateonceassignedtoaserver.
neversubsides. Modeling telecommunication network problems as dy-
Ourplanningalgorithmsrelyonexactlyknowingagents’ namic games has produced Nash equilibria solutions in
job sizes. This assumption is reasonable for certain appli- many settings such as capacity allocation in routing [10],
cation domains. However, we would like to relax this re- congestioncontrolinproductformnetworks[11],flowcon-
quirementin our future work byexploring other strategies trolinMarkovianqueuingnetworks[7],andcombinedrout-
whereagentshadprobabilitydistributionsfortheirjobsizes ing and flow control [1]. Incorporating electronic market
insteadoffixedscalarvalues. conceptssuchaspricingcongestiblenetworkresourceshas
Inconstructingour expenditure planningalgorithm, we beenshowntoencourageefficiency[14].Networkresource
assumed that agents have already chosen a route to com- allocationwhereusers werecharged perunittimehasalso
pletetheir itineraries. In theimplementation, however, we beeninvestigatedundervariouscriteriaoffairness[8,9].
can only estimate the future capacity of hosts based upon
aggregate host statistics and we choose agents’ routes in a 9 Conclusion
greedy fashion. This technique appears to work well for
a simple network topology but would fall short in a more
Wepresentedasimplemeansofresourceallocationthat
complex network, such as a wireless network where hosts
prioritizes agents through a single parameter, endowment
moveabouttheirenvironments.Thus,anotherareaoffuture
size. Toutilizethemechanism,wederivedabiddingpolicy
research is to investigate better ways of calculate agents’
that minimizes an agent’s execution time for an itinerary,
routes.
whilepreservingafixedbudgetconstraint.Furthermore,we
Finally, market systems require that agents have infor-
proved that the use of our optimal bidding strategy results
mationfromwhichtomakedecisions. Ourfutureresearch
ina uniquecomputableNashequilibrium. Wesimulateda
willattempt to evaluate both the costof distributinginfor-
networkofmobileagentsandtheirhostsusingourbidding
mation,suchassitecongestion,aswellasthecostofagents
strategyandresource-allocationpolicytoshowthatagents
usinginaccurateordatedinformation.
areprioritizedbytheirendowments.
8 RelatedWork
Appendix Proof ofConcavity ofgi(θ1)
Ourearlier work in this area adopted a currency model
ofresourceallocation, in whichagents use electroniccash For the variables αi,βi,γi defined in equations (12-4)
to purchase needed resources and participate in an elec- wedropthesuperscriptsandconsiderthebiddingfunction
tronicmarketwithabankingsystem[3]. Thereareseveral ofonlysingleagent. Letw(x)beafunctiondefinedas:
7
[3] J.Bredin,D.Kotz,andD.Rus. Market-basedresourcecon-
w(x)= x2+ x4 bx3+bαx2 (29) trolformobileagents.InProceedingsoftheSecondInterna-
− − tional Conferenceon Autonomous Agents,pages 197–204,
wherex (0,α)andb=(4(cid:112)γ2/β). Minneapolis,MN,May1998.ACMPress.
∈
Then,g (θ )= 1 w(α βθ )forθ O and [4] J.Bredin,D.Kotz,andD.Rus. Utilitydrivenmobile-agent
1 1 2γ2 − 1 1 ∈ 1 scheduling. Technical Report PCS-TR98-331, Dartmouth
College,May1998.
∂2g /∂θ2 =(β2/2γ2)(∂2w/∂x2). (30) [5] J.Bredin,D.Kotz,andD.Rus. Mobile-agentplanningina
1 1 market-orientedenvironment.TechnicalReportPCS-TR99-
345,DartmouthCollege,May1999.
Toprove the concavity of g on O , it suffices to show
1 1 [6] K. Calvert and E. Zegura. GT-ITM: Geor-
that(∂2w/∂x2)<0forx (0,α).Wehave
∈ gia Tech internetwork topology models, 1996.
http://www.cc.gatech.edu/fac/
Ellen.Zegura/gt-itm/gt-itm.tar.gz.
∂2w/∂x2 =(2p(x)p(cid:48)(cid:48)(x) p(cid:48)(x)2 8p(x)32/4p(x)23) (31) [7] M.T. Hsiao and A. A. Lazar. Optimal decentralized flow
− −
controlofMarkovianqueueingnetworkswithmultiplecon-
wherep(x) = x4 bx3+bαx2. Since p(x) > 0for x trollers. PerformanceEvaluation,13:181–204,Nov.1991.
− ∈
(0,α),itissufficienttoshow [8] F.P.Kelly.Chargingandratecontrolforelastictraffic.Euro-
peanTransactionsonTelecommunications,8:33–37,1997.
[9] F.P.Kelly,A.K.Maulloo,andD.K.H.Tan.Ratecontrolin
v(x)=2p(x)p(cid:48)(cid:48)(x) p(cid:48)(x)2 8p(x)32 <0. (32) communicationnetworks:shadowprices,proportionalfair-
− − nessandstability. JournaloftheOperationalResearchSo-
ciety,49:237–252,1998.
Aftersubstitutingforp(x)andsimplifying,weget
[10] Y. Korilis and A. Lazar. Architecting noncooperativenet-
works.IEEEJournalonSelectedAreasinCommunications,
v(b,x)= 4αb2x3+12αbx4+3b2x4+8x6 13:1241–1251,Sept.1995.
− (33) [11] Y. Korilisand A. Lazar. On the existenceof equilibria in
−12bx5−8(x4−bx3+bαx2)23 noncooperativeoptimalflowcontrol. JournaloftheAssoci-
ationforComputingMachinery,42:584–613,May1995.
Wenotethatv(0,x) = 0 x. Takingthepartialderiva- [12] D.B.LangeandM.Oshima. Sevengoodreasonsformo-
∀
tiveofv(x)w.r.t. b,weget bileagents.CommunicationsoftheACM,42(3):88–89,Mar.
1999.
[13] C. Langton, R. Burkhart, M. Daniels, and A. Lan-
∂v = (α x)[6bx3+12x2γ(x)] 2bαx3 caster. The Swarm simulation system, 1999.
∂b − − − (34)
γ(x) = x4+bx2(α x) x2 http://www.santafe.edu/projects/swarm.
− − [14] J.K.Mackie-MasonandH.R.Varian. Pricingcongestible
hence∂v/∂b(cid:112)isnegativeforx (0,α)andb>0. networkresources.IEEEJournalofSelectedAreasinCom-
∈ munications,13(7):1141–1149,Sept.1995.
v(b,x) < 0for allb > 0for x (0,α),whichimplies
∈ [15] R.T.Maheswaran,C¸agriImer,andT.Bas¸ar.Agentmobility
(∂2w/∂x2) < 0forx (0,α),andthus (∂2g /∂θ2) < 0
∈ 1 1 underpriceincentives.InProceedingsof38thIEEEConfer-
onO .
1 enceonDecisionandControl,Phoenix,AZ,Dec.1999. (to
appear).
Acknowledgments [16] O.RegevandN. Nisan. ThePOPCORN market—anon-
linemarketforcomputationalresources. InProceedingsof
theFirstInternationalConferenceonInformationandCom-
This work is supported in part by Department of De-
putation Economies,pages 148–157, Charleston, SC, Oct.
fensecontractMURI F49620-97-1-0382andDARPAcon- 1998.ACMPress.
tractF30602-98-2-0107. [17] C.F.Tschudin.Openresourceallocationformobilecode.In
ProceedingsofTheFirstWorkshoponMobileAgents,pages
186–197,Berlin,April1997.
References
[18] J.E.White. Telescripttechnology:Mobileagents. General
MagicWhitePaper,1996.
[1] E. Altman, T. Bas¸ar, and R. Srikant. Nash equilibria for
combinedflowcontrolandroutinginnetworks:Asymptotic
behaviorforalargenumberofusers.InProceedingsof38th
IEEE Conference on Decision and Control, Phoenix, AZ,
Dec.1999. (toappear).
[2] T.Bas¸arandG.J.Olsder. DynamicNoncooperativeGame
Theory.ClassicsinAppliedMathematics,SIAM,1999.
8