DTIC ADA492859: Tiered Auctions for Multi-Agent Coordination in Domains with Precedence Constraints

0.49 MB·English

by  Defense Technical Information Center

TIEREDAUCTIONSFORMULTI-AGENTCOORDINATIONINDOMAINSWITH
PRECEDENCECONSTRAINTS
E.GilJones*,M.BernardineDias,andAnthonyStentz
RoboticsInstitute,CarnegieMellonUniversity
Pittsburgh,PA,15213
ABSTRACT rather than employing a single robot operating indepen-
dently. A team of robots acting together can often out-
Many applications require teams of robots to coop-
perform single robot solutions in terms of quality and ro-
eratively execute complex tasks. Among these domains
bustness; at the same time, equipping multiple robots to
are some that require robots to interact closely at partic-
collectivelyaddressapplicationtasksrequiresaddressinga
ular times and locations to accomplish some task compo-
substantialresearchchallengeofcoordinatingtheeffortsof
nents, but otherwise allow the team members to act in-
therobots. Tomitigatethedifficulties associated withco-
dependently. Successful execution in such domains of-
ordinatingateamofrobotsresearchinmulti-robotsystems
ten requires agent interactions that must adhere to con-
hascenteredaroundapproacheswhereagentseachworkby
straints of precedence. Precedence requirements can oc-
themselves. These approaches are best suited to domains
cur when agents’ plans call for certain pre-conditions to
with independent tasks, where the efforts of single robots
bemetatparticular timesandplaces. Inthis workwefo-
actingindependentlyaresufficient. Thisworkconcernsit-
cusonprecedence-constrainedemergencyresponse.Inthis
self, however, with domains in which agents must work
domain a group of fire trucks agents attempt to navigate
togethermoreclosely;weareparticularlyinterestedindo-
throughacityinordertoextinguishasetoffiresthathave
mainsthatrequirethatsomeagentssatisfyprecedencecon-
occurredinthewakeofalarge-scaledisaster. Anotheref-
straintstoenableotheragentstoaddressdomaintasks.Co-
fectofthedisasteristhatdebrishaveblockedroadsinthe
ordinating agents in domains with precedence constraints
city, making roads impassable for the fire trucks. Debris
involves addressing substantial new challenges associated
canbeclearedbybulldozerrobots, whicharealsooperat-
withtheinterdependenceofagents’schedules.
ing in the environment. To maximize fire fighting perfor-
mancefiretrucksandbulldozersmustdeterminewhenand Inthisworkwefocusonasinglemotivatingdomain:
wheredebrisclearanceinteractionsshouldoccurandwho precedence-constrained emergency response. In this do-
shouldbeinvolved. main a team of robots capable of extinguishing fires is
operating in a city that has been ravaged by a disaster of
Our proposed method for coordination in domains
significant proportions. These fire truck robots are tasked
withprecedence-constrainedinteractionsisamarket-based
with moving to various locations around the city to extin-
approachtoplanning,allocating,andschedulingthatusesa
guish fires that have been reported by an autonomous air
noveltieredauctionframework. Thetieredauctionframe-
vehicle; new fires are frequently discovered. Time is of
workallowsagentstosolicittheassistanceofotheragents
the essence, and the robots must move to the locations of
in determining their suitability for a task; in this frame-
the fires and extinguish them as quickly as possible. Un-
workagentsholdsub-auctionstodecidewhatinteractions
fortunately, the disaster has rendered many of the roads
may best address application constraints. For the emer-
in the city impassable; they are covered with debris and
gencyresponsedomainweproposeatieredauctionmethod
wreckage, creating obstacles around which the bulky fire
that uses single-task fire truck assignment at the top tier
extinguishingrobotscannotnavigate. Supposethatthereis
andmulti-taskbulldozerassignmentatthesecondauction
another group of bulldozer robots in the team that are de-
tiersothatfiretruckscanfindmaximallyefficientroutesto
signedtomovefreelyinroughterrain,andcanclearroads
fires. We show that the tiered auction approach improves
of wreckage and debris. The bulldozer agents can assist
overastandardsingle-tiermarket-basedapproachinasim-
the fire truck agents by satisfying debris clearing precon-
ulatedemergency-responsedomain.
ditions along routes to fires. We suppose that there is a
globalobjectivefunctionthatisasumovertime-decreasing
1.INTRODUCTION rewardsofferedforextinguishingfires. Addressingtheco-
ordinationproblemposedbysuchadomainsoastomax-
Research efforts in robotics have increasingly turned imize global objective reward requires not only assigning
towardsusingteamsofrobotstocollectivelyaddresstasks firetruckstofires, butalsoselectingroutesthatthetrucks
1
Report Documentation Page Form Approved
OMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington
VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it

does not display a currently valid OMB control number.
1. REPORT DATE 3. DATES COVERED
2008 2. REPORT TYPE 00-00-2008 to 00-00-2008
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
Tiered Auctions for Multi-Agent Coordination in Domains with
5b. GRANT NUMBER
Precedence Constraints
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
Carnegie Mellon University,Robotics Institute,Pittsburgh,PA,15213 REPORT NUMBER
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT
NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT
Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES
14. ABSTRACT
see report
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF
ABSTRACT OF PAGES RESPONSIBLE PERSON
a. REPORT b. ABSTRACT c. THIS PAGE Same as 8
unclassified unclassified unclassified Report (SAR)
Standard Form 298 (Rev. 8-98)
Prescribed by ANSI Std Z39-18
willtakethroughthecityandassigningbulldozerstoclear mechanisms for coordinating given the presence of
debrisalongthoseroutes. inter-task constraints, all are computationally intensive
and not well-suited for efficiently searching the large
In this work we propose a novel method, tiered auc-
space of possible fire truck routes and associated debris
tions,thatenablesustoefficientlysearchthespaceofinter-
requirements associated with our domains of interest. We
dependentagentplanstodetermineallocationsandsched-
leave comparison of our methods with these methods in
ules that maximize an objective function associated with
termsofperformanceandcomputationasfuturework.
the emergency response domain. Our method expands on
existing auction-based coordination methods for domains
with independent tasks by equipping agents with the abil- 3.METHODS
itytosolicitassistanceduringallocationtoaccountfortask
constraints. We will show that our method outperforms a In this section we detail our approach to multi-agent
methodthatallocatestaskswithoutconsideringprecedence coordinationfordomainswithprecedenceconstraints.One
constraints in a simulated precedence constrained emer- reasonable approach we could take is to use a mini-
gencyresponsedomain. mally modified version of a method from our previous
work, where we used auctions for coordination in a fire-
We next address related work, and then describe our
fighting emergency response domain with deadlines but
approach in greater detail. Experimental results and our
no precedence constraints (hence no need for bulldoz-
conclusionsfollow.
ers); our sequential single-item auction approach is de-
scribedin[Jonesetal.,2007]. Inourapproachwhenemer-
2.RELATEDWORK gencytasksarediscoveredtheyarepassedtoacentraldis-
patcher/auctioneer (D/A), who keeps a list of tasks to be
allocated. This agent periodically announces a task auc-
Task allocation for tasks that are not inter-related by
tion and sequentially auctions tasks to all available emer-
constraints has been widely studied. TraderBots uses a
gency response agents. Each task is assigned to the fire
market-basedapproachwithsequencingforawidevariety
truck agent that determines that it can generate the most
of domainswith different independenttasks and objective
additional value in terms of an objective score for per-
functions[Diasetal.,2004][Zlotetal.,2002]. Berhaultet
formingataskgiventhatitalreadymayhaveanumberof
al. useacombinatorialauctionapproachfortime-extended
taskassignments. Onepossiblewaytoadaptthisapproach
assignment of exploration tasks with a travel cost metric
toourprecedence-constrainedemergencyresponsedomain
[Berhaultetal.,2003]. Zlot [Zlot,2006] focuses on do-
wouldbetohavefiretrucksbidfortasksbasedoncalculat-
mains with complex tasks that must be broken down into
ingshortestpathdistancestonewfiretasks;thisapproach
independenttasksforexecution. Aswewillshow,taskal-
wouldnothavefiretrucksreasonaboutdebrisalongpaths
location methods in precedence-constrained domains that
orbulldozerschedulesandallocations. Oncefiretaskshad
do not consider constraints between tasks do not perform
beenassignedthefiretruckswouldthenusesomemethod
wellincomparisontoourproposedmethod.
torecruitbulldozerstodebristasksasneeded. Wecallthis
Some previous work has explicitly consider do- the “allocate-then-coordinate” approach. While this ap-
mains where tasks are inter-related by constraints proachisstraight-forwardandwouldrequireminimalmod-
[MacKenzie,2003],[Lemaireetal.,2004]. Inneitherwork ificationstoourpreviousapproach,wethinkitwillachieve
do they propose a mechanism for searching a large space lowperformanceinscenarioswithmorethanafewdebris,
of possible plans for tasks; in our domains of interest fire asdebrisdensityandbulldozeravailabilitybecomeincreas-
trucks need to efficiently search a huge space of possible inglyimportantfactorsinsolutionquality.
routeplans.
Our proposed method for allocation in precedence-
The final relevant area of work is associated with constrained domains is a tiered auction approach where
multi-robot coordination for domains where tasks require agents try to reason simultaneously about allocation and
the joint efforts of multiple agent. Approaches to these route planning. In order to solicit the contributions of
domains are generally called coalition formation algo- otheragentsduringthebiddingprocesswemustextendthe
rithms,astheyseektoformcoalitionsofagentstoaddress single-tiered auction approach: our system for emergency
tasks [VigandAdams,2006] [TangandParker,2007] responsecoordinationusestwoauctiontiers.Atthetoptier
[Jonesetal.,2006][Sarieletal.,2007]. Koesetal. usean firetasksareauctionedtofiretrucksusingsingle-itemse-
MILP-based formulation for a search and rescue domain quentialauctions. Inourapproachtrucksonlybidonfires
where tasks have associated rewards that decay linearly, if they are currently idle, having completed their last as-
and the system can be constrained in a variety of ways signed fire. We use this approach – called instantaneous
including task ordering, time-oriented task constraints, allocationinaccordancewiththewidely-usedmulti-agent
capability constraints, and global resource constraints approach nomenclature proposed by Gerkey and Mataric´
[Koesetal.,2005]. While all of these methods contain [GerkeyandMataric´,2004] – in order to limit problem
2
complexity;multiplefireassignmentisleftasfuturework. ii. Eachtruckconsiderseachdebrispilealong
Inordertobidonfirestrucksmustsearchthroughthespace Pintheorderthattheywillbeencountered
ofpossibleroutestothefire;toassessaroutefiretruckcan whenfollowingP.
hold sub-auctions to recruit bulldozer assistance to clear iii. Some debris may already be allocated to
debris along the route. These sub-auctions comprise the bulldozersaspartofsomeexistingcommit-
second tier of auctions. Fire trucks may need to hold a ment. The truck confers with the D/A to
seriesofsub-auctionsfordifferentroutestodeterminethe get information on the scheduled comple-
route which will allow them to reach the fire being auc- tiontimeofthesedebrispiles.
tionedasquicklyaspossible.
iv. Ifsomedebrishavenotbeenpreviouslyal-
located, thefiretruckgeneratesanewauc-
While assigning a single fire per fire truck can sub-
tioncontextC associatedwithP-thiscon-
stantiallyreduceproblemcomplexity,assigningonlyasin-
textisassociatednotonlywiththefireFbut
gledebrispileperbulldozercouldpotentiallyresultinpoor
withP,aparticularrouteforthefiretruckto
performance,especiallyfordomainswithdensedebris,as
reachF.
theremaybemoredebrisalongdesirableroutesthanthere
v. ForeachunallocateddebrisDthefiretruck
areavailablebulldozers. Thuswhileweuseinstantaneous
runsasub-auction:
assignmentforfiretruckallocation,ourapproachsupports
assigning multiple tasks to bulldozers - we use a partially A. A call for bids for D is sent to each
time-extended approach in the Gerkey and Mataric´ par- bulldozer. Thecallcontainstheauction
lance. In our approach bulldozers can be assigned an ar- context,thedebrisID,andthedebrislo-
bitrary number of tasks, but will only bid on tasks in de- cation.
brissub-auctionsbasedonaddingtaskstotheendoftheir B. Ifabulldozerdoesnothaveaprevious
schedules. As we do not equip bulldozers to alter the or- schedule associated withC it creates a
derofassignedtasksintheirschedulesorinsertnewtasks new schedule associated with C - this
atarbitrarypointsintheirschedules, ourapproachisonly schedule is just a copy of its current
partiallytime-extended. Fullytime-extendedbulldozeral- schedule.
locationisleftasfuturework. C. EachbulldozeraddsDtotheendofits
schedule associated withC, creating a
Wefirstdescribeourapproachatahighlevel,andthen
candidateschedule.
illustrate approach details using an example scenario. We
D. The bulldozers then bid their comple-
concludeourmethodssessionwithadiscussionofbound-
tiontimeforDbasedontheircandidate
ingapproachesweusetolimitthesearchspaceofpossible
schedules.
coordinatedschedules.
E. Thefiretruckawardstheauctiontothe
bid corresponding to the earliest task
completiontime.
3.1 ApproachOverview
F. The winning bulldozer adopts the can-
didate schedule as its new context
The goal of the auction process is to assign each idle
scheduleforC.
firetruckasinglefire. Duringtheassignmentprocessfire
G. Thelosingbulldozersdiscardtheircan-
trucksdeterminewhichroutetotaketothefireandassign
didate schedules, and will bid on fu-
eachdebrispilealongtheroutetoabulldozer. Theauction
turetasksusingtheirpreviousCcontext
processproceedsasfollows:
schedules.
H. EachunallocateddebrisalongPisallo-
1. Thedispatcher/auctioneer(D/A)generatesalistofun- catedinthismanner.
allocatedfiresthatcanpotentiallybeassignedtoidle
vi. Whenallunallocateddebrishavebeenallo-
agents.
cated,truckscancomputetheircompletion
timeforF. IftherewardforF isbetterthan
(a) TheD/Aannouncesanauctionforthetaskatthe
the best previously considered route it be-
top of the list. An auction call for that fire F is
comestheleadingcandidateschedule. Oth-
senttoeachfiretruckagent.
erwisethescheduleandallassociatedbull-
(b) All idle fire trucks enter into an sub-auction dozerschedulescanbediscarded.
loop: (c) The sub-auction loop concludes when no paths
toF couldofferafastercompletiontime.
i. An A* algorithm returns a path P from a
truck’scurrentpositiontothefirebeingauc- (d) The fire trucks then place bids for F based on
tioned. scheduledtaskreward.
3
2. A task is awarded to the single fire truck that can
achievethehighestrewardgivenallbidsforalltasks. Truck C
3. The winning fire truck adopts the schedule it has Dozer B
X (120)
stored for the task it has won and informs bulldozers
to adopt their context schedules associated with the Dozer A Dozer C
winningbid.
Truck D
4. If any fire trucks remain untasked, the auction loop Y (160)
continues.
Truck B
Dozer D
Aftertheauctionprocessconcludesallagentscanthen Context 10001
executetheirschedules. Theauctionprocessrepeatswhen
anyfiretruckbecomesidle. Z (100)
Truck A
Figure 2: Fire truck A holds a sub-auction for its short-
3.2 Examplescenario
estpathroute(thebluedottedline)tofireY,whichpasses
through two unallocated debris. Truck A creates context
10001 for the route toY, and holds a sub-auction for the
Truck C first debris pile along the route. Bulldozers bid based on
shortestpathroutestothedebris.
X (120)
Dozer C
Dozer D
thenewcontextandaddthetasktotheiremptyschedules,
Dozer B bidding their completion times for the task. Bulldozer D
Dozer A Truck D
Y (160) winsthesub-auction,andreplacesitsemptycontextsched-
ulefor10001withthenewcontextschedulewiththedebris
Truck B task - all other bulldozers discard their candidate sched-
Dozer E ules and maintain their empty context schedules. The fire
truck then sub-auctions the next task along its path using
the same 10001 context designation. Now each bulldozer
Z (100)
bids based on adding the task to the end of their context
Truck A
schedules; bulldozer D bids for the new debris task based
Figure1: Examplescenariointheemergencyresponsedo- notonhavinganemptyschedulebutinsteadbasedonhav-
mainwithfourfiretrucks,fourbulldozers,andthreefires. ingalreadywonataskforthiscontext. Theresultingbull-
Current fire rewards are given in parentheses. Black lines dozerschedulesareshowninFigure3. Thesub-auctionis
represent roads, and debris are represented by oval boxes. awardedtobulldozerA.NotethatbulldozerDwasactually
FireYisthefirstfireselectedforauction. theclosetbulldozertobothdebrispiles,butthatassigning
both debris tasks to D would cause more delay to truck A
Wewillnowwalkthroughanexampleofourauction thantheallocationthatresultedusingcontexts.Withallde-
procedureinordertoexplicatethefunctionofoursystem. brisforthisparticularrouteallocatedtruckAcanactually
In Figure 1 we show a scenario with 4 fire trucks and 4 computeitscompletiontimeandassociatedrewardforfire
bulldozers. WesupposethattheD/Ahasgeneratedalistof Y given that this route would be taken. The computation
availablefiresandthatfireY isatthetopofthelistandwill of the completion time is shown in Figure 4. Now truck
be offered for auction first. We assume that all trucks are Amustconsiderotherroutestothefire;foreachrouteun-
idleandwillbidonthefire. der consideration it generates a new context and holds a
sub-auctionforallunallocateddebris. Figure5showsthe
We will focus on the planning procedure for truck A.
results of following another route toY - taking this route
Wesupposethatthetruck’sA*algorithmreturnstheshort-
resultsinaslightlyhigherrewardforY basedona2cycle
estdistancepathforconsiderationfirst. Thispathisshown
improvementincompletiontime.Notethatwhilethisroute
inblueinFigure2. TruckAdeterminesthatthereareunal-
islongerindistanceithasfewerdebrisandinthisinstance
locateddebrisalongthispathandgeneratesanewcontext
resultsinafastercompletiontimeforthefire.
10001associatedwiththisparticularrouteforreachingY.
Itthenusesasub-auctiontosolicitbulldozerassistancefor Each agent places a bid for fireY and reports the re-
thefirstdebristaskintheroute.Weassumethatallbulldoz- sults to the D/A, who then places a call for bids for fire
ersareidle;inthiscasetheygenerateemptyschedulesfor X, thenextfireintheD/A’slist. Whenallbidshavebeen
4
Truck C Truck C
Dozer B Dozer B
X (120) X (120)
Dozer A Dozer C Dozer A Dozer C
Truck D 15 Truck D
Y (160) Y (160)
4
5
Truck B Truck B
Dozer D 15 Dozer D
8
Context 10001 Context 10001
Z (100) Z (100)
Truck A Truck A
Figure3: Bulldozerschedulesusedforbiddingforthesec- Figure4: Withalldebrisallocatedthefiretruckcancom-
onddebrisintruckA’sroutetofireY. NotethatBulldozer puteitsarrivaltimeandresultingawardforthefireY. Pur-
D’s schedule for the second debris reflects that it has al- pleboxesshownpathlengthsfromtruckA’scurrentloca-
readywonapreviousdebrissub-auction. tiontothefirstdebrispile, fromthefirstdebrispiletothe
second debris pile, and from the second debris pile to the
fireY. ThebulldozerDtakes8cyclestoreachthedebris,
received for all fires, the D/A awards the single best bid
whichtakes6cyclestoclear,andsowillbeclearedattime
among all bids received for all fires. The fire truck with
14. Thus once the fire truck reaches the first debris pile
thewinningbidthenadoptstheassociatedcontextandin-
at time 15 it has already been cleared, and it can proceed
forms the bulldozers involved in the plan that they should
directlytotheseconddebrispile, whichitreachesattime
adopttheircontextschedulesaswell. Bulldozersthathave
20. Bulldozer A takes 15 cycles to reach the second de-
previouslybeenassigneddebriswillthenonlybidfornew
brispile, and8cyclestoclearit. Thusthefiretruckmust
debris tasks based on schedules that reflect their commit-
wait until time 23 before proceeding past the second de-
mentstoclearthepreviouslyallocateddebris. Theprocess
brispile. Itwillleavetheseconddebrispileattime23and
then repeats until all fires have been allocated or all fire
reach the fire at time 27. The fire takes 5 cycles to extin-
truckshavebeenassignedatask. Theresultingallocation
guish,meaningthattakingthisroutewillresultintruckA
forthisscenarioisshowninFigure6.
extinguishingthefireafter32cycles. Thefiredecaysat1.5
units/cyclefromanoriginalrewardof160,thusyieldinga
3.3 Boundingforfastercomputation scheduledrewardof112(160−(32∗1.5)).
Wehaveimplementedanumberofrefinementsinthe
above algorithm with the goal of making the auctioning
process faster and more efficient without sacrificing solu- coordinate”(ATC) approach we must describe our simu-
tionquality.Ourboundingmethodsstemfromtheobserva- lated experimental setup. Our fire truck agents operate in
tionthatduringanauctionroundonlyasingletaskwillbe a grid network of streets - truck planning is doing using a
awarded.Ifatanytime,then,anagentdeterminesthatafire graph-based map representation. Bulldozers plan using a
orroutetoafirecouldnotpossiblygeneratehigherreward grid-based map representation. In our simulated domain
thanonethatithaspreviouslyconsideredduringthesame we suppose that fire discovery is the product of a single
auction round it can stop evaluating that fire or route. We Poissonprocess, thestandarddistributionusedinqueuing
incorporatethisobservationintoboundingintoallstagesof theorytorepresentstochasticarrivaltimesofindependent
the route evaluation process, including path planning and tasks. The parameter λ for the Poisson process represents
thesub-auctionprocess. Thusinpracticeagentsgenerally the expected rate of task issuance. Our program is cur-
searchonlyasmallfractionofthefullspaceofallpossible rentlysetuptodirectlycomparetwodifferentapproaches;
pathstoallauctionedfires. Byemployingfrequentbound- to make the comparison as accurate as possible we have
ing and heuristics we can search the space more quickly agents operating under the two different approaches op-
withoutsacrificingsolutionquality. erate on exactly the same domain instance: agents be-
gin in the same locations, randomly-generated debris are
4.RESULTS placed in identical locations, and the same set of fires are
discovered at the same time between the two approaches.
Before we explore results comparing our tiered Theonlydifferencesbetweenthetestingenvironmentsare
auction allocation approach with an “allocate-then- causedbythedifferencesincoordinationmethods.
5
Truck C Truck C
Dozer B Dozer B
X (120) X (94)
Dozer A Dozer C Dozer A Dozer C
8
6 Y (160) Truck D Y (142)
Truck D
Truck B Truck B
19 Dozer D Dozer D
Context 10002
Z (100) Z (80)
Truck A Truck A
Figure 5: A different context yields a better route to fire Figure 6: All fires have been assigned. Truck paths are
Y. Path lengths for the fire truck shown in purple boxes. shown with blue dotted lines and green solid lines show
DozerAclearsthedebrisattime15,allowingthetruckto bulldozer paths to their assigned debris. Fire rewards at
reachY at25cyclesandextinguishitat30cycles,yielding theirscheduledcompletiontimesareshowninparentheses.
arewardof115.
We expect the performance difference between the
Thereareanumberofparametersthatgointoforming
ATCapproachandourtieredauctionapproachtobemost
aparticularinstanceofthedomain. Inourexperimentswe
pronounced in domain instances where precedence inter-
keep most of the parameters constant: we run our experi-
actionsarerequiredfrequently. Asnegotiatingprecedence
mentswiththreefiretrucksandthreebulldozersoperating
interactionsbecomesfrequentlyrequiredtocompletetasks
in a 4-by-6 city network. The composition of the build-
reasoningaboutthoseinteractionsduringallocationshould
ings in the city blocks, which the bulldozers must avoid
improve performance. Thus in our experiments we will
when moving to debris locations, are generated randomly
vary the frequency of required precedence interactions by
for each domain instance. In each domain instance there
alteringthenumberofrandomlygenerateddebrisinthedo-
were 10 known fires at time 0, and the λ value associated
main. Asweincreasethenumberofdebrisweexpectboth
withthePoissonprocesswassetto1,meaninganexpected
approaches to suffer in terms of overall reward obtained,
issuance rate of 1 fire per time cycle. Once at a fire site a
but we hope to show that the ATC approach performance
firetruckwillbeabletoextinguishthefireinasingletime
degradesatahigherratethanthetieredauctionapproach.
unit independent of fire size. We assume that fires have
an initial value that is drawn from a Gaussian distribution In the following experiments we tested 7 debris fre-
withameanof3000andastandarddeviationof100, and quency levels: 0, 50, 100, 150, 200, 250, and 300 ran-
that the reward for a fire linearly degrades at a rate of 25 domly generated debris. For each debris frequency level
units/cycleforallfires. we ran both approaches in 25 randomly generated shared
instances. In each case we tabulated results after 300 cy-
We now present the results of a comparison of our
clesbeforegeneratinganewdomaininstance.
tieredauctionapproach,whichconsidersprecedenceinter-
actionsduringallocationandanATCapproachwhichfirst Rewardsachievedforthetwoapproachesforthe7de-
allocates without considering precedence interactions and brisfrequencylevelsareshowninFigure7. With0debris
then determines coordination after allocation. We imple- thetwoapproachesperformidentically-inthiscase,each
mented the ATC approach using some of the components approach is just bidding using the shortest path distance.
of our tiered auction system. We use a single auction tier Eachapproachgainslessandlessoftheavailablereward-
for fire allocation; fire trucks bid on fires based on short- which varies around a constant average in each of the tri-
estpathschedulestofireswithoutholdingsub-auctionsor als - as the debris frequency increases. The most impor-
considering debris along the routes in any way. Once a tant trend to note is that the performance of agents using
truckhasbeenawardedafireitthensearchesamongpossi- thetieredauctionapproachdegradesmoreslowlythanthe
blepathstoreachthefireusingthesameprocedureusedin ATC approach, and is significantly higher at all non-zero
our tiered auction, holding auctions among the bulldozers debris levels. For non-zero debris frequencies, the tiered
as necessary. This approach essentially uses two separate auctionapproachoutperformstheATCapproachbyanav-
single-tierauctionsinsteadofourmulti-tieredauctionsys- erageof76%,witha140%improvementatthe300debris
tem. level.
6
x 105 250
s Tiered Auctions
Tiered Auctions sk Allocate−then−coordinate
5 a
Allocate−then−coordinate d t200
e
et
ard4 mpl
w o150
e c
e total r3 mber of 100
g u
a2 n
er e
v g 50
A a
er
1 v
A
0
0 50 100 150 200 250 300
0 Number of uniformly distributed debris
0 50 100 150 200 250 300
Number of uniformly distributed debris
Figure8: Averagenumberofsuccessfullycompletedtasks
fromthetrialsshownatleft.
Figure 7: Average performance averaged over 25 trials of
300timecyclesintermsofoverallrewardfortheATCand
tiered auction approaches for each of 7 different levels of
debris frequency in a simulated emergency response do- proachisover7timesascomputationallyexpensiveatthe
main. Standarddeviationsareshownaserrorbars. higherdebrislevels,evenwithoutfactoringinthecommu-
nicationtimethatwouldberequiredtorunsub-auctionson
realrobots.Thesubstantialincreaseinperformanceassoci-
atedwithusingtieredauctionscomesatthepriceofadded
Wecangainmoreinsightintothesourcesoftheperfor- computational costs. We could potentially lower the cost
mancedifferencebylookingatFigure8,whichshowsthe ofusingtieredauctionsbyusingmorerestrictivebounding
averagenumberofcompletedtasksforeachofthetwoap- atacostofreducedperformance.
proachesatthedifferentdebrislevels.Wecanseethatthese
curves closely resemble those for overall reward. Agents
CONCLUSIONS
using the tiered auction approach tend to complete tasks
more quickly on average as they are not overly optimistic
aboutthetimeitwilltaketoaccomplishtasksduringallo- In this technical report we have detailed meth-
cation. Considering coordination during allocation allows odsmulti-agentcoordinationforaprecedence-constrained
agents to precisely predict the time it will take to accom- emergency response domain. Our approach uses a novel
plish tasks which lets the system make more informed al- method, tiered auctions, that equips agents to efficiently
locationdecisions. search the space of routes and precedence interactions in
ordertodetermineplansthatwillyieldhighperformance.
While we have shown that using tiered auctions im- We validated our instantaneous approach by comparison
proves performance over an ATC approach, tiered auc- withan“allocate-then-coordinate”approachthatusedcon-
tionsarerelativelycomputationallyexpensive. Weapprox- ventional single-tiered auctions and did not allow agents
imatedthecomputationalcostoftheapproachesbyrecord- to reason about precedence interactions during allocation;
ing wall-clock time for each of algorithms during an auc- thisapproachiscomputationallycheapbutresultsinpoor
tioncycle. Anauctioncycleconsistsofallthecomputation performance, especially for domains with many required
necessarytoassignasingletasktoasingleagent-theroute precedenceinteractions.
planning and sub-auction times for each fire truck and all
thebulldozersforeachfirebeingauctioned,andthecostfor
the winning truck of adopting the fire, which for the ATC ACKNOWLEDGEMENTS
approachrequiresdoingfullrouteplanningtotheallocated
fire. Thisisonlyaroughapproximationofcomputational ThisworkwassponsoredbytheU.S.ArmyResearch
cost, as there are many factors that can affect wall clock Laboratory, under contract “Robotics Collaborative Tech-
time. Figure 9 shows the average time per auction for the nology Alliance” (contract number DAAD19-01-2-0012).
twodifferentapproaches.ThetimeperauctionfortheATC Theviewsandconclusionscontainedinthisdocumentare
approachgenerallyslowlyincreaseswiththenumberofde- thoseoftheauthorsandshouldnotbeinterpretedasrepre-
bris but remains very low for all levels, taking less than sentingtheofficialpoliciesorendorsementsofthetheU.S.
one-tenth of a second on average. The tiered auction ap- Government.
7
s) 1 [Jonesetal.,2007] Jones, E. G., Dias, M., and Stentz, A.
nd Tiered Auctions (2007). Learning-enhanced market-based task alloca-
o Allocate−then−coordinate
c tion for oversubscribed domains. In Proceedings of
se0.8 the IEEE/RSJ International Conference on Intelligent
n (
o RobotsandSystems(IROS).
cti
au0.6 [Koesetal.,2005] Koes, M., Nourbakhsh, I., and Sycara,
er K.(2005). Heterogeneousmultirobotcoordinationwith
p
e spatial and temporal constraints. In Proceedings of the
m0.4
al ti NationalConferenceonArtificialIntelligence(AAAI).
ot [Lagoudakisetal.,2005] Lagoudakis, M., Markakis, E.,
ge t0.2 Kempe, D., Keskinocak, P., Kleywegt, A., Koenig, S.,
a
er Tovey,C.,Meyerson,A.,andJain,S.(2005). Auction-
v
A 0 basedmulti-robotrouting.InRobotics:ScienceandSys-
0 50 100 150 200 250 300
tems.
Number of uniformly distributed debris
[Lagoudakisetal.,2004] Lagoudakis, M. G., Berhault,
Figure9: AveragetimeperauctionfortheATCandtiered M., Koenig, S., Keskinocak, P., and Kleywegt, A. J.
auction approaches for each of 7 different levels of debris (2004). Simple auctions with performance guaran-
frequencyinasimulatedemergencyresponsedomain. Tri- tees for multi-robot task allocation. In Proceedings of
alswereallruninasinglethreadonaquad-coreXeon3.8 the IEEE/RSJ International Conference on Intelligent
GHzCPUwith8GBsofmemory. RobotsandSystems(IROS).
[Lemaireetal.,2004] Lemaire, T., Alami, R., and
Lacroix, S. (2004). A distributed tasks allocation
REFERENCES
scheme in multi-UAV context. In Proceedings of
the IEEE International Conference on Robotics and
[Berhaultetal.,2003] Berhault, M., Huang, H., Ke-
Automation(ICRA).
skinocak, P., Koenig, S., Elmaghraby, W., Griffin, P.,
andKleywegt,A.(2003). Robotexplorationwithcom- [MacKenzie,2003] MacKenzie,D.C.(2003). Collabora-
binatorialauctions. InProceedingsoftheIEEE/RSJIn- tivetaskingoftightlyconstrainedmulti-robotmissions.
ternational Conference on Intelligent Robots and Sys- InMulti-RobotSystems:FromSwarmstoIntelligentAu-
tems(IROS). tomata: Proceedings of the 2003 International Work-
shop on Multi-Robot Systems, volume 2. Kluwer Aca-
[Dias,2004] Dias, M. B. (2004). TraderBots: A New
demicPublishers.
Paradigm for Robust and Efficient Multirobot Coordi-
nationinDynamicEnvironments. PhDthesis,Robotics [Sarieletal.,2007] Sariel, S., Balch, T., and Erdogan, N.
Institute,CarnegieMellonUniversity. (2007). Incremental multi-robot task selection for re-
source constrained and interrelated tasks. In Proceed-
[Diasetal.,2004] Dias, M. B., Zlot, R., Zinck, M., Gon-
ingsoftheIEEE/RSJInternationalConferenceonIntel-
zalez,J.P.,andStentz,A.(2004).Aversatileimplemen-
ligentRobotsandSystems(IROS).
tationoftheTraderBotsapproachtomultirobotcoordi-
nation. InProceedingsoftheInternationalConference [TangandParker,2007] Tang,F.andParker,L.(2007). A
onIntelligentAutonomousSystems(IAS). complete methodology for generating multi-robot task
solutionsusingAsyMTRe-Dandmarket-basedtaskal-
[GerkeyandMataric´,2004] Gerkey, B. P. and Mataric´,
location.InProceedingsoftheIEEEInternationalCon-
M. J. (2004). A formal analysis and taxonomy of task
ferenceonRoboticsandAutomation(ICRA).
allocationinmulti-robotsystems. InternationalJournal
ofRoboticsResearch,23(9). [VigandAdams,2006] Vig, L. and Adams, J. A. (2006).
Multi-robot coalition formation. In IEEE Transactions
[GrossandHarris,1998] Gross, D. and Harris, C. M.
onRobotics,volume22(4).
(1998).FundamentalsofQueueingTheory.Wiley,New
York,3rdedition. [Zlot,2006] Zlot,R.(2006). AnAuction-BasedApproach
toComplexTaskAllocationforMultirobotTeams. PhD
[Jonesetal.,2006] Jones, E. G., Browning, B., Dias,
thesis,RoboticsInstitute,CarnegieMellonUniversity.
M.B.,Argall,B.,Veloso,M.,andStentz,A.(2006).Dy-
namically formed heterogeneous robot teams perform- [Zlotetal.,2002] Zlot, R., Stentz, A., Dias, M. B., and
ing tightly-coupled tasks. In Proceedings of the IEEE Thayer,S.(2002).Multi-robotexplorationcontrolledby
International Conference on Robotics and Automation amarketeconomy. InProceedingsoftheIEEEInterna-
(ICRA). tionalConferenceonRoboticsandAutomation(ICRA).
8