Autonomous Bidding Agents: Strategies and Lessons from the Trading Agent Competition

244 Pages·2007·2.19 MB·English

by  Michael P. Wellman

Autonomous Bidding Agents
IntelligentRoboticsandAutonomousAgents
GeorgeA.Bekey,HenrikI.Christensen,EdmundH.Durfee,David
Kortenkamp,andMichaelWooldridge,AssociateSeriesEditors
RobotShaping:AnExperimentinBehaviorEngineering,MarcoDorigoand
MarcoColombetti,1997
Behavior-BasedRobotics,RonaldC.Arkin,1998
LayeredLearninginMultiagentSystems:AWinningApproachtoRobotic
Soccer,PeterStone,2000
EvolutionaryRobotics:TheBiology,Intelligence,andTechnologyof
Self-OrganizingMachines,StefanoNolfiandDarioFloreano,2000
ReasoningaboutRationalAgents,MichaelWooldridge,2000
IntroductiontoAIRobotics,RobinR.Murphy,2000
MechanicsofRoboticManipulation,MatthewT.Mason,2001
StrategicNegotiationinMultiagentEnvironments,SaritKraus,2001
DesigningSociableRobots,CynthiaL.Breazeal,2002
IntroductiontoAutonomousMobileRobots,RolandSiegwartandIllahR.
Nourbakhsh,2004
AutonomousRobots:FromBiologicalInspirationtoImplementationand
Control,GeorgeA.Bekey,2005
PrinciplesofRobotMotion:Theory,Algorithms,andImplementations,Howie
Choset,KevinM.Lynch,SethHutchinson,GeorgeKantor,WolframBurgard,
LydiaE.KavrakiandSebastianThrun,2005
ProbabilisticRobotics,SebastianThrun,WolframBurgard,andDieterFox,
2005
AutonomousBiddingAgents:StrategiesandLessonsfromtheTradingAgent
Competition,MichaelP.Wellman,AmyGreenwald,andPeterStone,2007
Autonomous Bidding Agents
StrategiesandLessonsfromtheTradingAgentCompetition
MichaelP.Wellman,AmyGreenwald,andPeterStone
TheMITPress
Cambridge,Massachusetts
London,England
(cid:2)c 2007MassachusettsInstituteofTechnology
All rights reserved. No part of this book may be reproduced in any form by any electronic
ormechanical means (including photocopying, recording, orinformation storage andretrieval)
withoutpermissioninwritingfromthepublisher.
ThisbookwassetinTimesRomanbytheauthorusingtheLATEXdocumentpreparationsystem.
PrintedonrecycledpaperandboundintheUnitedStatesofAmerica.
LibraryofCongressCataloging-in-PublicationData
Wellman,MichaelP.
Autonomousbiddingagents:strategiesandlessonsfromthetradingagent
competition/MichaelP.Wellman,AmyGreenwald,PeterStone.
p. cm. —(Intelligentroboticsandautonomousagentsseries)
Includesbibliographicalreferencesandindex.
ISBN978-0-262-23260-9(hardcover:alk.paper)
1.Electroniccommerce. 2.Intelligentagents(Computersoftware)
I.Greenwald,Amy II.Stone,Peter,1971– III.Title.
HF5548.32.W465 2006
338.4’3—dc22 2006034223
10987654321
toErika,Justin,andTammy
Contents
Preface ix
1 Introduction 1
2 TheTACTravel-ShoppingGame 9
3 BiddinginInterdependentMarkets 33
4 PricePrediction 61
5 BiddingwithPricePredictions 81
6 MachineLearningandAdaptivity 117
7 Market-SpecificBiddingStrategies 143
8 ExperimentalMethodsandStrategicAnalysis 169
9 Conclusion 195
AppendixA:
TournamentData 205
AppendixB:
IntegerLinearProgrammingFormulations 219
References 227
CitationIndex 233
SubjectIndex 235
Preface
This book distills the experiences and lessons learned from the international
TradingAgentCompetition(TAC)series.MotivatedbyTAC,acommunityof
academicandindustryresearchershasbeeninventingandpolishingtechniques
forautonomousbiddingbysoftwareagents. We, the authors,havebeenboth
organizersof TAC and successful participants. As such, we have tackled the
problemsposedbyTACwithourownindependentefforts,andwehaveclosely
observedtheevolutionofapproachesdevelopedbythecommunityasawhole.
TAC is a stylized setting exemplary of the rapidly advancing domain of
electronicmarketplaces.Itisalsoabenchmark,motivatingresearcherstoapply
innovative approaches to a common task. A key feature of TAC is that it
providesanacademicforumforopencomparisonof agentbiddingstrategies
inacomplexscenario,asopposedto,forexample,automatedtradinginreal-
worldsecuritiesmarkets,inwhichpractitionersarelessinclinedtosharetheir
technologies.Astheproductofsustainedfocusandcross-fertilizationofideas
over time, TAC provides a unique case study of the current capabilities and
limitationsofautonomousbiddingagents.
Throughoutthe text, we balance the contextualreporting of results from
thespecificTACscenariowiththedesiretogeneralizetothebroaderproblem
ofautonomousbidding.Togroundthediscussion,weincludesubstantialdata
from controlled TAC experimentsand TAC tournaments,methods employed
byparticularTACagents,andanecdotesfromTACevents.Togeneralizethese
lessons and techniques, we develop a generic trading agent architecture uni-
fyingthe approachesobserved,define abstractversionsof tradingagentsub-
problems,andhighlightimportantpropertiesoftheseproblemsandproposed
solutionsthroughtheoreticalandexperimentalanalysis.
We consider this dual approach—intensivedesign focused on a concrete
scenario, interleaved with abstraction and analysis aimed at drawing general
lessons—essential for deriving principled trading agent designs. Real-world
marketsaretoocomplextorelysolelyonabstractmodeling,andspecificmar-
kets are too idiosyncratic to admit direct transfer of techniques. By testing
general ideas in particular scenarios, we are forced to work through opera-
tional details that tend not to arise in more abstract models. Throughcareful
evaluationofproposeddesigns,wecanachievesomeconfidenceintheirvia-
bility,andgatherevidenceabouttheirlimitations.Liftingthemethodsbackup
tomoregenericmarketscenariosenablesadaptationtosimilarenvironments,
andeventransferacrossqualitativelydifferentmarketdomains.
The main contributionsof this book are (i) the story of the development
andevolutionofthe TAC researchinitiative,includinganecdotalaccountsof
x Preface
TAC agent interactions over the years; (ii) detailed analyses of specific TAC
agentdesignsandbiddingtechniques;and(iii)developmentofsome general
engineering foundationsfor trading agent design. Our intended audience in-
cludes individuals interested in developing TAC agents, but we expect most
readers are primarily motivated by other trading domains. By inviting all to
immerseyourselvesintheTACdomain,weleadyouontheverypathwehave
takenindevelopingourunderstandingofthecurrentbestpracticesfordesign-
ingautonomousbiddingagents.
Acknowledgments
By its very nature, the Trading Agent Competition is a collective enterprise,
andthisbookwouldnothavebeenpossiblewithoutthecontributionsofscores
of developerswho have implemented TAC agents over the years. We would
especially like to thank the people centralto the design and operationof the
TACTravelgameservers:PeterWurmanandKevinO’Malley(Universityof
Michigan)and Sverker Janson, Joakim Eriksson, and Niclas Finne (Swedish
Institute of Computer Science). Individualsserving as GameMaster over the
yearsare listed in AppendixA. Other volunteerswho have served important
roles enabling the TAC Travel tournamentsinclude Eric Aurell, Maria Fasli,
NickJennings,DavidParkes,NormanSadeh,andShihomiWada.
Many of the results presentedherein were producedcollaborativelywith
our colleagues and students. In most cases, specific attribution is provided
by citations throughout the book to articles previously published in various
journalsandconferenceproceedings.Thecontributionsofthesecoauthorsand
otherslistedbelowaretoagreatextentinseparablefromourown.
ATTac was developedby Peter Stone in collaborationwith colleaguesat
AT&TLabs—Research:MichaelLittman(ATTac-00andATTac-01);Michael
KearnsandSatinderSingh(ATTac-00);andJa´noszCsirik,DavidMcAllester,
and RobertSchapire (ATTac-01).Many of the ideasdescribed in connection
withATTac,particularlyinChapter6wouldnothavecomeaboutwithouttheir
cooperation.Stone’sresearchrelatedtothisbookwassupportedinpartbythe
NationalScienceFoundationunderGrantNo.IIS-0237699,andbyanAlfred
P.SloanFoundationresearchfellowship.
RoxyBot was developed by Amy Greenwald, initially in collaboration
withJustinBoyan(2000–2002),andlaterincollaborationwithherstudentsat
BrownUniversity,specificallyJesseFunaro(2003),JonathanBankard(2004),
Preface xi
Bryan Guillemette (2005), Seong Jae Lee (2006), and Victor Naroditskiy
(2003–2006).Victor deserves our sincerest gratitude for his substantial con-
tributionstothetheoreticaldevelopmentinChapter3(Theorems3.2and3.4)
andforcreatingtheseriesofexamplesofthestochasticbiddingproblempre-
sentedinChapter5.SeongJaealsodeservesspecialthanksforconductingthe
experimentsreportedinChapter5.Greenwald’sresearchrelatedto thisbook
was supported in part by the National Science Foundation under Grant No.
IIS-0133689,andbyanAlfredP.SloanFoundationresearchfellowship.
TheWalverineteam,supervisedbyMichaelWellman,hasengagedmany
graduate and undergraduate students at the University of Michigan. Partic-
ipants from 2002–present included Shih-Fen Cheng, Evan Leung, Kevin
Lochner, Daniel Reeves, Julian Schvartzman, and Yevgeniy Vorobeychik.
KevinO’Malley,ChristopherKiekintveld,DanielReeves,andWilliamWalsh
wereinstrumentalintheoperationofthefirsttwoTACevents.DanielReeves,
Kevin Lochner, and Rahul Suri provided indispensable testbed and analysis
tools.Wellman’sresearchrelatedtothisbookwassupportedinpartbytheNa-
tional Science Foundation under Grant Nos. IIS-9988715,IIS-0205435, and
IIS-0414710.
1
Introduction
Trade is a quintessential human activity. Throughouthistory, the institutions
andartifactsoftrade,suchasmarketsandcurrency,haveevolvedhandinhand
with major technological advances, such as the printing press and telecom-
munication networks. The rise of the Internet in the past decade is another
transformativeadvance.Inthenewonlinemarkets,buyersandsellershaveun-
precedentedopportunitiesfortradethroughawidearrayofnovelmechanisms.
Forexample,
• Consumers navigate the Internet through general search engines, special-
ized search services (e.g., online travel agencies), and customized shopping
facilities(e.g.,pricecomparisonsites).Thesediscoverytoolsquicklycompile
informationaboutgoodsandservicesprovidedbya multitudeofonlineven-
dors,allowingconsumerstomakemoreinformedchoices[Wanetal.,2003].
• Individualssellidiosyncraticgoods(e.g.,babyclothes,collectibles)through
onlinechannels,onaone-offbasisorasaregularbusiness.Thelargestonline
auction site, eBay [Cohen, 2002], is alone responsible for creating viable
marketsforgoodspreviouslyhardertotrade,andprovidingamediumwhere
hundredsofthousandsofsmallbusinessesearntheirlivelihood.1
• Investorstradefinancialsecuritiesthroughonlinebrokers.Brokeragefirms
and institutional investors in turn route orders through new electronic trad-
ingmechanisms,suchasECNs(electroniccommunicationsnetworks),which
automatethe matchingofcompatibleoffers.AccordingtoStoll[2006],elec-
tronictradinghasreducedtransactioncostsandimprovedtheaccuracyofprice
signals,contributingtoanoverallincreasedefficiencyofstockmarkets.
• Corporate officers procure goods and services through electronic reverse
auctions.Suchauctions(popularizedbyFreemarkets—nowAriba)enablebuy-
erstonegotiatewithmultiplesupplierssimultaneouslyona globalscale.So-
phisticated techniques optimize over numerous alternatives of suppliers in
large-scaleprocurementevents[Sandholmetal.,2006].
BeforetheadventoftheInternet,tradeswerenegotiatedbypeople:face-
to-face, over the phone, or by mail. Some conventional negotiations follow
a structured process, such as choosing listed-price items from a catalog or
1. A2005studybyACNielson,reportedbyECommerceGuide(http://ecommerce-guide.
com), found that 724,000 eBay sellers rely on eBay for their primary or secondary source of
income,andanother1.5millionindividualssupplementtheirincomethere.