Dynamic Pricing and Automated Resource Allocation for Complex Information Services: Reinforcement Learning and Combinatorial Auctions

305 Pages·2007·4.45 MB·English

by  Schwind M.

Lecture Notes in Economics
and Mathematical Systems 589
FoundingEditors:
M.Beckmann
H.P.Künzi
ManagingEditors:
Prof.Dr.G.Fandel
FachbereichWirtschaftswissenschaften
FernuniversitätHagen
Feithstr.140/AVZII,58084Hagen,Germany
Prof.Dr.W.Trockel
InstitutfürMathematischeWirtschaftsforschung(IMW)
UniversitätBielefeld
Universitätsstr.25,33615Bielefeld,Germany
EditorialBoard:
A.Basile,A.Drexl,H.Dawid,K.Inderfurth,W.Kürsten,U.Schittko
Michael Schwind
Dynamic Pricing
and Automated
Resource Allocation
for Complex
Information Services
Reinforcement Learning
and Combinatorial Auctions
With 83Figures and 53Tables
123
Dr.MichaelSchwind
FacultyofEconomicsandBusinessAdministration
JohannWolfgangGoetheUniversity
CampusBockenheim
Mertonstraße17
60325FrankfurtamMain
Germany
[e-mail]
LibraryofCongressControlNumber:2007920183
ISSN0075-8442
ISBN978-3-540-68002-4 SpringerBerlinHeidelbergNewYork
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerial
isconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broad-
casting,reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationof
thispublicationorpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLaw
ofSeptember9,1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfrom
Springer.ViolationsareliabletoprosecutionundertheGermanCopyrightLaw.
SpringerispartofSpringerScience+BusinessMedia
springer.com
©Springer-VerlagBerlinHeidelberg2007
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnot
imply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Production:LE-TEXJelonek,Schmidt&V¨ocklerGbR,Leipzig
Cover-design:WMXDesignGmbH,Heidelberg
SPIN11941217 88/3100YL-543210 Printedonacid-freepaper
Preface
Writing a PhD thesis is like climbing a high mountain in unknown terrain.
There are many ups and downs on the way to the summit, and sometimes
you think you will never reach it, especially if the summit is temporarily out
of sight because you have somehow lost the orientation in the vast amount
of academic literature and related problems. The best method to reach the
summit was not to look at it permanently, but only to watch your steps on
the way to the next peak, like a conference paper or a journal article. In the
end, if youhave to put things togetherandto provethat yourinitial idea did
not lead into no man’s land, but did help you to discover some new route in
the “terra icognita” of the scientific landscape, the air will be very thin. The
lastmetersareinparticularsoexhaustingthatyouhardlycanenjoytheview
from above when you have finally reached the summit.
I would like to thank all the people that helped me to reach the summit.
First of all I have to thank my academic adviser Prof. Dr. Wolfgang Ko¨nig
at the Institute for Information Systems (IWI) at Johann Wolfgang Goethe-
University for giving me the opportunity to venture on my academic adven-
ture.Healwaysprovidedme withinstitutionalsupportthatwasnecessaryon
my long way up and encouraged me to give my best. The second important
person on my journey was Prof. Dr. Oliver Wendt who not only showed me
how to make the first expeditions into academic terrain, but also was a very
good friend in the IWI base camp, where he never stopped teaching me how
complex the scientific landscape could be. The entire expedition would have
been impossible without my fellow mountaineer at the other end of the rope,
Dr.TimStockheimwhohadthenecessarycourageandmotivationtokeepus
on track, when I struggled during the climb.
I am also very glad for the good times I shared with the other members
of the IWI base camp, Prof. Dr. Tim Weitzel, Dr. Roman Beck, Dr. Norman
Hoppen, Dr. Sven Grolik, Dr. Rainer Fladung, Tobias Keim, Jochen Franke,
and Oleg Gujo. They were not only good fellows in having fun, but always
supported me with a helping mind when necessary, like proof reading the
VI Preface
final version of this work. I hope that we all can continue our friendship and
research partnership in the future.
I am also grateful to the German National Science Foundation and the
German Federal Ministry of Education and Research for providing the re-
sources of my scientific projects ‘Pricing of Distributed Information Services’
and ‘Dynamic Bundle Pricing’.
I would also like to thank my mother Eva-Maria who has always backed
my effortsandgivenme the motivationtotry the PhDadventureby showing
me the fascinating face of nature and science when I was a little boy.
Finally,Iproudlycansaythatthemostimportantpersoninthelastthree
years of my expedition and in my life, has been Katrinwho has given me the
confidence and strength to reach the summit. Hopefully this rope will never
be torn apart.
Michael Schwind Frankfurt, January 2007
Foreword Prof. Dr. Ko¨nig
Informationservicesareoftenusedtodistributevaluableinformationproducts
to customers via modern information and communication systems, e.g. the
departuretime ofatraingivenonthe Internet.Theprovisionofsuchservices
requires specific resources that are normally only available with a limited
capacity, like e.g. the server capacity for communication processes that is
necessary to bring the time-table information to the web users.
In many cases these capacities are offered by different institutions at zero
marginalcost,e.g.theGermanrailwaycorporationprovidestheservercapac-
ityforfreeandthecustomersnaturallyhaveInternetaccessviatheirpersonal
computers, such that a single request does not generate additional costs.
The developed economies that are undergoing a massive change from in-
dustrial production to the service society are especially confronted with an
increasing ‘digitalization’ of production. This means that, for example, an
increasedcoordinationeffort- realizedby more intensive data exchangemea-
suredin‘megabitsper second’- leadsto areductionoflogisticsexpensemea-
suredin‘tonskilometers’.Thisisoneofthereasonsthatafurthersubstantial
increase in the production and distribution of digital products and services
has to be expected in the future and with this the questions arises:
If, as shown in the previous example, the consumption of simple informa-
tionproductsis notnecessarilylinkedtoanincreaseinexpensesforthe users
fortheservices,whyshouldserviceprovidersthenfurtheroffermorecomplex
informationservicesforfreeatasatisfyingqualitylevel?Willserviceproviders
for these reasons offer more differentiated services above a basic level at all?
Indeed, an increasing number of ‘complex information products’ is neces-
sary to further develop a modern economy. These complex products consist
of elements of other information products and so forth. Let me explain this
interdependence andnestedstructure byusing the exampleofavideo confer-
ence:The conferenceusersneedthe availabilityofn videodevicesatdifferent
places jointly with communication capacity between these devices which re-
quiremorethanonephoneline.Alltheseresourcesarerequiredtobeavailable
at the same time. If they are not, the conference can not take place and the
VIII Foreword Prof. Dr. K¨onig
value of the disjoint resources that are reservedin advance will be zero. This
type of interdependence is called complementarity. The traditional way to
handle such a resource allocation problem from an economic viewpoint is to
use m disjoint auctions (m ≤ n) for each resource type. However, this does
not resolve the complementarity problem: You might receive e.g. three of six
requiredresourcesforanacceptableprice,the otherthreeresources,however,
might be very expensive or even unavailable, and therefore devalue the user
utility of the entire resource package in the video conference case. On way to
handle this problem is to use combinatorial auctions, that are able to deal
withthisproblem.Andthecostofperformingthecombinatorialauctionmay
be includedin the calculationofthe ‘businesscase’.Another wayis to havea
mediator learn the yield optimal allocation of the task bundles to the system
resources. Resource allocation tasks that include the complementarity prob-
lem are fairly widespread - the practical relevance of a solution for this kind
of problems is very high.
MichaelSchwinddecides to solvethis demanding problemwith the objec-
tive of achieving the efficient allocation and dynamic pricing of the resources
for complex information services in the domain of distributed systems. For
this purpose he uses the method of agent-based computational economics, a
young discipline that promises to provideinsights into economic processesby
formulating the behaviorof interacting subjects as softwareagents in a simu-
lation environment and by analyzing the evolving complex system properties
in various application scenarios that should be close to real world situations.
Altogether, the work of Michael Schwind not only presents, a broad spec-
trum of simulation methods, algorithms, and experiments especially in the
domain of agent-based computational economics, but also provides unique
and groundbreaking insights by applying and advancing recent methods of
artificial intelligence, like neural networks or reinforcement learning, and na-
ture oriented optimization methods, like genetic algorithms and simulated
annealing,to the problem of combinatorialresourceallocationunder comple-
mentarities.
Wolfgang Ko¨nig Frankfurt, January 2007
Foreword Prof. Dr. Wendt
Since the beginning of economic research,the efficient allocation of resources
to production and consumption processes has always been the central focus
of the discipline. During the last century modern micro-economic theory has
madetremendousprogressalongthispathbyprovingthatPareto-efficiental-
locationscanbeachievedbyamarketmechanism,adaptingaglobalpricevec-
tor for allgoodsuntil anequilibriumis reachedandno consumeror producer
wants to buy or sell any quantity of any goods. Furthermore, the proof that
this equilibrium can even be reached by a fully decentralized “peer to peer”
mechanism not requiring any central control (like an auctioneer or commod-
ity exchange) seemed to provide the theoretical underpinning for the market
economies’ superiority compared to the eastern economies’ obsolete concept
of a benevolent global planner.
However,mostofthetheoryreliesonstrongassumptions:Thepreferences
ofproducersandconsumersmustberepresentablebyconcaveutilityfunctions
(prohibiting any complementarities) and all goods must be divisible. We do
not even have to resort to the “digital economy” of information production
to realizehowfarfetched these assumptionsare:Adishcanonly be prepared
when all ingredients are available, a nice vacation requires a flight, a hotel
anda rentalcar for the same periodof time and no canceledflightcould ever
be substituted by an increased number of hotel rooms or rental cars. The
smaller the number of suppliers in each of these markets, the higher the risk
of possibly getting stuck with one of the three resources not being available
when bidding in three separate auctions.
InhisthesisMichaelSchwindaddressestheseissuesofcomplementarityby
pricinganallocationofresourcebundles fromthe perspective of(automated)
informationproduction.Here,mostprocessestypically requireacombination
ofnetworkbandwidth, CPUusage,mainmemory andmassstorage.Interest-
inglyenough,eventhoughpeertopeerfilesharingprotocolsorinitiativeslike
UCBerkeley’sSETI@homeprojectmotivatethousandsofusersto(temporar-
ily) share these resources for free, none of the major operating systems offer
a “pricing protocol” to implement a market mechanism. Most of today’s op-
X Foreword Prof. Dr. Wendt
erating systems still much rather rely on simple pre-defined process priorities
and “fairness criteria” to allocate their resources.
Michael Schwind shows that with the advent of GRID systems these de-
ficiencies have become obvious and resorting to economic mechanisms of re-
sourceallocationhasrecentlybeenpromotedbymanyresearchersinthecom-
putersciencedomain.Histhesisgivesacomprehensiveandthoroughreviewof
the different fields ofeconomictheoryandcomputerscience whichcontribute
to the theoretical foundations of what would be needed to construct an in-
centive compatible GRID operating system, allowing for a Pareto-efficient
allocation of all its resources by a dynamic pricing mechanism.
Heclassifiestheexistingresearchintotwomajorstrands,namelythe(gen-
eral)game theoretic approachesandthe market-basedallocationapproaches.
The former frame the allocation problem as a problem of mechanism design
for a cooperative n-agent game and frequently but not necessarily use prices
to solve this problem. The latter explicitly picture the information system
network as a global market on which buyers compete for scarce processing
and storage capacity provided by the owners of these hardware resources.To
this strand of research the use of prices and money to solve the allocation
problem appears to be natural and is not questioned.
However, the focus of the decision processes analyzed varies: While com-
binatorial yield management as well as combinatorial auctions both allow
reservationoffutureresourcebundleusage(incontrastto“spotmarkets”only
permittingad-hocrequestsforimmediatetaskprocessing),yieldmanagement
assumesthateachofthecompetingrequestshastobedecideduponatrequest
time(renderingthepricingdecisionbeastochasticcombinatorialoptimization
problem),whereascombinatorialauctionspostponethewinnerdetermination
andpricingdecisiontoaspecificpointintimepre-selectedbytheauctioneerin
most cases (rendering the dynamic pricing decision an iterated deterministic
problem).Forbothcases-whichleadtodifferentimplicationsforthebidders’
strategic behavior - Michael Schwind not only reviews and contrasts recent
literature but also develops innovative dynamic pricing approaches to solve
them,basedonreinforcementlearning,localsearchheuristicsandrelaxations
of the set packing problem posed by the combinatorial auctions.
As an empirical study which is part of this work reveals, what prevents
most companies engaged in e-commerce from using dynamic pricing mecha-
nisms is notthe fearoflosing customergoodwillbyimposing a pricingmodel
onthem,whichmightbeperceivedasunfairandobscurebutmuchratherthe
companies’fearthatthecostsofimplementingsuchamechanismoutweighthe
potentialprofitsitcangenerate.Althoughthisthesisdoesinfactconfirmthat
dynamic pricing and allocationof resourcebundles being are a complex issue
italsoillustratesthatthepotentialgainsaretremendousandtheroadtowards
anautomatedandcost-efficientdecisionmakingisabouttobepaved,render-
ing the above-mentionedperceptionof the cost-benefitratio a misperception.
Oliver Wendt Kaiserslautern,January 2007