Game Theory and Decision Theory in Agent-Based Systems

402 Pages·2002·17.77 MB·English

by  Simon Parsons

GAME THEORY AND
DECISION THEORY
IN AGENT -BASED SYSTEMS
MUL TIAGENT SYSTEMS,
ARTIFICIAL SOCIETIES,
AND SIMULATED ORGANIZATIONS
International Book Series
Series Editor: Gerhard Weiss
Technische UniversiUit Mtinchen
Editorial Board:
Kathleen M. Carley, Carnegie Mellon University, PA, USA
Yves Demazeau, CNRS Laboratoire LEffiNIZ, France
Ed Durfee, University of Michigan, USA
Les Gasser, University of Illinois at Urbana-Champaign, IL, USA
Nigel Gilbert, University of Surrey, United Kingdom
Michael Huhns, University of South Carolina, SC, USA
Nick Jennings, University of Southampton, UK
Victor Lesser, University of Massachusetts, MA, USA
Katia Sycara, Carnegie Mellon University, PA, USA
Gerhard Weiss, Technical University of Munich, Germany (Series Editor)
Michael Wooldridge, University of Liverpool, United Kingdom
Books in the Series:
CONFLICTING AGENTS: Conflict Management in Multi-Agent
Systems, edited by Catherine Tessier, Laurent Chaudron and Heinz-1tirgen
MUller, ISBN: 0-7923-7210-7
SOCIAL ORDER IN MULT IAGENT SYSTEMS, edited by Rosaria
Conte and Chrysanthos Dellarocas, ISBN: 0-7923-7450-9
SOCIALLY INTELLIGENT AGENTS: Creating Relationships with
Computers and Robots, edited by Kerstin Dautenhahn, Alan H. Bond, Lola
Cafiamero and Bruce Edmonds, ISBN: 1-4020-7057-8
CONCEPTUAL MODELLING OF MULTI-AGENT SYSTEMS: The
CoMoMAS Engineering Environment, by Norbert Glaser, ISBN: 1-4020-
7061-6
GAME THEORY AND
DECISION THEORY
IN AGENT -BASED SYSTEMS
Edited by
Simon Parsons
Brooklyn College, City University of New York
Piotr Gmytrasiewicz
University of Illinois at Chicago
Michael Wooldridge
University of Liverpool
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
Library of Congress Cataloging-in-Publication Data
Game theory and decision theory in agent-based systems I edited by Simon Parsons, Piotr
Gmytrasiewicz, Michael Wooldridge
p. cm. --(Multiagent systems, artificial societies, and simulated organizations)
Includes bibliographical references and index.
ISBN 978-1-4613-5398-0 ISBN 978-1-4615-1107-6 (eBook)
DOI 10.1007/978-1-4615-1107-6
1. Game theory. 2. Statistical decision. 3. Intelligent agents (Computer software) 1.
Parsons, Simon. II. Gmytrasiewicz, Piotr. J. III. Wooldridge, Michael. IV. Series.
QA269 .G353 2002
519.3--dc21 2002067473
Copyright © 2002 by Springer Science+Business Media New York
Originally published by Kluwer Academic Publishers in 2002
Softcover reprint of the hardcover Ist edition 2002
Ali rights reserved. No part of this work may be reproduced, stored in a retrieval system,
or transmitted in any form or by any means, electronic, mechanical, photocopying,
microfilming, recording, Of otherwise, without the written permission from the Publisher,
with the exception of any material supplied specifically for the purpose of being entered and
executed on a computer system, for exclusive use by the purchaser of the work.
Permission for books published in Europe: [e-mail]
Permissions for books published in the United States of America: [e-mail]
Primed on acidjree paper.
Contents
List of Figures ix
List of Tables xiii
Preface xv
Contributing Authors XIX
Introduction xxv
Simon Parsons, Piotr Gmytrasiewicz, and Michael Wooldridge
1
Game Theory and Decision Theory 1
Simon Parsons and Michael Wooldridge
1 Introduction 1
2 Decision Theory 2
3 Game Theory 10
4 The Prisoner's Dilemma 20
5 Summary 26
2
Selecting Partners 29
Bikramjit Banerje and Sandip Sen
1 Introduction 29
2 A Sample Problem 30
3 Coalition Formation 31
4 Payoff-structures of Partnerships 32
5 Selecting the Potentially most Beneficial Partnership for Lim
ited Interactions 32
6 An Example Evaluation 34
7 Comparative Evaluation with a Decision Mechanism for Infi
nite Interactions 36
8 Evaluation Scenario 37
9 Experimental Results 37
10 Conclusions 39
3
Evolution of Agents with Moral Sentiments in an IPD Exercise 43
VI Game Theory and Decision Theory in Agent-based Systems
Ana L. C. Bazzan, Rafael H. Bordini and John A. Campbell
1 Introduction 44
2 Moral Sentiments in the Iterated Prisoner's Dilemma 45
3 The Unwitting Benefits of Altruism 48
4 An Evolutionary Setting for the Simulations 51
5 Playing with Moral Sentiments Secures the Survival of the
Group 52
6 A Case for Benevolence in Agents 56
7 Conclusions 60
4
Dynamic Desires 65
Mehdi Dastani, Zhisheng Huang and Leendert van der Torre
1 Introduction 66
2 Utilities, Desires and Goals 67
3 An Example from Negotiation 72
4 Dynamic Logic 74
5 Future Research 79
6 Concluding Remarks 79
5
Emotions and Personality 81
Piotr J. Gmytrasiewicz and Christine L. Lisetti
1 Introduction 82
2 Decision-Theoretic Preliminaries 84
3 Emotional States and Personality 87
4 Conclusions and Future Work 91
5 Acknowledgements 93
6
Decision-Theoretic Approach to Game Theory 97
Piotr J. Gmytrasiewicz and Sanguk Noh
1 Background on Decision-Theoretic Approach to Game Theory 98
2 Knowledge Base Design 101
3 RMM's Solution Method 106
4 Rational Communication 109
5 Experiments in Air Defense Domain 110
6 Conclusions 114
7
Shopbot Economics 119
Jeffrey O. Kephart and Amy R. Greenwald
1 Introduction 120
2 Model 121
3 Sellers 123
4 Buyers 128
5 Shopbot 138
6 Related Work 150
7 Conclusions and Future Work 152
Appendix: No Pure Strategy Nash Equilibrium 153
8
Finding the Best Way to Join in 159
Contents VII
Laszl6 A. K 6czy
1 Introduction 159
2 Game Theoretic Foundations 160
3 The Accession Game 163
4 Solving More General Games 168
5 Conclusions 174
9
Shopbots and Pricebots in Electronic Service Markets 177
Panos M. Markopoulos and Lyle H. Ungar
1 Introduction and Definitions 177
2 Our Model of an Electronic Service Market 180
3 Comparisons 186
4 Related Literature 189
5 Conclusions 191
Appendix: The Derivation of Equilibrium for the XX Market Config-
uration 192
10
Polynomial Time Mechanisms 197
Thomas C. 0 'Connell and Richard E. Stearns
1 Introduction 198
2 Mechanism Design 200
3 Polynomial Time Mechanisms 203
4 Dominant Strategy Implementation 206
5 Nash Implementation 208
6 Upper Bounds OIl Approximability 212
7 Conclusions 213
11
Multi-Agent Q-Iearning and Regression Trees 217
Manu Sridharan and Gerald Tesauro
1 Introduction 218
2 Model Agent Economies 219
3 Single and Multi-Agent Q-Iearning 222
4 Q-Iearning with Regression Trees 225
5 Results 227
6 Conclusions 231
12
Satisficing Equilibria 235
Wynn C. Stirling, Michael A. Goodrich and Dennis J. Packard
1 Introduction 236
2 A Comparative Paradigm 237
3 Praxeic Utility 243
4 Multi-Agent Systems 247
5 Satisficing Games 252
6 Summary and Discussion 258
13
Investigating Commitment Flexibility in Multi-agent Contracts 267
Vanessa Teague and Liz Sonenberg
1 Introduction 267
VIll Game Theory and Decision Theory in Agent-based Systems
2 The Scenario 270
3 Some Analysis of Special Cases 272
4 Empirical Work 280
5 Conclusions and Further Work 288
14
Pricing in Agent Economies using Multi-agent Q-learning 293
Gerald Tesaum and Jeffrey O. Kephart
1 Introduction 294
2 Model Agent Economies 297
3 Single-agent Q-learning 301
4 Multi-agent Q-learning 305
5 Conclusions 310
15
Using Hypergames to Increase Planned Payoff and Reduce Risk 315
Russell Vane and Paul Lehner
1 Introduction 315
2 Discussion 317
3 Results 327
4 Conclusions 333
16
Bilateral Negotiation with Incomplete and Uncertain Information 337
Julita Vassileva and Chhaya Mudgal
. 1 Introduction 338
2 The Application Area 338
3 The Negotiation Mechanism 340
4 Evaluation 343
5 Related Work on Negotiation 350
6 Conclusions 351
17
Robust Combinatorial Auction Protocol against False-name Bids 355
Makoto Yokoo, Yuko Sakurai and Shigeo Matsubara
1 Introduction 356
2 Generalized Vickrey Auction Protocol (GVA) 358
3 Robust Protocol against False-name Bids 361
4 Proof of Incentive Compatibility 366
5 Evaluation 368
6 Discussion 369
7 Conclusions 371
Index 375
List of Figures
1.1 An example Bayesian network. 4
1.2 The relationship between money and utility. 7
1.3 An example influence diagram. 9
1.4 Determining a mixed strategy for Agent i. 18
1.5 Determining a mixed strategy for Agent j. 19
2.1 Simulation of benefit (1). 39
2.2 Simulation of benefit (2). 40
3.1 Effects of altruism and of the use of the MS strategy. 50
3.2 Scores (number of points) of groups in 10,000 steps. 53
3.3 Simulations with 3 groups, 4 agents each. 54
3.4 Simulations with 3 groups, 20 agents each. 55
4.1 Utilities, desires and goals. 67
4.2 The smoking example. 68
4.3 Enriching BDI with utilitarian desires. 72
4.4 The washing machine example. 73
4.5 A model S for the washing machine. 77
5.1 An example taxonomy of emotional states. 87
5.2 Simple personality model of a tit-for-two-tats agent. 89
6.1 A simulation of the air defense domain. 102
6.2 A high-level view of a KB design for the air defense
domain scenario. 103
6.3 Influence diagram spanning the KB of Battery1. 103
6.4 Three alternative models of Battery2's state of knowl-
edge Battery 1 maintains. 105
6.5 Hierarchy of payoff matrices corresponding to nested
models of agents. 106
6.6 Average total expected damage (over 100 runs). 111
7.1 The shopbot model. 122
7.2 Probability density functions for three buyer strat-
egy vectors. 127
x Game Theory and Decision Theory in Agent-based Systems
7.3 Probability distributions over prices paid by buyers
and expected prices. 130
7.4 Benefits and costs of search for linear search costs. 132
7.5 Evolution of buyer strategy. 135
7.6 Evolution of buyer strategy and cross-section of basin
of attraction. 137
7.7 Optimal linear shopbot price schedule and equilib-
rium values of buyer strategy vector. 143
7.8 Average buyer surplus, total seller profits, and total
search cost. 144
7.9 Example of how a shopbot can adjust its prices dy-
namically. 146
7.10 Example of how a shopbot can adjust its prices dy-
namically using an adaptive procing algorithm. 148
9.1 Average prevailing price in a market with shopbots
and pricebots as the number of sellers increases. 185
9.2 Price distribution in a market with 2 servers with
shopbots and pricebots. 186
9.3 Servers and clients utility. 187
9.4 System load for different market configurations. 190
11.1 Sample profit landscape for seller 1 in Price-Quality
model, as a function of seller 1 price PI and seller 2
price P2. 221
11.2 Results of simultaneous Q-Iearning with lookup ta-
bles in the Shopbot model. 224
11.3 Results of single-agent Q-Iearning with regression
trees in the Shopbot model. 228
11.4 Results of varying Minobjs, the minimum no. of
cases per leaf node. Single-agent Q-Iearning with
regression trees in the Shopbot model. 229
11.5 Plot of the Q-function for single-agent Q-Iearning
vs. a myopic opponent in the Shopbot model. 230
12.1 Decision regions for (a) bilateral decisions and (b)
unilateral decisions. 258
13.1 Average group income vs. "brownie point" weight. 281
13.2 Average group income vs. decommitment penalty. 282
13.3 Running average group income over time for fixed
reliability weight. 283
13.4 Average group income vs. reliability weight. 284