Agents, games, and evolution : strategies at work and play

506 Pages·2011·3.578 MB·English

by  Steve Kimbrough

AGENTS, GAMES,
AND EVOLUTION
Strategies at Work and Play
Steven Orla Kimbrough
The Wharton School
University of Pennsylvania
Philadelphia, USA
K11564_FM.indd 1 11/15/11 2:51:58 PM
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2012 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20111026
International Standard Book Number-13: 978-1-4398-3471-8 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
List of Figures xi
List of Tables xvii
Preface xxiii
I Starters 1
1 Contexts of Strategic Interaction 3
1.1 Two Kinds of Decisions . . . . . . . . . . . . . . . . . . . . 4
1.2 Categorizing Games . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Game Theory and the Theory of Games . . . . . . . . . . . 8
1.4 Why Study Games? . . . . . . . . . . . . . . . . . . . . . . 12
1.5 Methods of Study . . . . . . . . . . . . . . . . . . . . . . . 12
1.6 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.7 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 18
2 Games in the Wild and the Problems of Play 21
2.1 War . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2 Trading and Investing . . . . . . . . . . . . . . . . . . . . . 24
2.3 Athletic Contests . . . . . . . . . . . . . . . . . . . . . . . . 25
2.4 Gambling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5 Business Strategy . . . . . . . . . . . . . . . . . . . . . . . . 29
2.6 Negotiation . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.7 Coordination, Symbiosis, Mutualism, Cooperation . . . . . 31
2.8 Conversation . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.9 Games against Yourself . . . . . . . . . . . . . . . . . . . . 33
2.10 Confidence Games . . . . . . . . . . . . . . . . . . . . . . . 33
2.11 Statesmanship . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.12 Problems of Play . . . . . . . . . . . . . . . . . . . . . . . . 35
2.13 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.14 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 41
iii
iv Agents, Games, and Evolution: Strategies at Work and Play
II Mixed Motives 43
3 Playing Prisoner’s Dilemma 45
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.2 Iterated Prisoner’s Dilemma . . . . . . . . . . . . . . . . . . 45
3.3 Pragmatic Strategic Rationality . . . . . . . . . . . . . . . . 48
3.4 Axelrod’s Tournaments . . . . . . . . . . . . . . . . . . . . 51
3.5 Further IPD Tournaments . . . . . . . . . . . . . . . . . . . 52
3.6 Other 2 2 Social Dilemma Games . . . . . . . . . . . . . . 57
×
3.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.8 Strategy Selection Criteria . . . . . . . . . . . . . . . . . . . 62
3.8.1 Do Well Overall . . . . . . . . . . . . . . . . . . . . . 62
3.8.2 Do Well against Itself . . . . . . . . . . . . . . . . . . 62
3.8.3 DoWellWhenPlayedwithOthersThatDoWellwith
Themselves . . . . . . . . . . . . . . . . . . . . . . . 62
3.8.4 Do Well against Other High-Quality Strategies . . . 63
3.8.5 Be Robust to Starting Conditions . . . . . . . . . . . 63
3.8.6 Be Robust to Conditions of Play . . . . . . . . . . . 63
3.8.7 Not Be Exploitable . . . . . . . . . . . . . . . . . . . 63
3.8.8 Avoid Self-Destructive Behavior . . . . . . . . . . . . 63
3.8.9 Be Viable in Small Numbers . . . . . . . . . . . . . . 64
3.8.10 Be an Effective Signaler . . . . . . . . . . . . . . . . 64
3.9 Comments on the Strategy Selection Criteria . . . . . . . . 64
3.10 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.11 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 68
4 Fanning out: 2 2 Games and Models 69
×
4.1 Fanning out . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.2 A Simple Gridscape Model . . . . . . . . . . . . . . . . . . 72
4.3 A Richer Gridscape Model . . . . . . . . . . . . . . . . . . . 77
4.4 An Evolutionary Model . . . . . . . . . . . . . . . . . . . . 82
4.5 Game #9 (Deadlock) . . . . . . . . . . . . . . . . . . . . . . 86
4.6 Game #68 . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.7 Game #69 . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.8 Game #7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.9 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.10 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 91
5 Stag Hunt 93
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.2 Stag Hunt and a Framing of the Program . . . . . . . . . . 94
5.3 The Gridscape: A Simple Society . . . . . . . . . . . . . . . 97
5.4 A Model for Exploring Rationality . . . . . . . . . . . . . . 103
5.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.6 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 110
Contents v
5.7 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 110
6 Pareto versus Nash 111
6.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6.2 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . 114
6.2.1 Simple Q-Learning . . . . . . . . . . . . . . . . . . . 114
6.2.2 Implementation of Q-Learning for 2 2 Games . . . . 115
×
6.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 115
6.3.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . 115
6.3.2 The Games and the Parameterization . . . . . . . . . 116
6.3.3 Settings for the Experiments . . . . . . . . . . . . . . 117
6.3.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
6.5 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 130
6.6 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 130
7 Affording Cooperation 133
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
7.1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . 134
7.1.2 Existing Proposals . . . . . . . . . . . . . . . . . . . 135
7.2 A Game That Affords Cooperation . . . . . . . . . . . . . . 137
7.3 What’s That Technology? . . . . . . . . . . . . . . . . . . . 142
7.3.1 Hi-Lo . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
7.3.2 Pro-Social Framing . . . . . . . . . . . . . . . . . . . 145
7.4 What about Evidence? . . . . . . . . . . . . . . . . . . . . . 147
7.5 Ramifications beyond Cooperation? . . . . . . . . . . . . . 151
7.6 Summary and Conclusion . . . . . . . . . . . . . . . . . . . 153
7.7 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 155
7.8 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 157
III Markets and Applications 159
8 Competitive Markets 161
8.1 Competitive Markets . . . . . . . . . . . . . . . . . . . . . . 161
8.2 Competitive Story 1: The Standard Account . . . . . . . . 163
8.3 Competitive Story 2: Random, Bargaining . . . . . . . . . . 170
8.4 Competitive Story 3: ZI-C Agents . . . . . . . . . . . . . . 173
8.5 Competitive Story 4: Trade on the Sugarscape . . . . . . . 177
8.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
8.7 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 180
8.8 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 184
vi Agents, Games, and Evolution: Strategies at Work and Play
9 Monopoly Stories 185
9.1 Monopoly Story 1: The Standard Account . . . . . . . . . . 185
9.2 Monopoly Story 2: Quantity Adjustment . . . . . . . . . . . 187
9.3 Monopoly Story 3: Randomly Varying Demand . . . . . . . 193
9.4 Monopoly Story 4: Demand as a Random Walk . . . . . . . 194
9.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9.6 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 198
9.7 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 199
10 Oligopoly: Cournot Competition 201
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
10.2 Overview of the Literature . . . . . . . . . . . . . . . . . . . 202
10.3 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . 206
10.4 Reference Model . . . . . . . . . . . . . . . . . . . . . . . . 207
10.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
10.5.1 Profit Maximization . . . . . . . . . . . . . . . . . . 212
10.5.1.1 Detailed Statistical Analysis of the Model
Results . . . . . . . . . . . . . . . . . . . . 214
10.5.2 Maximizing Total Market Profits . . . . . . . . . . . 222
10.5.2.1 Detailed Statistical Analysis of the Model
Results . . . . . . . . . . . . . . . . . . . . 225
10.5.3 Mixed Maximizing of Market and Own Returns . . . 227
10.5.4 Market Returns, Constrained by Own Returns . . . . 229
10.5.4.1 Detailed Statistical Analysis of the Model
Results . . . . . . . . . . . . . . . . . . . . 230
10.5.5 Number Effects . . . . . . . . . . . . . . . . . . . . . 231
10.5.6 Differential Costs . . . . . . . . . . . . . . . . . . . . 238
10.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
10.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
10.8 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 242
10.9 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 242
11 Oligopoly: Bertrand Competition 243
11.1 Price Competition . . . . . . . . . . . . . . . . . . . . . . . 243
11.2 Probing and Adjusting on Price in an Oligopoly . . . . . . 244
11.3 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 249
11.4 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 249
12 Supply Curve Bidding 251
12.1 Introduction and Setup . . . . . . . . . . . . . . . . . . . . 251
12.2 Computational Model . . . . . . . . . . . . . . . . . . . . . 252
12.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
12.3.1 Monopolist. . . . . . . . . . . . . . . . . . . . . . . . 257
12.3.2 One and Two Firms, Zero Cost . . . . . . . . . . . . 260
12.3.3 Six Firms . . . . . . . . . . . . . . . . . . . . . . . . 265
Contents vii
12.4 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 265
12.5 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 268
13 Two-Sided Matching 269
13.1 Introduction: Matching in Centralized Markets . . . . . . . 269
13.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
13.3 Matching with Agents . . . . . . . . . . . . . . . . . . . . . 273
13.4 Evolutionary Matching . . . . . . . . . . . . . . . . . . . . . 275
13.5 Agent Model Comparison . . . . . . . . . . . . . . . . . . . 277
13.6 Nearly Stable Matches . . . . . . . . . . . . . . . . . . . . . 279
13.7 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . 279
13.8 Summary and Discussion . . . . . . . . . . . . . . . . . . . 282
13.9 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 285
13.10 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 286
14 IDS Games 287
14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
14.2 Comparing Strategies in Iterated Play . . . . . . . . . . . . 289
14.2.1 Description of the Strategies . . . . . . . . . . . . . . 291
14.2.2 Setup for the Experiments . . . . . . . . . . . . . . . 298
14.3 Full Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . 298
14.4 Partial Feedback . . . . . . . . . . . . . . . . . . . . . . . . 302
14.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
14.6 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 308
14.7 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 309
15 Organizational Ambidexterity 311
15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
15.2 The Basic Story . . . . . . . . . . . . . . . . . . . . . . . . 312
15.3 Description of the Agent Model . . . . . . . . . . . . . . . . 318
15.4 Explorations with the Model . . . . . . . . . . . . . . . . . 321
15.4.1 Strategies . . . . . . . . . . . . . . . . . . . . . . . . 322
15.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 325
15.4.3 Tournament with the Base Case Scenario . . . . . . . 331
15.5 Robustness Tests . . . . . . . . . . . . . . . . . . . . . . . . 331
15.5.1 Second Scenario . . . . . . . . . . . . . . . . . . . . . 331
15.5.2 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . 333
15.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
15.7 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 337
15.8 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 338
16 Bargaining 341
16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
16.2 Coalition Formation . . . . . . . . . . . . . . . . . . . . . . 342
16.2.1 Description . . . . . . . . . . . . . . . . . . . . . . . 342
16.2.2 An Experimental Study . . . . . . . . . . . . . . . . 343
viii Agents, Games, and Evolution: Strategies at Work and Play
16.2.3 Quota Values . . . . . . . . . . . . . . . . . . . . . . 344
16.3 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
16.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
16.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
16.6 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 353
16.7 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 353
IV Topics in Strategic Analysis 355
17 Lying and Related Abuses 357
17.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
17.2 Concept Concepts . . . . . . . . . . . . . . . . . . . . . . . 358
17.3 Two Kinds of Ought . . . . . . . . . . . . . . . . . . . . . . 362
17.4 Lying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
17.5 Spin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
17.6 Misdirection . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
17.7 Humbug . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
17.8 Bullshit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
17.9 Mindfucking . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
17.10 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
17.11 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 378
17.12 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 378
18 Evolutionary Models 381
18.1 Biological Evolution . . . . . . . . . . . . . . . . . . . . . . 381
18.2 Representations . . . . . . . . . . . . . . . . . . . . . . . . . 383
18.3 Evolutionary Programming . . . . . . . . . . . . . . . . . . 387
18.4 Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . . 389
18.5 Coevolution . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
18.6 Examples of Applications to Games . . . . . . . . . . . . . 391
18.6.1 Discovering Strategies for IIPD . . . . . . . . . . . . 391
18.6.2 Bidding in Auctions. . . . . . . . . . . . . . . . . . . 394
18.6.3 Blondie24 and Friends . . . . . . . . . . . . . . . . . 395
18.7 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 395
18.8 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 396
19 Backward Induction 399
19.1 Three Problems . . . . . . . . . . . . . . . . . . . . . . . . . 399
19.1.1 The Surprise Examination . . . . . . . . . . . . . . . 399
19.1.2 Backward Induction in Games . . . . . . . . . . . . . 400
19.1.3 Dynamic Programming . . . . . . . . . . . . . . . . . 401
19.1.4 What to Make of All This?. . . . . . . . . . . . . . . 402
19.2 Revisiting the Surprise Examination . . . . . . . . . . . . . 403
19.3 Revisiting Definitely Iterated PD . . . . . . . . . . . . . . . 409
19.4 Revisiting Dynamic Programming . . . . . . . . . . . . . . 412
19.5 Rationality Redux . . . . . . . . . . . . . . . . . . . . . . . 414
Contents ix
19.6 For Exploration . . . . . . . . . . . . . . . . . . . . . . . . . 415
19.7 Concluding Notes . . . . . . . . . . . . . . . . . . . . . . . . 416
20 Summing up 417
20.1 Taking Stock . . . . . . . . . . . . . . . . . . . . . . . . . . 417
20.2 Problems of Play . . . . . . . . . . . . . . . . . . . . . . . . 417
20.2.1 Agent Questions. . . . . . . . . . . . . . . . . . . . . 419
20.2.1.1 Strategy Selection . . . . . . . . . . . . . . 419
20.2.1.2 Learning and Strategy Discovery . . . . . . 420
20.2.2 Institutional Questions . . . . . . . . . . . . . . . . . 421
20.2.2.1 Efficiency . . . . . . . . . . . . . . . . . . . 421
20.2.2.2 Cooperation and Collusion. . . . . . . . . . 422
20.2.2.3 Multiple Objectives. . . . . . . . . . . . . . 422
20.3 In a Nutshell . . . . . . . . . . . . . . . . . . . . . . . . . . 423
V Appendices 425
A Game Concepts 427
A.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
A.2 Concepts and Characterizations . . . . . . . . . . . . . . . . 427
A.2.1 Game. . . . . . . . . . . . . . . . . . . . . . . . . . . 427
A.2.2 Cooperative and Noncooperative Games . . . . . . . 428
A.2.3 Strategy, Policy . . . . . . . . . . . . . . . . . . . . . 428
A.2.4 Games in Strategic Form . . . . . . . . . . . . . . . . 428
A.2.5 Games in Extensive Form . . . . . . . . . . . . . . . 429
A.2.6 Games in Characteristic Function Form. . . . . . . . 430
A.2.7 Risk and Uncertainty . . . . . . . . . . . . . . . . . . 431
A.2.8 Game Payoffs . . . . . . . . . . . . . . . . . . . . . . 431
A.2.9 Pareto Optimal . . . . . . . . . . . . . . . . . . . . . 432
A.2.10 Resource or Hicks Optimality . . . . . . . . . . . . . 432
A.2.11 Equilibrium . . . . . . . . . . . . . . . . . . . . . . . 432
A.2.12 Utility . . . . . . . . . . . . . . . . . . . . . . . . . . 433
A.2.13 Rational Choice Theory . . . . . . . . . . . . . . . . 433
A.2.14 Mutual and Common Knowledge . . . . . . . . . . . 434
A.2.15 The Folk Theorem . . . . . . . . . . . . . . . . . . . 434
A.2.16 Counting Strategies in Repeated Play . . . . . . . . . 434
A.2.17 Repeated Play, Iterated Play. . . . . . . . . . . . . . 435
B Useful Mathematical Results 437
B.1 Geometric Series . . . . . . . . . . . . . . . . . . . . . . . . 437
B.2 Present Value Factor . . . . . . . . . . . . . . . . . . . . . . 438
B.3 Solving for Mixed Equilibria in 2 2 Games . . . . . . . . . 438
×
B.4 Boltzman Function . . . . . . . . . . . . . . . . . . . . . . . 439
B.5 Invasion Inequalities on the Gridscape . . . . . . . . . . . . 440
C Further Arguments on the Surprise Exam 447