An Introduction to Dynamic Games

125 Pages·2000·0.68 MB·english

by  Haurie A.

An Introduction to Dynamic Games
A. Haurie J. Krawczyk
March 28, 2000
2
Contents
1 Foreword 9
1.1 WhatareDynamicGames? . . . . . . . . . . . . . . . . . . . . . . . 9
1.2 OriginsoftheseLectureNotes . . . . . . . . . . . . . . . . . . . . . 9
1.3 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
I Elements of Classical Game Theory 13
2 DecisionAnalysiswithManyAgents 15
2.1 TheBasicConceptsofGameTheory . . . . . . . . . . . . . . . . . . 15
2.2 GamesinExtensiveForm . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.1 Descriptionofmoves,informationandrandomness . . . . . . 16
2.2.2 ComparingRandomPerspectives . . . . . . . . . . . . . . . 18
2.3 Additionalconceptsaboutinformation . . . . . . . . . . . . . . . . . 20
2.3.1 Completeandperfectinformation . . . . . . . . . . . . . . . 20
2.3.2 Commitment . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.3 Bindingagreement . . . . . . . . . . . . . . . . . . . . . . . 21
2.4 GamesinNormalForm . . . . . . . . . . . . . . . . . . . . . . . . 21
3
4 CONTENTS
2.4.1 Playinggamesthroughstrategies . . . . . . . . . . . . . . . . 21
2.4.2 Fromtheextensiveformtothestrategicornormalform . . . 22
2.4.3 MixedandBehaviorStrategies . . . . . . . . . . . . . . . . . 24
3 Solutionconceptsfornoncooperativegames 27
3.1 introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 MatrixGames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.1 Saddle-Points . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2.2 Mixedstrategies . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2.3 AlgorithmsfortheComputationofSaddle-Points . . . . . . . 34
3.3 BimatrixGames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3.1 NashEquilibria . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3.2 ShortcommingsoftheNashequilibriumconcept . . . . . . . 38
3.3.3 Algorithms for the Computation of Nash Equilibria in Bima-
trixGames . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.4 Concavem-PersonGames . . . . . . . . . . . . . . . . . . . . . . . 44
3.4.1 ExistenceofCoupledEquilibria . . . . . . . . . . . . . . . . 45
3.4.2 NormalizedEquilibria . . . . . . . . . . . . . . . . . . . . . 47
3.4.3 UniquenessofEquilibrium . . . . . . . . . . . . . . . . . . . 48
3.4.4 Anumericaltechnique . . . . . . . . . . . . . . . . . . . . . 50
3.4.5 Avariationalinequalityformulation . . . . . . . . . . . . . . 50
3.5 Cournotequilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.5.1 ThestaticCournotmodel . . . . . . . . . . . . . . . . . . . . 51
CONTENTS 5
3.5.2 Formulation of a Cournot equilibrium as a nonlinear comple-
mentarityproblem . . . . . . . . . . . . . . . . . . . . . . . 52
3.5.3 ComputingthesolutionofaclassicalCournotmodel . . . . . 55
3.6 Correlatedequilibria . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.6.1 Exampleofagamewithcorrelatedequlibria . . . . . . . . . 56
3.6.2 Ageneraldefinitionofcorrelatedequilibria . . . . . . . . . . 59
3.7 Bayesianequilibriumwithincompleteinformation . . . . . . . . . . 60
3.7.1 Exampleofagamewithunknowntypeforaplayer . . . . . . 60
3.7.2 Reformulationasagamewithimperfectinformation . . . . . 61
3.7.3 AgeneraldefinitionofBayesianequilibria . . . . . . . . . . 63
3.8 AppendixonKakutaniFixed-pointtheorem . . . . . . . . . . . . . . 64
3.9 exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
II Repeated and sequential Games 67
4 Repeatedgamesandmemorystrategies 69
4.1 Repeatingagameinnormalform . . . . . . . . . . . . . . . . . . . 70
4.1.1 Repeatedbimatrixgames . . . . . . . . . . . . . . . . . . . . 70
4.1.2 Repeatedconcavegames . . . . . . . . . . . . . . . . . . . . 71
4.2 Folktheorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.2.1 Repeatedgamesplayedbyautomata . . . . . . . . . . . . . . 74
4.2.2 Minimaxpoint . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.2.3 Setofoutcomesdominatingtheminimaxpoint . . . . . . . . 76
4.3 CollusiveequilibriuminarepeatedCournotgame . . . . . . . . . . . 77
6 CONTENTS
4.3.1 Finitevsinfinitehorizon . . . . . . . . . . . . . . . . . . . . 79
4.3.2 A repeated stochastic Cournot game with discounting and im-
perfectinformation . . . . . . . . . . . . . . . . . . . . . . . 80
4.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5 Shapley’sZeroSumMarkovGame 83
5.1 Processandrewardsdynamics . . . . . . . . . . . . . . . . . . . . . 83
5.2 Informationstructureandstrategies . . . . . . . . . . . . . . . . . . 84
5.2.1 Theextensiveformofthegame . . . . . . . . . . . . . . . . 84
5.2.2 Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.3 Shapley’s-Denardooperatorformalism . . . . . . . . . . . . . . . . . 86
5.3.1 Dynamicprogrammingoperators . . . . . . . . . . . . . . . 86
5.3.2 Existenceofsequentialsaddlepoints . . . . . . . . . . . . . 87
6 Nonzero-sumMarkovandSequentialgames 89
6.1 SequentialGamewithDiscretestateandactionsets . . . . . . . . . . 89
6.1.1 Markovgamedynamics . . . . . . . . . . . . . . . . . . . . 89
6.1.2 Markovstrategies . . . . . . . . . . . . . . . . . . . . . . . . 90
6.1.3 Feedback-Nashequilibrium . . . . . . . . . . . . . . . . . . 90
6.1.4 Sobel-Whittoperatorformalism . . . . . . . . . . . . . . . . 90
6.1.5 ExistenceofNash-equilibria . . . . . . . . . . . . . . . . . . 91
6.2 SequentialGamesonBorelSpaces . . . . . . . . . . . . . . . . . . . 92
6.2.1 Descriptionofthegame . . . . . . . . . . . . . . . . . . . . 92
6.2.2 Dynamicprogrammingformalism . . . . . . . . . . . . . . . 92
CONTENTS 7
6.3 ApplicationtoaStochasticDuopoloyModel . . . . . . . . . . . . . . 93
6.3.1 Astochasticrepeatedduopoly . . . . . . . . . . . . . . . . . 93
6.3.2 Aclassoftriggerstrategiesbasedonamonitoringdevice . . . 94
6.3.3 Interpretationasacommunicationdevice . . . . . . . . . . . 97
III Differential games 99
7 Controlleddynamicalsystems 101
7.1 Acapitalaccumulationprocess . . . . . . . . . . . . . . . . . . . . . 101
7.2 Stateequationsforcontrolleddynamicalsystems . . . . . . . . . . . 102
7.2.1 Regularityconditions . . . . . . . . . . . . . . . . . . . . . . 102
7.2.2 Thecaseofstationarysystems . . . . . . . . . . . . . . . . . 102
7.2.3 Thecaseoflinearsystems . . . . . . . . . . . . . . . . . . . 103
7.3 Feedbackcontrolandthestabilityissue . . . . . . . . . . . . . . . . 103
7.3.1 Feedbackcontrolofstationarylinearsystems . . . . . . . . . 104
7.3.2 stabilizingalinearsystemwithafeedbackcontrol . . . . . . 104
7.4 Optimalcontrolproblems . . . . . . . . . . . . . . . . . . . . . . . . 104
7.5 Amodelofoptimalcapitalaccumulation . . . . . . . . . . . . . . . . 104
7.6 Theoptimalcontrolparadigm . . . . . . . . . . . . . . . . . . . . . 105
7.7 TheEulerequationsandtheMaximumprinciple . . . . . . . . . . . . 106
7.8 AneconomicinterpretationoftheMaximumPrinciple . . . . . . . . 108
7.9 Synthesisoftheoptimalcontrol . . . . . . . . . . . . . . . . . . . . 109
7.10 Dynamicprogrammingandtheoptimalfeedbackcontrol . . . . . . . 109
8 CONTENTS
7.11 Competitivedynamicalsystems . . . . . . . . . . . . . . . . . . . . 110
7.12 Competitionthroughcapitalaccumulation . . . . . . . . . . . . . . . 110
7.13 Open-loopdifferentialgames . . . . . . . . . . . . . . . . . . . . . . 110
7.13.1 Open-loopinformationstructure . . . . . . . . . . . . . . . . 110
7.13.2 Anequilibriumprinciple . . . . . . . . . . . . . . . . . . . . 110
7.14 Feedbackdifferentialgames . . . . . . . . . . . . . . . . . . . . . . 111
7.14.1 Feedbackinformationstructure . . . . . . . . . . . . . . . . 111
7.14.2 Averificationtheorem . . . . . . . . . . . . . . . . . . . . . 111
7.15 WhyarefeedbackNashequilibriaoutcomesdifferentfromOpen-loop
Nashoutcomes? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
7.16 Thesubgameperfectnessissue . . . . . . . . . . . . . . . . . . . . . 111
7.17 Memorydifferentialgames . . . . . . . . . . . . . . . . . . . . . . . 111
7.18 Characterizingallthepossibleequilibria . . . . . . . . . . . . . . . . 111
IV A Differential Game Model 113
7.19 AGameofR&DInvestment . . . . . . . . . . . . . . . . . . . . . . 115
7.19.1 DynamicsofR&D competition . . . . . . . . . . . . . . . . 115
7.19.2 ProductDifferentiation . . . . . . . . . . . . . . . . . . . . . 116
7.19.3 Economicsofinnovation . . . . . . . . . . . . . . . . . . . . 117
7.20 Informationstructure . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.20.1 Statevariables . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.20.2 Piecewiseopen-loopgame. . . . . . . . . . . . . . . . . . . . 118
7.20.3 ASequentialGameReformulation . . . . . . . . . . . . . . . 118
Chapter 1
Foreword
1.1 What are Dynamic Games?
DynamicGamesaremathematicalmodelsoftheinteractionbetweendifferentagents
who are controlling a dynamical system. Such situations occur in many instances like
armed conflicts (e.g. duel between a bomber and a jet fighter), economic competition
(e.g. investments in R&D for computer companies), parlor games (Chess, Bridge).
Theseexamplesconcerndynamicalsystemssincetheactionsoftheagents(alsocalled
players)influencetheevolutionovertimeofthestateofasystem(positionandvelocity
of aircraft, capital of know-how for Hi-Tech firms, positions of remaining pieces on a
chess board, etc). The difficulty in deciding what should be the behavior of these
agentsstemsfromthefactthateachactionanagenttakesatagiventimewillinfluence
the reaction of the opponent(s) at later time. These notes are intended to present the
basic concepts and models which have been proposed in the burgeoning literature on
gametheoryforarepresentationofthesedynamicinteractions.
1.2 Origins of these Lecture Notes
These notes are based on several courses on Dynamic Games taught by the authors,
in different universities or summer schools, to a variety of students in engineering,
economics and management science. The notes use also some documents prepared in
cooperationwithotherauthors,inparticularB.Tolwinski[Tolwinski,1988].
Thesenotesarewrittenforcontrolengineers,economistsormanagementscien-
tists interested in the analysis of multi-agent optimization problems, with a particular
9
10 CHAPTER1. FOREWORD
emphasis on the modeling of conflict situations. This means that the level of mathe-
maticsinvolvedinthepresentationwillnotgobeyondwhatisexpectedtobeknownby
a student specializing in control engineering, quantitative economics or management
science. Thesenotesareaimedatlast-yearundergraduate,firstyeargraduatestudents.
The Control engineers will certainly observe that we present dynamic games as an
extensionofoptimalcontrolwhereaseconomistswillseealsothatdynamicgamesare
only a particular aspect of the classical theory of games which is considered to have
been launched in [VonNeumann&Morgenstern1944]. Economic models of imper-
fectcompetition,presentedasvariationsonthe”classic”Cournotmodel[Cournot,1838],
will serve recurrently as an illustration of the concepts introduced and of the theories
developed. Aninterestingdomainofapplicationofdynamicgames,whichisdescribed
in these notes, relates to environmental management. The conflict situations occur-
ringinfisheriesexploitationbymultipleagentsorinpolicycoordinationforachieving
global environmental control (e.g. in the control of a possible global warming effect)
arewellcapturedintherealmofthistheory.
The objects studied in this book will be dynamic. The term dynamic comes from
Greek dynasthai (which means to be able) and refers to phenomena which undergo
a time-evolution. In these notes, most of the dynamic models will be discrete time.
This implies that, for the mathematical description of the dynamics, difference (rather
than differential) equations will be used. That, in turn, should make a great part of the
notesaccessible,andattractive,tostudentswhohavenotdoneadvancedmathematics.
However,therewillstillbesomedevelopmentsinvolvingacontinuoustimedescription
ofthedynamicsandwhichhavebeenwrittenforreaderswithastrongermathematical
background.
1.3 Motivation
There is no doubt that a course on dynamic games suitable for both control engineer-
ing students and economics or management science students requires a specialized
textbook.
Sinceweemphasizethedetaileddescriptionofthedynamicsofsomespecificsys-
tems controlled by the players we have to present rather sophisticated mathematical
notions, related to control theory. This presentation of the dynamics must be accom-
panied by an introduction to the specific mathematical concepts of game theory. The
originality of our approach is in the mixing of these two branches of applied mathe-
matics.
There are many good books on classical game theory. A nonexhaustive list in-