Dynamical systems and optimal control

338 Pages·2018·1.83 MB·english

by  Salsa S.

MyBook
D
y
n
The volume provides and elementary introduc- am
tion of the mathematical modelling in those ar- ic
a
eas on Dynamical Systems and Control Theory l
S
that arise frequently in Economics. y
s
t
The book is divided in two parts. e
m
In the first part the fundamental aspects of mathemati- Dynamical Systems
s

cal modelling are developed, dealing with both contin- a
n
uous time systems (differential equations) and discrete d
and Optimal

O
time systems (difference equations). Particular atten-
p
tion is devoted to equilibria, their classification in the t
im Control
linear case, and their stability. An effort has been made
a
l
to convey intuition and emphasize concrete aspects,
C
without omitting the necessary theoretical tools. o
n
In the second part the basic concepts and techniques tr
o
of Dynamic Optimization are introduced, covering the l

first elements of Calculus of Variations, the variational Sa Sandro Salsa
ls
formulation of the most common problems in deter- a
ministic Optimal Control, with a brief introduction to · S Annamaria Squellati
q
Dynamic Programming, both in continuous and dis- u
e
crete versions. lla
t
This book is designed as an advanced undergraduate i
Aor aG firlsot-ybeaar gl rEadcuaoten cooumrsei cfo rH stuisdteontsr yfrom vari- AA GFrlioebnadll yE Icnotnroodmuicct iHoinstory
ous disciplines and in particular from Economics and
Social Sciences.
Sandro Salsa is Professor of Mathematical Analysis at the Depart-
ment of Mathematics, Politecnico di Milano.
Annamaria Squellati was Lecturer of Mathematics at Bocconi
University, Milan and Contract Professor at Politecnico di Milano.
ISBN 978-88-85486-52-2
MyBook http://mybook.egeaonline.it
MyBook is the gateway to access accompanying resources (both text and
multimedia), the BookRoom, the EasyBook app and your purchased books.
Dynamical Systems and Optimal Control.indd 1 17 mm 18/05/18 14:40

Dynamical Systems
and Optimal
Control
Sandro Salsa
Annamaria Squellati
A Friendly Introduction
Frontespizio Dynamical Systems and Optimal Control.indd 1 14/05/18 14:16
Copyright © 2018, Bocconi University Press
EGEA S.p.A.

EGEA S.p.A.
Via Salasco, 5 - 20136 Milan, Italy
Phone + 39 02 5836.5751 - Fax [phone]
[e-mail] - www.egeaeditore.it

All rights reserved, including but not limited to translation, total or partial adaptation,
reproduction, and communication to the public by any means on any media
(including microfilms, films, photocopies, electronic or digital media), as well as
electronic information storage and retrieval systems. For more information or
permission to use material from this text, see the website www.egeaeditore.it

Given the characteristics of Internet, the publisher is not responsible for any changes
of address and contents of the websites mentioned.

First edition: May 2018

ISBN Domestic Edition 978-88-99902-11-7
ISBN International Edition 978-88-85486-52-2
ISBN Pdf Edition 978-88-85486-53-9

Print: Digital Print Service, Segrate (Milan)
Contents
1 Introduction to Modelling 1
1.1 Some Classical Examples . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.1 Malthus model . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 Logistic models . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.3 Phillips model . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.1.4 Accelerator model . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.1.5 Evolution of supply . . . . . . . . . . . . . . . . . . . . . . . . 12
1.1.6 Leslie model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.1.7 Lotka-Volterra predator-prey model. . . . . . . . . . . . . . . . 14
1.1.8 Time-delay logistic equation . . . . . . . . . . . . . . . . . . . 16
1.2 Continuous Time and Discrete Time Models . . . . . . . . . . . . . . . 17
1.2.1 Differential and difference equations . . . . . . . . . . . . . . . 18
1.2.2 Systems of differential and difference equations . . . . . . . . . 21
2 First Order Differential Equations 25
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.2 Some Solvable Equations . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.1 Differential equations with separable variables . . . . . . . . . . 26
2.2.2 Solow-Swan model . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2.3 Logistic model . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2.4 Linear equations . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2.5 Market dynamics . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.2.6 Other types of equations . . . . . . . . . . . . . . . . . . . . . 38
2.3 The Cauchy Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.3.1 Existence and uniqueness . . . . . . . . . . . . . . . . . . . . . 44
2.3.2 Maximal interval of existence . . . . . . . . . . . . . . . . . . . 48
2.4 Autonomous Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.4.1 Steady states, stability, phase space . . . . . . . . . . . . . . . 52
2.4.2 Stability by linearization . . . . . . . . . . . . . . . . . . . . . . 56
2.5 A Neoclassical Growth Model . . . . . . . . . . . . . . . . . . . . . . . 57
2.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3 First Order Difference Equations 65
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.2 Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.2.1 Linear homogeneous equations . . . . . . . . . . . . . . . . . . 65
3.2.2 Nonhomogeneous linear equations . . . . . . . . . . . . . . . . 66
3.2.3 Simple and compounded capitalization . . . . . . . . . . . . . . 69
V
vi
3.2.4 Cobweb model . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.3 Nonlinear Autonomous Equations . . . . . . . . . . . . . . . . . . . . . 72
3.3.1 Orbits, stairstep diagram, steady states (fixed or equilibrium
points) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.3.2 Steady states and stability . . . . . . . . . . . . . . . . . . . . 74
3.3.3 Stability of periodic orbits. . . . . . . . . . . . . . . . . . . . . 78
3.3.4 Chaotic behavior . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.3.5 Discrete logistic equation . . . . . . . . . . . . . . . . . . . . . 83
3.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4 Linear Differential Equations with Constant Coefficients 91
4.1 Second Order Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.1.1 Homogeneous equations. . . . . . . . . . . . . . . . . . . . . . 91
4.1.2 Nonhomogeneous equations. . . . . . . . . . . . . . . . . . . . 95
4.2 Higher Order Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.2.1 Homogeneous equations. . . . . . . . . . . . . . . . . . . . . . 97
4.2.2 Nonhomogeneous equations. . . . . . . . . . . . . . . . . . . . 98
4.2.3 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.2.4 Phillips model . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5 Linear Difference Equations with Constant Coefficients 107
5.1 Second Order Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.1.1 Homogeneous equations. . . . . . . . . . . . . . . . . . . . . . 107
5.1.2 Fibonacci sequence . . . . . . . . . . . . . . . . . . . . . . . . 110
5.1.3 Nonhomogeneous equations. . . . . . . . . . . . . . . . . . . . 112
5.2 Higher Order Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.2.1 Homogeneous equations. . . . . . . . . . . . . . . . . . . . . . 113
5.2.2 Nonhomogeneous equations. . . . . . . . . . . . . . . . . . . . 114
5.2.3 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.2.4 Accelerator model . . . . . . . . . . . . . . . . . . . . . . . . . 117
5.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
6 Systems of Differential Equations 119
6.1 The Cauchy Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.2 Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
6.2.1 Global existence and uniqueness . . . . . . . . . . . . . . . . . 121
6.2.2 Homogeneous systems . . . . . . . . . . . . . . . . . . . . . . 121
6.2.3 Nonhomogeneous systems. . . . . . . . . . . . . . . . . . . . . 123
6.2.4 Equations of order  . . . . . . . . . . . . . . . . . . . . . . . 125
6.3 Bidimensional Systems with Constant Coefficients . . . . . . . . . . . . 126
6.3.1 General integral . . . . . . . . . . . . . . . . . . . . . . . . . . 126
6.3.2 Stability of the zero solution . . . . . . . . . . . . . . . . . . . 130
6.4 Systems with Constant Coefficients (higher dimension) . . . . . . . . . 130
6.4.1 Exponential matrix . . . . . . . . . . . . . . . . . . . . . . . . 130
vii
6.4.2 Cauchy problem and general integral . . . . . . . . . . . . . . . 136
6.4.3 Nonhomogeneous systems. . . . . . . . . . . . . . . . . . . . . 138
6.4.4 Stability of the zero solution . . . . . . . . . . . . . . . . . . . 139
6.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7 Bidimensional Autonomous Systems 143
7.1 Phase Plane Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
7.1.1 Orbits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
7.1.2 Steady states, cycles and their stability . . . . . . . . . . . . . . 147
7.1.3 Phase portrait . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
7.2 Linear Systems. Classification of steady states . . . . . . . . . . . . . . 154
7.3 Non-linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
7.3.1 The linearization method . . . . . . . . . . . . . . . . . . . . . 164
7.3.2 Outline of the Liapunov method . . . . . . . . . . . . . . . . . 167
7.4 Some Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
7.4.1 Lotka-Volterra model . . . . . . . . . . . . . . . . . . . . . . . 170
7.4.2 A competitive equilibrium model . . . . . . . . . . . . . . . . . 176
7.5 Higher Dimensional Systems . . . . . . . . . . . . . . . . . . . . . . . 180
7.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
8 Systems of Difference Equations 183
8.1 Linear Systems with Constant Coefficients . . . . . . . . . . . . . . . . 183
8.1.1 Homogeneous systems . . . . . . . . . . . . . . . . . . . . . . 184
8.1.2 Bidimensional homogeneous systems . . . . . . . . . . . . . . . 185
8.1.3 Nonhomogeneous systems. . . . . . . . . . . . . . . . . . . . . 189
8.2 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
8.2.1 Election polls . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
8.2.2 A model of students partition . . . . . . . . . . . . . . . . . . . 193
8.2.3 Leslie model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
8.3 Autonomous systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
8.3.1 Discrete Lotka-Volterra model . . . . . . . . . . . . . . . . . . 196
8.3.2 Logistic equation with delay . . . . . . . . . . . . . . . . . . . 197
8.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
9 The Calculus of Variations 203
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
9.2 The Simplest Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
9.2.1 Fixed boundaries. Euler equation . . . . . . . . . . . . . . . . . 206
9.2.2 Special cases of the Euler-Lagrange equation . . . . . . . . . . 212
9.2.3 Free end values. Transversality conditions . . . . . . . . . . . . 216
9.3 A Sufficient Condition of Optimality . . . . . . . . . . . . . . . . . . . 219
9.4 Infinite Horizon. Unbounded Interval. An Optimal Growth Problem . . . 220
9.5 The General Variation of a Functional . . . . . . . . . . . . . . . . . . 224
9.6 Isoperimetric Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 229
9.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
viii
10 Optimal Control Problems. Variational Methods 239
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
10.1.1 Structureofacontrolproblem.One-dimensionalstateandcontrol239
10.1.2 Main questions and techniques . . . . . . . . . . . . . . . . . . 242
10.2 Continuous Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . 243
10.2.1 Free final state. Necessary conditions . . . . . . . . . . . . . . . 243
10.2.2 Sufficient conditions . . . . . . . . . . . . . . . . . . . . . . . . 247
10.2.3 Interpretation of the multiplier . . . . . . . . . . . . . . . . . . 249
10.2.4 Maximum principle. Bounded controls . . . . . . . . . . . . . . 250
10.2.5 Discounting. Current values . . . . . . . . . . . . . . . . . . . . 251
10.2.6 Applications. Infinite horizon. Comparative analysis . . . . . . . 252
10.2.7 Terminal payoff and various endpoints conditions . . . . . . . . 257
10.2.8 Discontinuous and bang-bang control. Singular solutions . . . . 264
10.2.9 An advertising model control . . . . . . . . . . . . . . . . . . . 266
10.3 Discrete Time Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 270
10.3.1 The simplest problem . . . . . . . . . . . . . . . . . . . . . . . 270
10.3.2 A discrete model for optimal growth . . . . . . . . . . . . . . . 273
10.3.3 State and control constraints . . . . . . . . . . . . . . . . . . . 274
10.3.4 Interpretation of the multiplier . . . . . . . . . . . . . . . . . . 280
10.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
11 Dynamic Programming 287
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
11.2 Continuous Time System. The Bellman Equation . . . . . . . . . . . . 290
11.3 Infinite Horizon. Discounting . . . . . . . . . . . . . . . . . . . . . . . 295
11.4 Discrete Time Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 297
11.4.1 The value function. The Bellman equation . . . . . . . . . . . . 297
11.4.2 Optimal resource allocation . . . . . . . . . . . . . . . . . . . . 301
11.4.3 Infinite horizon. Autonomous problems . . . . . . . . . . . . . . 303
11.4.4 Renewable resource management . . . . . . . . . . . . . . . . . 304
A Appendix 307
A.1 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . . 307
A.2 Functional Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
A.3 Static Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
A.3.1 Free optimization . . . . . . . . . . . . . . . . . . . . . . . . . 312
A.3.2 Constrained optimization. Equality constraints . . . . . . . . . . 313
A.3.3 Constrained optimization. Inequality constraints . . . . . . . . . 315
References 321
Subject index 323
Preface
Thisbookisdesignedasanadvancedundergraduateorafirst-yeargraduatecourse
for students from various disciplines and in particular from Economics and Social
Sciences. It has evolved while teaching courses of Advanced Mathematics at the
Bocconi University. The heterogeneous background of the students from different
areas, has suggested an almost self-contained presentation. The result is a book
divided in two parts.
In the first part, constituted by the chapters from 1 to 8, the fundamental as-
pectsofmathematicalmodellingaredeveloped,dealingwithbothcontinuoustime
systems (differential equations) and discrete time systems (difference equations).
Particular attention is devoted to equilibria, their classification in the linear case,
andtheirstability.Anefforthasbeenmadetoconveyintuitionandemphasizecon-
nections and concrete aspects, without giving up the necessary theoretical tools.
In the second part, from chapters 9 to 11, the basic concepts and techniques
of Dynamic Optimization are introduced, covering the first elements of Calculus
of Variations, the variational formulation of the most common problems in deter-
ministic Optimal Control, both in continuous and discrete versions. Chapter 11
contains a brief introduction to Dynamic Programming.
To avoid heavy technicalities and to facilitate the understanding of the funda-
mentalideas,bothstateandcontrolvariablesareone-dimensional.Webelievethat,
onceunderstoodtheone-dimensionalcase,thereaderwillbeabletogeneralizethe
results to any number of dimensions without much effort, using the specialized
books on the subject, listed in the references.
For the first part the preliminary requirements are limited to a knowledge of
the Riemann integral and the multidimensional differential calculus, besides basic
notions of linear algebra, briefly recalled in the Appendix.
Thesecondpartrequirestheknowledgeofstatic(freeandconstrained)optimiza-
tionforfunctionsofseveralvariables.TheFermattheorem,themethodofLagrange
and Karush-Kuhn-Tucker multipliers are briefly recalled in the Appendix.
At the end of every chapter, a list of exercises is proposed, whose solutions can
be found in the website: www.egeaonline.it.
Milan, January 2018 The Authors