Optimal Control Theory: Applications to Management Science and Economics

511 Pages·2000·5.727 MB·English

by  Suresh P. Sethi, Gerald L. Thompson (auth.)

OPTIMAL
CONTROL
THEORY
OPTIMAL
CONTROL
THEORY
Applications to Management Science
and Economics
Second Edition
Suresh P. Sethi
The University of Texas at Dallas
Gerald L. Thompson
Carnegie Mellon University
Springer
Library of Congress Cataloging-in-Publication Data
Sethi, Suresh P.
Optimal control theory : applications to management science and economics /
Suresh P. Sethi, Gerald L. Thompson.—2"^* ed.
p. cm.
Includes bibliographical references and index.
ISBN: 0-7923-8608-6 (hardcover)
ISBN: 0-387-28092-8 (paperback)
1. Management—Mathematical models. 2. Control theory. 3. Operations research.
I. Thompson, Gerald Luther, 1923- II. Title.
HD30.25 .S47 2000
658.4'033—dc21 00-040540
First paperback printing, 2006
© 2000 Springer Science+Business Media, Inc.
All rights reserved. This work may not be translated or copied in whole or in part
without the written permission of the publisher (Springer Science+Business
Media, Inc., 233 Spring Street, New York, NY 10013, USA), except for brief
excerpts in connection with reviews or scholarly analysis. Use in connection with
any form of information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter
developed is forbidden.
The use in this publication of trade names, trademarks, service marks and similar
terms, even if they are not identified as such, is not to be taken as an expression of
opinion as to whether or not they are subject to proprietary rights.
Printed in the United States of America.
9 8 7 6 5 4 3 21 SPIN 11532040
springeronline.com
This book is dedicated to the raemory of our parents:
Manak Bai and Gulab Chand Sethi
Sylva and Luther Thompson
Contents
Preface to First Edition xiii
Preface to Second Edition xv
What is Optimal Control Theory? 1
1.1 Basic Concepts and Definitions 2
1.2 Formulation of Simple Control Models 4
1.3 History of Optimal Control Theory 7
1.4 Notation and Concepts Used 10
The Maximum Principle: Continuous Time 23
2.1 Statement of the Problem 23
2.1.1 The Mathematical Model 24
2.1.2 Constraints 24
2.1.3 The Objective Function 25
2.1.4 The Optimal Control Problem 25
2.2 Dynamic Programming and the Maximum Principle . .. 27
2.2.1 The Hamilton-Jacobi-Bellman Equation 27
2.2.2 Derivation of the Adjoint Equation 31
2.2.3 The Maximum Principle 33
2.2.4 Economic Interpretations of the Maximimi
Principle 34
2.3 Elementary Examples 36
2.4 Sufficiency Conditions 44
2.5 Solving a TPBVP by Using Spreadsheet Software 48
The Maximum Principle: Mixed Inequality Constraints 57
3.1 A Maximum Principle for Problems with Mixed Inequality
Constraints 58
3.2 Sufficiency Conditions 64
viii Contents
3.3 Current-Value Formulation 65
3.4 Terminal Conditions 69
3.4.1 Examples Illustrating Terminal Conditions . . .. 74
3.5 Infinite Horizon and Stationarity 80
3.6 Model Types 83
4 The Maximum Principle: General Inequality
C!onstraints 97
4.1 Pure State Variable Inequality Constraints: Indirect Method 98
4.1.1 Jimip Conditions 103
4.2 A Maximimi Principle: Indirect Method 104
4.3 Current-Value Maximum Principle: Indirect Method . .. Ill
4.4 Sufficiency Conditions 113
5 Applications to Finance 119
5.1 The Simple Cash Balance Problem 120
5.1.1 The Model 120
5.1.2 Solution by the Maximum Principle 121
5.1.3 An Extension Disallowing Overdraft and
Short-Selhng 124
5.2 Optimal Financing Model 129
5.2.1 The Model 129
5.2.2 Application of the Maximum Principle 131
5.2.3 Synthesis of Optimal Control Paths 133
5.2.4 Solution for the Infinite Horizon Problem 144
6 Applications to Production and Inventory 153
6.1 A Production-Inventory System 154
6.1.1 The Production-Inventory Model 154
6.1.2 Solution by the Maximimi Principle 156
6.1.3 The Infinite Horizon Solution 159
6.1.4 A Complete Analysis of the Constant Positive S
Case with Infinite Horizon 160
6.1.5 Special Cases of Time Varying Demands 162
6.2 Continuous Wheat Trading Model 164
6.2.1 The Model 165
6.2.2 Solution by the Maximum Principle 166
6.2.3 Complete Solution of a Special Case 167
6.2.4 The Wheat Trading Model with No Short-SelUng . 170
6.3 Decision Horizons and Forecast Horizons 173
Contents ix
6.3.1 Horizons for the Wheat Trading Model 174
6.3.2 Horizons for the Wheat Trading Model with
Warehousing Constraint 175
7 Applications to Marketing 185
7.1 The Nerlove-Arrow Advertising Model 186
7.1.1 The Model 186
7.1.2 Solution by the Maximum Principle 188
7.1.3 A Nonlinear Extension 191
7.2 The Vidale-Wolfe Advertising Model 194
7.2.1 Optimal Control Formulation for the Vidale-Wolfe
Model 195
7.2.2 Solution Using Green's Theorem when Q is Large 196
7.2.3 Solution When Q Is Small 205
7.2.4 Solution When T Is Infinite 206
8 The Maximum Principle: Discrete Time 217
8.1 Nonlinear Programming Problems 217
8.1.1 Lagrange Multipliers 218
8.1.2 Inequahty Constraints . 220
8.1.3 Theorems from Nonlinear Programming 227
8.2 A Discrete Maximimi Principle 228
8.2.1 A Discrete-Time Optimal Control Problem . . .. 228
8.2.2 A Discrete Maximum Principle 229
8.2.3 Examples 231
8.3 A General Discrete Maximum Principle 234
9 Maintenance and Replacement 241
9.1 A Simple Maintenance and Replacement Model 242
9.1.1 The Model 242
9.1.2 Solution by the Maximum Principle 243
9.1.3 A Nimierical Example 245
9.1.4 An Extension 247
9.2 Maintenance and Replacement for a Machine Subject to
Failure 248
9.2.1 The Model 249
9.2.2 Optimal Policy 251
9.2.3 Determination of the Sale Date 253
9.3 Chain of Machines 254
9.3.1 The Model 254
X Contents
9.3.2 Solution by the Discrete Maximum Principle . . . 256
9.3.3 Special Case of Bang-Bang Control 257
9.3.4 Incorporation into the Wagner-Whitin Framework
for a Complete Solution 258
9.3.5 A Nimierical Example 259
10 Applications to Natural Resources 267
10.1 The Sole Owner Fishery Resource Model 268
10.1.1 The Dynamics of Fishery Models 268
10.1.2 The Sole Owner Model 269
10.1.3 Solution by Green's Theorem 270
10.2 An Optimal Forest Thinning Model 273
10.2.1 The Forestry Model 273
10.2.2 Determination of Optimal Thinning 274
10.2.3 A Chain of Forests Model 276
10.3 An Exhaustible Resource Model 279
10.3.1 Formulation of the Model 279
10.3.2 Solution by the Maximum Principle 282
11 Economic Applications 289
11.1 Models of Optimal Economic Growth 289
11.1.1 An Optimal Capital Accumulation Model 290
11.1.2 Solution by the Maximimi Principle 290
11.1.3 A One-Sector Model with a Growing Labor Force . 291
11.1.4 Solution by the Maximum Principle 292
11.2 A Model of Optimal Epidemic Control 295
11.2.1 Formulation of the Model 295
11.2.2 Solution by Green's Theorem 296
11.3 A Pollution Control Model 299
11.3.1 Model Formulation 299
11.3.2 Solution by the Maxim-um Principle 300
11.3.3 Phase Diagram Analysis 301
11.4 Miscellaneous Applications 303
12 Differential Games, Distributed Systems, and
Impulse Control 307
12.1 Differential Games . 308
12.1.1 Two Person Zero-Sum Differential Games 308
12.1.2 Nonzero-Simm Differential Games 310
Contents xi
12.1.3 An Application to the Common-Property Fishery
Resources 312
12.2 Distributed Parameter Systems 315
12.2.1 The Distributed Parameter Maximum Principle . . 317
12.2.2 The Cattle Ranching Problem 318
12.2.3 Interpretation of the Adjoint Fimction 322
12.3 Impulse Control 322
12.3.1 The Oil Driller's Problem 324
12.3.2 The Maximimi Principle for Impulse Optimal
Control 325
12.3.3 Solution of the Oil Driller's Problem 327
12.3.4 Machine Maintenance and Replacement 331
12.3.5 Application of the Impulse Maximum Principle . . 332
13 Stochastic Optimal Control 339
13.1 The Kahnan Filter 340
13.2 Stochastic Optimal Control 345
13.3 A Stochastic Production Planning Model 347
13.3.1 Solution for the Production Planning Problem . . 350
13.4 A Stochastic Advertising Problem 352
13.5 An Optimal Consumption-Investment Problem 355
13.6 Concluding Remarks 360
A Solutions of Linear Differential Equations 363
A.l Linear Differential Equations with Constant Coefficients . 363
A.2 Homogeneous Equations of Order One 364
A.3 Homogeneous Equations of Order Two 364
A.4 Homogeneous Equations of Order n 365
A.5 Particular Solutions of Linear D.E. with Constant
Coefficients 366
A.6 Integrating Factor 368
A.7 Reduction of Higher-Order Linear Equations to
Systems of First-Order Linear Equations 369
A.8 Solution of Linear Two-Point Boundary Value Problems . 372
A.9 Homogeneous Partial Differential Equations 372
A. 10 Inhomogeneous Partial Differential Equations 374
A. 11 Solutions of Finite Difference Equations 375
A. 11.1 Changing Polynomials in Powers of k into Facto
rial Powers of k 376