Calculus of Variations and Optimal Control Theory: A Concise Introduction

254 Pages·2012·3.245 MB·English

by  Daniel Liberzon

cvoc-formatted August 24, 2011 7x10
Calculus of Variations and
Optimal Control Theory
cvoc-formatted August 24, 2011 7x10
cvoc-formatted August 24, 2011 7x10
Calculus of Variations and
Optimal Control Theory
A Concise Introduction
Daniel Liberzon
PRINCETON UNIVERSITY PRESS
PRINCETON AND OXFORD
Copyright © 2012 by Princeton University Press

Published by Princeton University Press, 41 William Street, Princeton, New Jersey
08540
In the United Kingdom: Princeton University Press, 6 Oxford Street, Woodstock,
Oxfordshire OX20 1TW

All Rights Reserved
ISBN: 978-0-691-15187-8
Library of Congress Control Number: 2011935625
British Library Cataloging-in-Publication Data is available
This book has been composed in LA TEX
The publisher would like to acknowledge the author of this volume for providing the
digital files from which this book was printed
Printed on acid-free paper ∞
press.princeton.edu
Printed in the United States of America
10 9 8 7 6 5 4 3 2 1
cvoc-formatted August 24, 2011 7x10
Since the building of the universe is perfect and is created by the
wisdom creator, nothing arises in the universe in which one cannot
see the sense of some maximum or minimum.
|Leonhard Euler
The words \control theory" are, of course, of recent origin, but the
subject itself is much older, since it contains the classical calculus
of variations as a special case, and the (cid:12)rst calculus of variations
problems go back to classical Greece.
|Hector J. Sussmann
cvoc-formatted August 24, 2011 7x10
cvoc-formatted August 24, 2011 7x10
Contents
Preface xiii
1 Introduction 1
1.1 Optimal control problem 1
1.2 Some background on (cid:12)nite-dimensional optimization 3
1.2.1 Unconstrained optimization . . . . . . . . . . . . . . . 4
1.2.2 Constrained optimization . . . . . . . . . . . . . . . . 11
1.3 Preview of in(cid:12)nite-dimensional optimization 17
1.3.1 Function spaces, norms, and local minima . . . . . . . 18
1.3.2 First variation and (cid:12)rst-order necessary condition . . . 19
1.3.3 Second variation and second-order conditions . . . . . 21
1.3.4 Global minima and convex problems . . . . . . . . . . 23
1.4 Notes and references for Chapter 1 24
2 Calculus of Variations 26
2.1 Examples of variational problems 26
2.1.1 Dido’s isoperimetric problem . . . . . . . . . . . . . . 26
2.1.2 Light re(cid:13)ection and refraction . . . . . . . . . . . . . . 27
2.1.3 Catenary . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.1.4 Brachistochrone . . . . . . . . . . . . . . . . . . . . . 30
2.2 Basic calculus of variations problem 32
2.2.1 Weak and strong extrema . . . . . . . . . . . . . . . . 33
2.3 First-order necessary conditions for weak extrema 34
2.3.1 Euler-Lagrange equation . . . . . . . . . . . . . . . . . 35
2.3.2 Historical remarks . . . . . . . . . . . . . . . . . . . . 39
2.3.3 Technical remarks . . . . . . . . . . . . . . . . . . . . 40
vii
cvoc-formatted August 24, 2011 7x10
viii CONTENTS
2.3.4 Two special cases . . . . . . . . . . . . . . . . . . . . . 41
2.3.5 Variable-endpoint problems . . . . . . . . . . . . . . . 42
2.4 Hamiltonian formalism and mechanics 44
2.4.1 Hamilton’s canonical equations . . . . . . . . . . . . . 45
2.4.2 Legendre transformation . . . . . . . . . . . . . . . . . 46
2.4.3 Principle of least action and conservation laws . . . . 48
2.5 Variational problems with constraints 51
2.5.1 Integral constraints . . . . . . . . . . . . . . . . . . . . 52
2.5.2 Non-integral constraints . . . . . . . . . . . . . . . . . 55
2.6 Second-order conditions 58
2.6.1 Legendre’s necessary condition for a weak minimum . 59
2.6.2 Su(cid:14)cient condition for a weak minimum . . . . . . . . 62
2.7 Notes and references for Chapter 2 68
3 From Calculus of Variations to Optimal Control 71
3.1 Necessary conditions for strong extrema 71
3.1.1 Weierstrass-Erdmann corner conditions . . . . . . . . 71
3.1.2 Weierstrass excess function . . . . . . . . . . . . . . . 76
3.2 Calculus of variations versus optimal control 81
3.3 Optimal control problem formulation and assumptions 83
3.3.1 Control system . . . . . . . . . . . . . . . . . . . . . . 83
3.3.2 Cost functional . . . . . . . . . . . . . . . . . . . . . . 86
3.3.3 Target set . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.4 Variational approach to the (cid:12)xed-time, free-endpoint problem 89
3.4.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . 89
3.4.2 First variation . . . . . . . . . . . . . . . . . . . . . . 92
3.4.3 Second variation . . . . . . . . . . . . . . . . . . . . . 95
3.4.4 Some comments . . . . . . . . . . . . . . . . . . . . . 96
3.4.5 Critique of the variational approach and preview of
the maximum principle . . . . . . . . . . . . . . . . . 98
3.5 Notes and references for Chapter 3 100
cvoc-formatted August 24, 2011 7x10
CONTENTS ix
4 The Maximum Principle 102
4.1 Statement of the maximum principle 102
4.1.1 Basic (cid:12)xed-endpoint control problem . . . . . . . . . . 102
4.1.2 Basic variable-endpoint control problem . . . . . . . . 104
4.2 Proof of the maximum principle 105
4.2.1 From Lagrange to Mayer form . . . . . . . . . . . . . 107
4.2.2 Temporal control perturbation . . . . . . . . . . . . . 109
4.2.3 Spatial control perturbation . . . . . . . . . . . . . . . 110
4.2.4 Variational equation . . . . . . . . . . . . . . . . . . . 112
4.2.5 Terminal cone. . . . . . . . . . . . . . . . . . . . . . . 115
4.2.6 Key topological lemma . . . . . . . . . . . . . . . . . . 117
4.2.7 Separating hyperplane . . . . . . . . . . . . . . . . . . 120
4.2.8 Adjoint equation . . . . . . . . . . . . . . . . . . . . . 121
4.2.9 Properties of the Hamiltonian . . . . . . . . . . . . . . 122
4.2.10 Transversality condition . . . . . . . . . . . . . . . . . 126
4.3 Discussion of the maximum principle 128
4.3.1 Changes of variables . . . . . . . . . . . . . . . . . . . 130
4.4 Time-optimal control problems 134
4.4.1 Example: double integrator . . . . . . . . . . . . . . . 135
4.4.2 Bang-bang principle for linear systems . . . . . . . . . 138
4.4.3 Nonlinear systems, singular controls, and Lie brackets 141
4.4.4 Fuller’s problem . . . . . . . . . . . . . . . . . . . . . 146
4.5 Existence of optimal controls 148
4.6 Notes and references for Chapter 4 153
5 The Hamilton-Jacobi-Bellman Equation 156
5.1 Dynamic programming and the HJB equation 156
5.1.1 Motivation: the discrete problem . . . . . . . . . . . . 156
5.1.2 Principle of optimality . . . . . . . . . . . . . . . . . . 158
5.1.3 HJB equation . . . . . . . . . . . . . . . . . . . . . . . 161
5.1.4 Su(cid:14)cient condition for optimality . . . . . . . . . . . 165
5.1.5 Historical remarks . . . . . . . . . . . . . . . . . . . . 167
5.2 HJB equation versus the maximum principle 168