Table Of ContentGeometric Optimal Control
Interdisciplinary Applied Mathematics
Editors
S.S.Antman P.Holmes
K.Sreenivasan
SeriesAdvisors
L.Glass P.S.Krishnaprasad
R.V.Kohn J.D.Murray
S.S.Sastry
Problems in engineering, computational science, and the physical and biological
sciences are using increasingly sophisticated mathematical techniques. Thus, the
bridgebetweenthemathematicalsciencesandotherdisciplinesisheavilytraveled.
The correspondingly increased dialog between the disciplines has led to the
establishmentoftheseries:InterdisciplinaryAppliedMathematics.
Thepurposeofthisseriesistomeetthecurrentandfutureneedsfortheinteraction
betweenvariousscienceandtechnologyareasontheonehandandmathematicson
the other. This is done, firstly, by encouragingthe ways that mathematicsmay be
applied in traditional areas, as well as point towards new and innovative areas of
applications;and,secondly,byencouragingotherscientificdisciplinestoengagein
adialogwithmathematiciansoutliningtheirproblemstobothaccessnewmethods
andsuggestinnovativedevelopmentswithinmathematicsitself.
Theserieswillconsistofmonographsandhigh-leveltextsfromresearchersworking
ontheinterplaybetweenmathematicsandotherfieldsofscienceandtechnology.
Forfurthervolumes:
http://www.springer.com/series/1390
Heinz Scha¨ttler • Urszula Ledzewicz
Geometric Optimal Control
Theory, Methods and Examples
123
HeinzScha¨ttler UrszulaLedzewicz
WashingtonUniversity SouthernIllinoisUniversity
St.Louis,MO, 63130 EdwardsvilleIL,62026
USA USA
SeriesEditors:
S.S.Antman P.Holmes
DepartmentofMathematics DepartmentofMechanicalandAerospace
and Engineering
InstituteforPhysicalScience PrincetonUniversity
andTechnology 215FineHall
UniversityofMaryland Princeton,NJ,08544
CollegePark,MD,20742 USA
USA pholmes@math.princeton.edu
ssa@math.umd.edu
K.Sreenivasan
DepartmentofPhysics
NewYorkUniversity
70WashingtonSquareSouth
NewYorkCity,NY,10012
USA
katepalli.sreenivasan@nyu.edu
ISSN0939-6047
ISBN978-1-4614-3833-5 ISBN978-1-4614-3834-2(eBook)
DOI10.1007/978-1-4614-3834-2
SpringerNewYorkHeidelbergDordrechtLondon
LibraryofCongressControlNumber:2012939056
MathematicsSubjectClassification(2010):49K15,49L20,34H05,93C15
©SpringerScience+BusinessMedia,LLC2012
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
To ourparents,
Elfriedeand Oswald,Teresa andJerzy,
andoursons,
FilipandMax
Preface
Optimal control theory is a discipline that has its historical origin in the calculus
ofvariations,datingbacktotheformulationofJohannBernoulli’sbrachistochrone
problemmorethan300yearsago.Itdevelopedintoafieldofitsowninthe1960sin
connectionwiththedevelopmentofspaceexploration.Infact,itwastheengineering
problemoflaunchingasatelliteintoasustainedorbit—Sputnik—thatgeneratedthe
activitiesintheSovietUnionthatledtotheearlydevelopmentsofthetheory.This
theory provides techniques to analyze far-reaching problems in various fields of
science,engineering,economics,andmorerecentlyalsobiomedicine.Generally,in
these problems the underlying task is to transfer the state of a dynamical system
fromagiveninitialpositionintoadesiredterminalcondition,e.g.,deployasatellite
into a prescribed orbit; guide a spacecraft to some remote planet, possibly even
making a soft landing (Mars rover); perform tasks using robotic manipulators;
achieve positions of wealth in economic endeavors through investment decisions;
eliminate,ifpossible,cancercellsfromourbodyorinfectedcellsfromabiological
host;andsoon,withthenumberofrealisticexampleslimitless.Alltheseproblems
havein commonthatthe dynamicsof thesystem can be influenced(“controlled”)
bymeansofsomeexternalvariable,e.g.,thefuelburnedinsidearockettogenerate
thrust,theallocationofeconomicresourcesbetweenconsumptionandinvestment,
the amounts of therapeutic agents given to treat cancer. Naturally, there always
exist practical constraints that are imposed by a particular situation—the amount
of fuel that a rocket can carry is limited, drugs cannot be given without careful
considerationoftheirsideeffects,andsoon.Still,andwithinthephysicalandother
limitsimposedbyaparticularsituation,generallythereexiststremendousfreedom
in the choice of the controls over time to achieve a desired objective. This leads
to optimization problems. Sometimes, problems are naturally associated with an
objective function to be minimized or maximized; in other instances, there is no
suchchoice,andimposingacriterionmaysimplybeameanstogenerateprocedures
(i.e.,throughnecessaryconditionsforoptimality)thatallowonetocomeupwitha
reasonablesolutiontotheunderlyingproblem.Theproblemoftransferringthestate
ofa dynamicalsystemfroma giveninitialconditionintoa setofdesiredterminal
conditions,while atthe same time minimizingsome objectiveassociated with the
vii
viii Preface
motion, and possibly a penalty on the terminal state, thus is a most natural one.
These belongto the generaltypeof problemsthatare analyzedwith the toolsand
techniquesofoptimalcontroltheory.
More precisely, the kinds of problems that will be considered in this text are
finite-dimensionaldeterministicoptimalcontrolproblems.Thesearecharacterized
bythefactthatthetimeevolutionoftheunderlyingdynamicalsystemisdescribed
bysolutionstoordinarydifferentialequations(“finite-dimensional”)asopposedto
partialdifferentialequations(“infinite-dimensional”),andstochasticeffectsrelated
to noise and other modeling perturbationsassociated with random effects are not
includedin themodeling(“deterministic”).Clearly,these are importantaspectsas
well.However,methodsthatdealwiththese structuresareofa verydifferenttype
andarewell representedin the literature.The problemswe areanalyzinghereare
among the most classical ones in mathematics and physics and have their origin
in the calculus of variations. In fact, a calculus of variation problem simply is a
specialoptimalcontrolproblemwithatrivialdynamics(givenbythederivativeof
the curve) and no constraints imposed on the control. While such constraints are
importantinpracticalproblems,thesedonot,fromourpointofview,constitutethe
maindifferencebetweenproblemsinthecalculusofvariationsandoptimalcontrol
problems.Rather,itisthepresenceofanontrivialandtypicallynonlineardynamics
that connects the controls and states. For this reason, optimal control problems
becomemuchmoredifficultthanmereextensionsofoptimizationproblemsfroma
finite-toaninfinite-dimensionalsetting.Whiletherehasbeentremendousprogress
in numericalmethodsin optimalcontroloverthe pastfifteen yearsthat has led to
the solutions of some specific and very difficult problems—the design of optimal
controlsbyNASAforthepositioningoftheinternationalspacestationusinggyros
with pseudospectral techniques or the experimental design of highly complicated
pulse sequences in nuclear magnetic resonance (NMR) spectroscopy, to mention
justtwooftheoutstandingachievements—therestilldonotexistreliablenumerical
proceduresthatcouldsimplybeappliedtoanyoptimalcontrolproblemandgivethe
solution. Specific methods, such as pseudospectral techniques, shooting methods,
andarcparameterizationtechniques,havetheirstrengthsandshortcomings,simply
becausethereexistsfartoogreatavarietyinthedynamics.Nonlinearsystemsdefy
simpleclassifications,andfromthepracticalside,problemsoftenhavetobesolved
onacase-by-casebasis.
Yet, there does exist a common framework that can be used to tackle these
problems, and it is this framework that we describe in our text. We give a
comprehensivetreatmentofthefundamentalnecessaryandsufficientconditionsfor
optimalityandillustratehowthesecanbeusedto solveoptimalcontrolproblems.
Ouremphasisisonthegeometricaspectsofthetheory,andinthiscontext,wealso
provide tools and techniques that go well beyond standard conditions (including
a comprehensivetreatmentof envelopesand singularitiesin the flow of extremals
as well as a Lie-algebra-based framework for explicit computations in canonical
coordinates)andcanbeusedtoobtaina fullunderstandingoftheglobalstructure
ofsolutionsfortheunderlyingproblem,notjustanisolatednumericalcomputation
for specific parameter values. We include a palette of examples that are worked
Preface ix
out in detail and range from classical to novel and from elementary to the highly
nontrivial. All these examples, in one way or another, illustrate the power of
geometrictechniquesandmethods.
The text is quite versatile and contains material on different levels ranging
from the introductory and elementary to the quite advanced. In this sense, some
parts of our text can be viewed as a comprehensive textbook for both advanced
undergraduate and all levels of graduate courses on optimal control in both
mathematics and engineering departments. In fact, this text grew out of lecture
notesoftheauthorsforcoursestaughtattheDepartmentsofSystemsScienceand
MathematicsandElectricalandSystemsEngineeringintheSchoolofEngineering
andAppliedScienceatWashingtonUniversityinSt.LouisandtheDepartmentof
MathematicsandStatisticsandvariousengineeringdepartmentsatSouthernIllinois
University Edwardsville. The variety of fully solved examples that illustrate the
theory,rarelypresenttothisextentinmoreadvancedtextbooksandmonographsin
thisfield,makesthistextastrongeducationalasset.Thetextmovessmoothlyfrom
themoreintroductorytopicstothosepartsthatareinamonographstylewheremore
advancedtopicsarepresented.Whilethispresentationismathematicallyrigorous,
itiscarriedoutinatutorialstylethatmakesthetextaccessibletoawideaudience
ofresearchersandstudentsfromvariousfields, notjustthe mathematicalsciences
and engineering. In a sequel, in which applications of geometric optimal control
to biomedical problems will be analyzed, the tools and techniques developed in
thistextwillbeusedtosolvevariousoptimalcontrolproblemsthatariseincancer
treatmentsthatrangefromclassicalproceduressuchaschemo-andradiotherapyto
novelapproachesthatincludeantiangiogenicagentsandimmunotherapy.
Wearegreatlyindebtedtonotonlyourteacherswhohaveinfluencedourviews
onthesubject,especiallytoourdoctoraladvisors,HectorSussmannandStanislaw
Walczak, but also to H.W. Knobloch,who introducedthe first author to the fields
ofdifferentialequationsandoptimalcontrol.Thisbookwouldnothavecomeinto
existencewithouttheguidinginfluenceandpassionforthesubjectinstilledinusby
ourmentorandgoodfriendHectorSussmann,whointroducedustothebeautyof
thegeometricapproachtooptimalcontrol.Manyprofessionalcolleagueshavebeen
instrumental in our academic careers, and we would like to take the opportunity
to thank some of them, in particular E. Roxin, A. Nowakowski, A. Krener, V.
Lakshmikantham,andT.J.Tarn.Especiallywewouldliketoacknowledgethelate
J. Zaborszky, a true engineer who appreciated mathematics, but always insisted
on a practical connection. Thanks are also due to all our students who at one
stage or another have contributed to the writing of this text. We also would like
tothankouruniversities,WashingtonUniversityinSt.LouisandSouthernIllinois
UniversityEdwardsville,andtheNationalScienceFoundation,whichhassupported
our research at various stages for by now over 20 years. Finally, we would like
to thank David Kramer, who so carefully read our text, and all the editors at
Springer,especiallyAchiDosanjhandDonnaChernyk,whohavebeenveryhelpful
throughouttheentireproductionprocess.
Edwardsville,Illinois,USA HeinzScha¨ttler
UrszulaLedzewicz
Outline of the Chapters of the Text
Below we givea briefoutlineof thechaptersthatcan serve asa roadmap forthe
scientificjourneythroughourtext.
Chapter 1 introduces the fundamental results of the calculus of variations
organized around complete solutions of two cornerstone classical examples: the
brachistochrone problem and the problem of surfaces of revolution of minimum
area.Theideasandconceptspresentedinthischapterservebothasanintroduction
toandasamotivationforthecorrespondingnotionsinoptimalcontroltheorytobe
discussedinsubsequentchapters.
The Pontryagin maximum principle, which gives the fundamental necessary
conditionsforoptimalityinoptimalcontrolproblems,willbeintroducedinChap.2
withthefocusonillustratinghowthisresultcanbeusedtosolveproblems.Tothis
end, we introduce important Lie-derivative-based techniques that form the basis
for geometric optimal control and use them to give a detailed derivation of H.
Sussmann’sresultsonthestructureoftime-optimalcontrolsfornonlinearcontrol-
affine systems in the plane. These results serve as a first illustration of the power
ofgeometricmethodsthatgowellbeyondtheconditionsofthemaximumprinciple
andleadtodeepresultsaboutthestructureofoptimalsolutions.
Whiletheemphasisofourtextisonmethodsfornonlinearsystems,inChaps.2
and3wealsogivesomeoftheclassicalresultsaboutlineartime-invariantsystems.
Theyincludeaproofoftheconvexityofthereachablesetsandtwoformulationsof
thecelebratedbang-bangtheorem.
In Chap.4 we then prove the Pontryagin maximum principle. Necessary con-
ditions for optimality follow from separation results about convex cones that
approximatethe reachableset andthe set of pointswherethe objectivedecreases,
respectively. These constructions equally apply to the classical needle variations
used by Pontryagin et al. and to high-order variations. Specific variations will be
madetoprovetheLegendre–Clebschcondition,theKelleycondition,andtheGoh
condition for optimality of singular controls. For this, an adequate computational
framework is needed that is provided by exponential representations of solutions
to differentialequations and the associated Lie-algebraic formalism related to the
Baker–Campbell–Hausdorffformula.
xi
xii OutlineoftheChaptersoftheText
Chapters 5 and 6 then deal with sufficient conditionsfor optimality, both local
andglobal.InChap.5weintroduceparameterizedfamiliesofextremals,i.e.,collec-
tionsofcontrolledtrajectoriesthatsatisfytheconditionsofthemaximumprinciple.
Throughoutthetext,weemphasizetheroletheyplayintheconstructionofsolutions
totheHamilton–Jacobi–Bellmanequation,afirst-orderpartialdifferentialequation
coupledwiththesolutionofaminimizationproblemforthecontrolsthatdescribe
theminimumvalueoftheoptimalcontrolproblemasafunctionoftheinitialdata.
Weadaptthemethodofcharacteristics,aclassicalsolutionprocedureforfirst-order
partialdifferentialequations,totheoptimalcontrolsettinganduseittoconstructthe
value function associated with a parameterized family of extremals. For example,
in this way we give an elementaryproofof the optimality of the synthesisfor the
Fullerproblemforwhichoptimalsolutionsconsistofchatteringarcswhosecontrols
switchinfinitelyoftenonfiniteintervals.Thesegeometricconstructionsprovidethe
generalizationoftheconceptofafieldofextremalsfromthecalculusofvariationsto
optimalcontroltheoryandclearlybringouttherelationshipsbetweenthenecessary
conditionsof the maximumprincipleandthe sufficientconditionsof the dynamic
programmingprinciple.
WhiletheresultsinChap.5haveamostlylocalcharacterandarealldeveloped
in the context of continuous controls, in Chap.6 we extend the constructions
to broken extremals that are finite concatenations of bang and singular controls.
Geometric transversality and matching conditions will be developed that allow
us to investigate the optimality of the flow of extremals as various patches are
combined. The main result of this chapter is a verification theorem due to H.
Sussmann that implies the optimality of a synthesis of controlled trajectories if
the associated value function satisfies some weak continuity properties and is
a continuously differentiable solution of the Hamilton–Jacobi–Bellman equation
awayfromalocallyfiniteunionofembeddedsubmanifoldsofpositivecodimension.
TheresultsthatwillbedevelopedinChaps.5and6preciselyleadtothesepiecewise
differentiabilityproperties.Itisnotrequiredthatthevaluefunctionbecontinuous.
Chapter7concludesourtextwithillustratinghowthesetechniquescanalsobe
used in low dimensions to determine small-time reachable sets exactly. This also
providesan alternativegeometric viewpointto the results on time-optimalcontrol
fornonlinearsystemsintheplanethatwerederivedinChap.2.Thematerialinthis
chapterhasneverbeenpresentedbeforeinbookform.Bitsandpiecesareavailable
intheresearchliterature,andheretheseapproachesareunified,andforthefirsttime
anaccessibleaccountofthissubjectisgiven.
Throughoutourpresentation,thetextisasmuchself-containedaspossible,and
wedoincludemoretechnicalanddifficultcomputationsiftheyarerequiredinthe
proofsortogivecompletesolutionsforsomeoftheexamples.Atvariousstages,we
revisitthesametopicfromdifferentangles,andbelowisashortroadmaptosome
ofthesetopics:
• Linear-QuadraticRegulatorandPerturbationFeedbackControl:Sects.2.1,2.4,
and5.3
• Time-OptimalControlforLinearSystems:Sects.2.5and2.6andChap.3
