Dynamic Programming for Impulse Feedback and Fast Controls: The Linear Systems Case

279 Pages·2020·4.956 MB·English

by  Alexander B. Kurzhanski, Alexander N. Daryin

Lecture Notes in Control and Information Sciences 468
Alexander B. Kurzhanski
Alexander N. Daryin
Dynamic
Programming for
Impulse Feedback
and Fast Controls
The Linear Systems Case
Lecture Notes in Control and Information
Sciences
Volume 468
Series editors
Frank Allgöwer, Institute for Systems Theory and Automatic Control,
Universität Stuttgart, Stuttgart, Germany
Manfred Morari, Institute of Automatics, ETH Zürich, Zürich, Switzerland
Thisseriesaimstoreportnewdevelopmentsinthefieldsofcontrolandinformation
sciences—quickly, informally and at a high level. The type of material considered
for publication includes:
1. Preliminary drafts of monographs and advanced textbooks
2. Lectures on a new field, or presenting a new angle on a classical field
3. Research reports
4. Reports of meetings, provided they are
(a) of exceptional interest and
(b) devoted to a specific topic. The timeliness of subject material is very
important.
Indexed by EI-Compendex, SCOPUS, Ulrich’s, MathSciNet, Current Index to
Statistics, Current Mathematical Publications, Mathematical Reviews,
IngentaConnect, MetaPress and Springerlink.
More information about this series at http://www.springer.com/series/642
Alexander B. Kurzhanski
(cid:129)
Alexander N. Daryin
Dynamic Programming
for Impulse Feedback
and Fast Controls
The Linear Systems Case
123
Alexander B. Kurzhanski Alexander N.Daryin
Faculty of Computational Mathematics Google Research
andCybernetics Zürich,Switzerland
LomonosovMoscow State University
Moscow,Russia
ISSN 0170-8643 ISSN 1610-7411 (electronic)
Lecture Notesin Control andInformation Sciences
ISBN978-1-4471-7436-3 ISBN978-1-4471-7437-0 (eBook)
https://doi.org/10.1007/978-1-4471-7437-0
LibraryofCongressControlNumber:2019931961
©Springer-VerlagLondonLtd.,partofSpringerNature2020
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
This Springer imprint is published by the registered company Springer-Verlag London Ltd. part of
SpringerNature
Theregisteredcompanyaddressis:TheCampus,4CrinanStreet,London,N19XW,UnitedKingdom
Preface
The text of this book deals with an important class of modern control problems
motivated by applied issues, namely, those that are governed by controls of
impulsive nature. Its first part describes dynamic processes under controls that
include delta functions. The emphasis is on formalizing the theory of impulse
open-loop target control, but not only. The further text deals with impulsive
feedback—thetheoryofclosed-loopcontrolledtrajectoriesthatmayhavejumpsof
coordinates caused by delta functions on their way to the end-point target.
Described are the optimized dynamics of such systems under “ordinary impulse”
controlsandalsointhepresenceofunknownbutboundeddisturbances.Furtheron
considered are controlled dynamics under state constraints and closed-loop pro-
cesses under incomplete on-line information on the system coordinates. The latter
issue involves the theory of closed-loop observability that deals with on-line state
estimation under disturbances of impulsive type. Indicated are duality properties
between state-constrained control and state estimation under impulsive inputs.
Emphasized are two types of duality—in the sense of mathematical optimization,
between primal and dual variables, and in the sense of system theory—between
solutions to problems of state control and state estimation. The second part of the
bookdealswithcontrolsdescribednotonlybyordinarydeltafunctionsbutalsoby
generalized functions of higher orders that include delta functions and their higher
derivatives. The mathematics of dynamic systems under such controls includes
specific behavior of linear controlled systems. Namely, if the input controls are
linearcombinationsofdeltafunctionsandtheirderivatives,thentheymaysolvethe
two-point boundary problems in zero time (+0). Related mathematical techniques
are thoroughly explained in the text for both open-loop and closed-loop versions.
Specialattentionisdevotedtostate-constrainedsystemsunderhigh-orderimpulsive
controlinputsandtostateestimationofsystemssubjectedtounknownbutbounded
high-order impulsive disturbances. Formulated also is a generalized duality prin-
ciple between high-order impulse control and state estimation under high-order
impulses. But impulses of any order are ideal constructions. Their application
requires additional schemes that would include some types of computable proce-
dures of regularization. This yields the last chapters of the book that describe
v
vi Preface
physically realizable solutions and the theory of realizable the so-called fast con-
trols.Suchcontrolsaredesignedbysolvingaproblemwithdoubleconstraints:the
constraint that yields impulses combined with a hard bound on the values of the
control. Fast controls are “ordinary functions” that may be selected as such that
allow to solve two-point boundary problems in arbitrary small “nano”-time. The
physical realization of impulse controls is achieved using fast functions.
A bibliography of related prior published literature is given in the sequel followed
by an Appendix on nonlinear systems with impulsive inputs. The mathematical
level of the book presumes knowledge of advanced calculus and functional anal-
ysis, linear algebra and differential equations, elements of set-valued calculus, and
basic computational methods. The authors express their gratitude to F. Allgower,
K. Astrom, T. Basar, P. Kokotovic, A. Krener, A. A. Kurzhanskiy, A. Lindquiust,
G.Leitmann,I.Mitchell,A.Rantzer, andP. Varaiya for theirvaluablediscussions
on the topics of this book thus helping to finalize its contents.
We also thank B. Ananyev, T. Filippova, M. Gusev, E. Kostousova, I. Rublev,
P. Tochilin, and I. Vostrikov for reading parts of the manuscript, their useful
comments, and contributed illustrations.
Contents
1 Introduction: Why Impulses? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 The Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Why Impulse Controls?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 The Mathematical Nature . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Physical Nature. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.1 The Physical Nature . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Notations and Preliminaries. . . . . . . . . . . . . . . . . . . . . . . . . . . 9
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Part I Ordinary Impulses
2 Open-Loop Impulse Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1 Linear Systems: Open-Loop Control Under
Ordinary Impulses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 The Impulse Control Problem . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 Reachability Under Impulse Controls: Direct Solutions . . . . . . . 18
2.4 Solution of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4.1 In the Absence of Controllability. . . . . . . . . . . . . . . . . 30
2.4.2 Controlling a Subset of Coordinates . . . . . . . . . . . . . . 30
2.4.3 The Problem with Set-Valued Boundary Conditions. . . 31
2.5 Time-Optimal Impulse Control . . . . . . . . . . . . . . . . . . . . . . . . 32
2.5.1 Scheme A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.5.2 Scheme B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.6 The Mayer–Bolza Problem with Controls as Measures . . . . . . . 36
2.7 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3 Closed-Loop Impulse Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.1 Feedback Solutions and the HJB Equation . . . . . . . . . . . . . . . . 43
3.1.1 The Problem and the Value Function . . . . . . . . . . . . . 43
vii
viii Contents
3.1.2 The Hamilton–Jacobi–Bellman Equation . . . . . . . . . . . 48
3.1.3 The Control Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.1.4 Reachability and Solvability Through Dynamic
Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.2 The Problem of Feedback Control Under Impulses. . . . . . . . . . 58
3.2.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.2.2 Constructive Motions . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.2.3 Space-Time Transformation . . . . . . . . . . . . . . . . . . . . 63
3.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.4 Solvability (Backward Reachability) and the Construction
of Invariant Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.4.1 The HJB Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.4.2 Equations of the Backward Reach Set . . . . . . . . . . . . . 73
3.5 Stabilization by Impulses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4 Impulse Control Under Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . 77
4.1 The Problem of Impulse Control Under Uncertainty . . . . . . . . . 77
4.2 The HJBI Equation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.2.1 The Principle of Optimality Under Uncertainty . . . . . . 80
4.2.2 The Hamilton–Jacobi–Bellman–Isaacs Equation . . . . . . 84
4.3 Calculating Value Functions Under Uncertainty . . . . . . . . . . . . 90
4.3.1 Min max and Max min Value Functions . . . . . . . . . . . 91
4.3.2 Value Function with Corrections. . . . . . . . . . . . . . . . . 94
4.3.3 The Closed-Loop Value Function . . . . . . . . . . . . . . . . 95
4.4 A 1D Impulse Control Problem. . . . . . . . . . . . . . . . . . . . . . . . 96
4.4.1 An Open-Loop Min max Value Function in 1D. . . . . . 96
4.4.2 A Value Function with Corrections in 1D . . . . . . . . . . 98
4.4.3 The Closed-Loop Value Function in 1D . . . . . . . . . . . 99
4.4.4 An Example in 1D. . . . . . . . . . . . . . . . . . . . . . . . . . . 100
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5 State-Constrained Impulse Control. . . . . . . . . . . . . . . . . . . . . . . . . 103
5.1 The Main Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
5.2 Open-Loop Impulse Control Under State Constraints . . . . . . . . 104
5.3 The HJB Equation Under State Constraints . . . . . . . . . . . . . . . 110
5.4 Reachability Under State Constraints . . . . . . . . . . . . . . . . . . . . 113
5.5 Backward Reachability and the Problem
of Control Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5.6 State-Constrained Control Under Uncertainty . . . . . . . . . . . . . . 121
5.6.1 The Feedback Control Problem. . . . . . . . . . . . . . . . . . 122
5.6.2 The Principle of Optimality and the HJBI Equation . . . 123
5.6.3 Open-Loop Min max and Max min Value Functions. . . 126
Contents ix
5.6.4 Backward Reachability Under Uncertainty
and State Constraints . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.6.5 Feedback Control Strategies Under Uncertainty
and State Constraints . . . . . . . . . . . . . . . . . . . . . . . . . 133
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
6 State Estimation Under Ordinary Impulsive Inputs . . . . . . . . . . . . 137
6.1 The Problem of Observation (Guaranteed Estimation). . . . . . . . 137
6.1.1 The Solution Scheme . . . . . . . . . . . . . . . . . . . . . . . . . 137
6.1.2 Duality of Observation and Control Under Ordinary
Impulsive Inputs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
6.1.3 On-Line State Estimation. The Information Set . . . . . . 142
6.2 Optimal Estimation Through Discrete Measurements . . . . . . . . 145
6.2.1 Open-Loop Assignment of Measurement Times. . . . . . 145
6.2.2 Closed-Loop Calculation of Information Sets. . . . . . . . 148
6.2.3 Closed-Loop Estimation Under Given Observation
Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.2.4 Calculating the Information Set. Ellipsoidal Method. . . 152
6.3 Closed-Loop Control Under Incomplete Measurements. . . . . . . 153
6.3.1 The System and the Information Set . . . . . . . . . . . . . . 153
6.3.2 The Problem of Output Feedback Control . . . . . . . . . . 155
6.3.3 The Dynamic Programming Approach. . . . . . . . . . . . . 158
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
Part II Impulses of Higher Order. Realizability and Fast Control
7 The Open-Loop and Closed-Loop Impulse Controls. . . . . . . . . . . . 163
7.1 Linear Systems Under Higher Order Controls:
The Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
7.2 Solutions. Controllability in Zero Time. Ultrafast Controls . . . . 165
7.2.1 The Open-Loop Solution . . . . . . . . . . . . . . . . . . . . . . 165
7.2.2 The Types of Open-Loop Control . . . . . . . . . . . . . . . . 167
7.3 Reduction to First-Order Systems Under Vector Measures. . . . . 167
7.4 HJB Theory and High-Order Impulsive Feedback. . . . . . . . . . . 169
7.5 Reduction to the “Ordinary” Impulse Control Problem . . . . . . . 171
7.6 Reachability Under High-Order Impulse Controls. . . . . . . . . . . 172
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
8 State-Constrained Control Under Higher Impulses. . . . . . . . . . . . . 179
8.1 The Problem of State-Constrained Control . . . . . . . . . . . . . . . . 179
8.1.1 Solvability of Problem 8.1 . . . . . . . . . . . . . . . . . . . . . 182
8.1.2 Optimization of the Generalized Control.
The Maximum Principle . . . . . . . . . . . . . . . . . . . . . . . 187