Fixed-Endpoint Optimal Control of Bilinear Ensemble Systems

0.32 MB·

by  Shuo Wang

FIXED-ENDPOINTOPTIMALCONTROLOFBILINEARENSEMBLESYSTEMS∗
SHUOWANG† AND JR-SHIN LI‡
Abstract. Optimal control ofbilinear systems has been awell-studied subject in the area ofmathematical
control. However,techniquesforsolvingemergingoptimalcontrolproblemsinvolvinganensembleofstructurally
identical bilinear systems are underdeveloped. In this work, we develop an iterative method to effectively and
6 systematicallysolvethesechallengingoptimalensemblecontrolproblems,inwhichthebilinearensemblesystem
1 is represented as a time-varying linear ensemble system at each iteration and the optimal ensemble control law
0 isthenobtainedbythesingularvalueexpansionoftheinput-to-state operatorthatdescribes thedynamicsofthe
2 linear ensemble system. Weexamine the convergence ofthe developed iterative procedure and pose optimality
conditionsfortheconvergentsolution.Wealsoprovideexamplesofpracticalcontroldesignsinmagneticresonance
n todemonstratetheapplicabilityandrobustnessofthedevelopediterativemethod.
a
J
Keywords. Ensemblecontrol, iterative methods,sweepmethod,fixed-endpoint problems, bilinearsystems,

3 optimalityconditions,magneticresonance.
1
AMSsubjectclassifications.

]
C
1. Introduction. Newly emerging fields in science and engineering, such as systems
O
neuroscience,synchronizationengineering,andquantumscienceandtechnology,giveriseto
.
h newclassesofoptimalcontrolproblemsthatinvolveunderactuatedmanipulationofindivid-
t ualand collective behaviorof dynamicunits in a largeensemble. Representativeexamples
a
m include neural stimulation for alleviating the symptoms of neurological disorders such as
Parkinson’sdisease,whereapopulationofneuronsinthebrainisaffectedbyasmallnumber
[
ofelectrodes[1]; pulse designsforexcitingandtransportingquantumsystemsbetweende-

1 siredstates, whereanensembleofquantumsystemsisdrivenbyasingleormultiplepulses
v
inapulsesequence[2,3];andtheengineeringofdynamicalstructuresforcomplexoscillator
9
networks, where sequential patterns of a network of nonlinear rhythmic elements are cre-
2
3 atedandalteredbyamildglobalwaveform[4]. Solvingthesenontraditionalandlarge-scale
3 underactuatedcontrolproblemsrequiresthedevelopmentofsystematicandcomputationally
0 tractableandeffectivemethods.
.
1 Amongtheseemergingcontrolproblems,inthispaper,wewillstudyfixed-endpointop-
0 timalcontrolproblemsinvolvingbilinearensemblesystems,whicharisefromthedomainof
6 quantumcontrol[5]andappearinavarietyofotherdifferentfields,suchascancerchemother-
1
apy[6] androbotics[7]. Thecontrolofbilinearsystemshasbeena well-studiedsubjectin
:
v theareaofmathematicalcontrol. FromPontryagin’smaximumprincipletospectralcolloca-
i
X tionmethods,awidevarietyoftheoreticalandcomputationalmethodshavebeendeveloped
to solve optimal control problems of bilinear systems [8, 9]. In particular, the numerical
r
a methodsare in principlecategorizedinto direct, e.g., pseudospectralmethods[10, 11], and
indirectapproaches,e.g.,indirecttranscriptionmethod[12]andshootingmethods[13]. Im-
plementingtheseexistingnumericalmethodstosolveoptimalcontrolproblemsinvolvingan
ensemble,i.e.,alargenumber(finitelyorinfinitelymany)oraparameterizedfamily,ofbilin-
earsystemsmayencounterlowefficiency,slowconvergence,andinstabilityissues,because
most of these methodsrely on suitable discretization of the continuous-timedynamicsinto
alarge-scalenonlinearprogram(LSNLP).Inaddition,theglobalconstraintforsuchanop-
∗ThisworkwassupportedinpartbytheNationalScienceFoundationundertheawardsCMMI-1301148,CMMI-
1462796,andECCS-1509342.
†DepartmentofElectricalandSystemsEngineering,WashingtonUniversity,St. Louis,Missouri,63130,USA
([email protected]).
‡DepartmentofElectricalandSystemsEngineering,WashingtonUniversity,St. Louis,Missouri,63130,USA
([email protected]).Questions,comments,orcorrectionstothisdocumentmaybedirectedtothisemailaddress.
1
2 S.WANGANDJ.-S.LI
timalensemblecontrolproblem,inwhicheachindividualsystemreceivesthe samecontrol
input,makesthediscretizedLSNLPveryrestrictiveandintractabletosolveoreventofinda
feasiblesolution[14].
On the other hand, optimalcontrol problemsinvolving a linear system, or a linear en-
semblesystem,areoftencomputationallytractableandanalyticallysolvableformanyspecial
cases,suchasthelinearquadraticregulator(LQR)[15]andtheminimum-energycontrolof
harmonicoscillatorensembles[16].Thissuggestsabypasstosolveoptimalcontrolproblems
ofbilinearensemblesystemsthroughsolvingthatoflinearensemblesystemsandmotivates
thedevelopmentoftheiterativemethodinthiswork.Thecentralideaistorepresentthebilin-
earensemblesystemasalinearensemblesystemateachiteration,andthenfeasiblycalculate
theoptimalcontrolandtrajectoryforeachiterationuntilaconvergentsolutionisfound.Itera-
tivemethodshavebeenintroducedandadoptedtodealwithdiversecontroldesignproblems,
including the free-endpointquadratic optimal control of bilinear systems [17] and optimal
state trackingfornonlinearsystems[18], while the fixed-endpointproblemsalongwith the
emergingproblemsthatinvolvecontrollingabilinearensemblesystemremainunexplored.
Inthispaper,wecombinetheideaoftheaforementionediterativemethodwithourpre-
viousworkonoptimalcontroloflinearensemblesystemstoconstructaniterativealgorithm
forsolvingoptimalcontrolproblemsinvolvingatime-invariantbilinearensemblesystemof
theform,
d X(t,b )=A(b )X(t,b )+B(b )u(t)+ (cid:229)m u(t)B(b ) X(t,b ),
i i
dt
i=1
(cid:16) (cid:17)
whereX =(x ,...,x )T ∈M⊂Rn denotesthe state, b ∈K ⊂Rd with K compactand d a
1 n
positiveinteger,u(t)=(u (t),...,u (t))T ∈Rmisthecontrol,andthematricesA(b )∈Rn×n,
1 m
B(b )∈Rn×m,andB(b )∈Rn×n,i=1,...,m,forb ∈K.
i
This paper is structuredas follows. In the nextsection, we presentthe developediter-
ative method for fixed-endpointoptimal control of a time-invariant bilinear system, where
weintroduceasweepmethodthataccountsfortheterminalconditionbasedonthenotionof
flowmappingfromtheoptimalcontroltheory. InSection3,weexaminetheconvergenceof
theiterativemethodusingthefixed-pointtheorem. InSection4, weproposetheconditions
forglobaloptimalityoftheconvergentsolution.Then,inSection5,weextendthedeveloped
iterativemethodtosolveoptimalcontrolproblemsinvolvingbilinearensemblesystemsand
showtheconvergenceofthemethod. Finally,examplesandsimulationsofpracticalcontrol
designproblemsareillustrated in Section6 to demonstratethe applicabilityandrobustness
ofthedevelopediterativeprocedure.
2. Iterativemethodforoptimalcontrolofbilinearsystems. Westartwithconsider-
ingafixed-endpoint,finite-time,quadraticoptimalcontrolprobleminvolvingatime-invariant
bilinearsystemoftheform
min J= 1 tf xT(t)Qx(t)+uT(t)Ru(t) dt,
2 0
Z
h m i
(cid:229)
(P1) s.t. x˙=Ax+Bu+ uB x,
i i
i=1
h i
x(0)=x , x(t )=x ,
0 f f
wherex(t)∈Rnisthestateandu(t)∈Rmisthecontrol;A∈Rn×n,B ∈Rn×n,andB∈Rn×m
i
areconstantmatrices;R∈Rm×m≻0ispositivedefiniteandQ∈Rn×n(cid:23)0ispositivesemi-
definite; and x ,x ∈Rn are the initial andthe desired terminalstate, respectively. We first
0 f
FIXED-ENDPOINTCONTROLFORBILINEARENSEMBLES 3
representthetime-invariantbilinearsystemin(P1)asatime-varyinglinearsystem,
n
(cid:229)
(2.1) x˙(t)=Ax+Bu+ x (t)N u,
j j
"j=1 #
inwhichwewritethebilinearterm (cid:229) m uB x= (cid:229) n x N uwithx the jthelementofx,
i=1 i i j=1 j j j
N ∈Rn×m for j=i,...,n,andu=(u ,...,u )T ∈Rm. Then,wesolvethisoptimalcontrol
j (cid:0) 1 m(cid:1) (cid:0) (cid:1)
problembyPontryagin’smaximumprinciple.TheHamiltonianofthisproblemis
(2.2) H(x,u,l )= 1(xTQx+uTRu)+l T Ax+ B+((cid:229) n x N ) u ,
j j
2
j=1
n h i o
wherel (t)∈Rnistheco-statevector.Theoptimalcontrolisthenobtainedbythenecessary
condition, ¶ H =0,givenby
¶ u
(2.3) u∗=−R−1 B+(cid:229) n x N Tl ,
j j
j=1
(cid:16) (cid:17)
andtheoptimaltrajectoryofthestatexandtheco-statel satisfy,fort∈[0,t ],
f
n n
(2.4) x˙ = Ax − B+(cid:229) x N R−1 B+(cid:229) x N Tl ,
i i j j j j i
j=1 j=1
(cid:2) (cid:3) h(cid:0) (cid:1) (cid:0) (cid:1) i
n n
(2.5) l˙ =− Qx − ATl +l T NR−1 B+(cid:229) x N T+ B+(cid:229) x N R−1NT l
i i i i j j j j i
j=1 j=1
(cid:2) (cid:3) (cid:2) (cid:3) n (cid:0) (cid:1) (cid:0) (cid:1) o
withtheboundaryconditionsx(0)=x andx(t )=x ,wherex,l and[·],i=1,...,n,are
0 f f i i i
theith componentoftheassociatedvectors.Bythefollowingchangeofvariables,
n n
(2.6) A˜ =A − (N R−1 B+(cid:229) x N T+ B+(cid:229) x N )R−1NT l ,
ij ij j j j j j j
i
j=1 j=1
h (cid:0) (cid:1) (cid:0) (cid:1) i
n n
(2.7) B˜R−1B˜T =BR−1BT− (cid:229) x N R−1 (cid:229) x N T,
j j j j
j=1 j=1
(cid:0) (cid:1) (cid:0) (cid:1)
(2.8) Q˜ =Q,
wecanrewrite(2.4)and(2.5)intotheform
(2.9) x˙=A˜x−B˜R−1B˜Tl , x(0)=x , x(t )=x ,
0 f f
(2.10) l˙ =−Q˜x−A˜Tl ,
whichcoincideswiththecanonicalformofthestateandco-stateequationscharacterizingthe
optimal trajectories for the analogousoptimalcontrol problem involvingthe time-invariant
linearsystemx˙=A˜x+B˜u[19]. Inthisway,theoptimalstateandco-statetrajectoriesforthe
optimalcontrolproblem(P1)involvingatime-invariantbilinearsystemarenowexpressedin
termsoftheequationsrelatedtoatime-varyinglinearsystemasin(2.9)and(2.10).
Usingthis“linear-systemrepresentation”togetherwiththeSweepmethod[19,20],we
will solve the optimal control problem (P1) in an iterative manner. Specifically, we will
considerateachiterationthefixed-endpointlinearquadraticoptimalcontrolproblem,
min J= 1 tf (x(k+1))T(t)Qx(k+1)(t)+(u(k+1))T(t)Ru(k+1)(t) dt,
2 0
Z
h i
(P2) s.t. x˙(k+1)(t)=A˜(k)x(k+1)+B˜(k)u(k+1)
x(k+1)(0)=x , x(k+1)(t )=x ,
0 f f
4 S.WANGANDJ.-S.LI
by treating the previoustrajectory x(k) as a known quantity, where k∈N denotesthe itera-
tion. Inthefollowingsections,wewillintroducetheSweepmethodandpresenttheiterative
procedure.
2.1. Sweep method for fixed-endpoint problems. Observe that in (2.9) and (2.10)
there are two boundaryconditionsfor the state x while none for the co-state l . It requires
implementingspecializedcomputationalmethods,suchasshootingmethods,tosolvesucha
two-pointboundaryvalue problem, which in generalinvolveintensive numericaloptimiza-
tions. Here,weadopttheideaoftheSweepmethodbyletting
(2.11) l (t)=K(t)x(t)+S(t)n ,
withl (t )=n ,whereK(t), S(t)∈Rn×n andn isthemultiplier,aconstantassociatedwith
f
theterminalconstrainty ,whichinthiscaseisy (x(t ))=x(t )=x . Fromthetransversality
f f f
conditionin Pontryagin’smaximumprinciple, we know that K(t )=0 because there is no
f
¶y
terminalcostandS(t )= =I.Moreover,ifKischosentosatisfytheRiccatiequation
f ¶ x x(tf)
(2.12) K˙(t)=−(cid:12)(cid:12) Q−A˜TK(t)−K(t)A˜+K(t)B˜R−1B˜TK(t),
withtheterminalconditionK(t )=0,thenSsatisfiesthematrixdifferentialequation
f
(2.13) S˙(t)=−(A˜T−K(t)B˜R−1B˜T)S(t),
withtheterminalconditionS(t )=I,bytakingthetimederivativeof(2.11)andusing(2.9),
f
(2.10)and(2.12). Inaddition,inordertofulfilltheterminalconditiony (x(t ))=x attime
f f
t ,themultipliern associatedwithy mustsatisfy
f
(2.14) x =ST(t)x(t)+P(t)n
f
forallt∈[0,t ],whereP(t)∈Rn×nobeysthematrixdifferentialequation
f
(2.15) P˙(t)−ST(t)B˜R−1B˜TS(t)=0,
withtheterminalconditionP(t )=0. Itfollowsfrom(2.14)usingt=0that
f
(2.16) n = P(0) −1 x −ST(0)x ,
f 0
providedP(t)isinvertiblefort∈[0(cid:2),tf]. M(cid:3)or(cid:2)edetailsabou(cid:3)ttheSweepmethodbasedonthe
notionofflowmappingareprovidedinAppendix8.1.1.
2.2. Iterationprocedure. Theoptimalsolutionoftheproblem(P1)ischaracterizedby
thehomogeneoustime-varyinglinearsystemdescribedin(2.9)and(2.10),andwewillsolve
forxandl viaaniterativeprocedure,whichisbasedonanalyticalexpressionsandrequiresno
numericaloptimizations.Toproceedthis,wewrite(2.9)and(2.10)astheiterationequations,
(2.17) x˙(k+1)=A˜(k)x(k+1)−B˜(k)R−1(B˜(k))Tl (k+1),
(2.18) l˙(k+1)=−Q˜(k)x(k+1)−(A˜(k))Tl (k+1),
with identicalboundaryconditionsx(k+1)(0)=x andx(k+1)(t )=x forallk=0,1,2,...,
0 f f
whereA˜(k),B˜(k)R−1(B˜(k))T,andQ˜(k) aredefinedaccordingto(2.6),(2.7),and(2.8),by
n n
(2.19) A˜(k)=A − (N R−1 B+(cid:229) x(k)N T+ B+(cid:229) x(k)N )R−1NT l (k) ,
ij ij j j j j j j
i
j=1 j=1
h (cid:0) (cid:1) (cid:0) (cid:1) i
n n
(2.20) B˜(k)R−1(B˜(k))T =BR−1BT− (cid:229) x(k)N R−1 (cid:229) x(k)N T,
j j j j
j=1 j=1
(cid:0) (cid:1) (cid:0) (cid:1)
(2.21) Q˜(k)=Q.
FIXED-ENDPOINTCONTROLFORBILINEARENSEMBLES 5
ApplyingtheSweepmethodintroducedinSection2.1,weletl (k+1)(t)=K(k+1)(t)x(k+1)(t)+
S(k+1)(t)n (k+1)fort∈[0,t ],whereK(k) satisfiestheRiccatiequation
f
(2.22) K˙(k+1)=−Q(k)−K(k+1)A˜(k)−(A˜(k))TK(k+1)+K(k+1)B˜(k)R−1(B˜(k))TK(k+1),
withtheboundaryconditionK(k+1)(t )=0,andS(k)follows
f
(2.23) S˙(k+1)=− (A˜(k))T−K(k+1)B˜(k)R−1(B˜(k))T S(k+1), S(k+1)(t )=I.
f
h i
Moreover,themultipliern (k) satisfies
(2.24) n (k+1)= P(k+1)(0) −1 x −(S(k+1))T(0)x ,
f 0
where P(·)(t)∈Rn×n is invertibl(cid:2)e (see Lem(cid:3)ma(cid:2) 3.1 in Section 3) a(cid:3)nd satisfies the dynamic
equation
(2.25) P˙(k+1)=(S(k+1))TB˜(k)R−1(B˜(k))TS(k+1)
with the terminalconditionP(k+1)(t )=0. Then, the optimalcontrol(2.3) for the original
f
Problem(P1)canbeexpressedas
(2.26) u∗(t)=−R−1 B+(cid:229) n x∗(t)N T[K∗(t)x∗(t)+S∗(t)n ∗],
j j
j=1
h i
ifthisiterativeprocedureisconvergent,wherex(k)→x∗,K(k)→K∗,andS(k)n (k)→S∗n ∗.
REMARK 1. The iterative method can be initialized by convenientlyusing the optimal
controlofthesysteminvolvingonlythelinearpartofthebilinearsystemin(P1),i.e.,theLQR
control. Thatis,thesolution(x(0)(t),l (0)(t))tothehomogeneoussystem
x˙(0)=Ax(0)−BR−1BTl (0), x(0)(0)=x , x(0)(t )=x ,
0 f f
l˙(0)=−ATl (0).
However, the linearsystem x˙=Ax+Bu may be uncontrollableso that the desired transfer
between x and x is impossible and the LQR solution does not exist. In such a case, any
0 f
statetrajectorywiththeendpointsx andx canbeafeasibleinitialtrajectoryx(0)(t)ofthe
0 f
iterativeprocedure.
2.3. Aspecialcase: minimum-energycontrolofbilinearsystems. Beforeanalyzing
the convergence of the iterative method, we illustrate the procedure using the example of
minimum-energycontrolof bilinear systems, which is a special case of Problem (P1) with
Q=0. Considerthefollowingfixed-endpointoptimalcontrolproblem,
min J= 1 tf uT(t)Ru(t)dt,
2 0
Z
n
(cid:229)
(P3) s.t. x˙(t)=Ax+Bu+ x (t)N u,
j j
"j=1 #
x(0)=x , x(t )=x .
0 f f
TheHamiltonianofthisproblemisH(x,u,l )= 1uTRu+l T[Ax+Bu+((cid:229) n x N )u],where
2 j=1 j j
l (t)∈Rnistheco-statevector.Theoptimalcontrolisoftheformasin(2.3)andtheoptimal
6 S.WANGANDJ.-S.LI
stateandco-statetrajectoriessatisfy(2.9)and(2.10),respectively,withQ=0.Therespective
iterationequationsfollow(2.17)and(2.18)withQ(k)=0forallk=0,1,2,....
FollowingtheiterativemethodpresentedinSection2.2,werepresentthecostatel (k+1)(t)=
K(k+1)(t)x(k+1)(t)+S(k+1)(t)n (k+1),t ∈[0,t ],andthematrixK(k+1)(t)∈Rn×n satisfiesthe
f
Riccatiequation,
(2.27) K˙(k+1)=−K(k+1)A˜(k)−(A˜(k))TK(k+1)+K(k+1)B˜(k)R−1(B˜(k))TK(k+1),
with the terminal condition K(k+1)(t )= 0, which has the trivial solution, K(k+1)(t)≡ 0,
f
∀k=0,1,2,...,andfort∈[0,t ]. Thisgives
f
(2.28) l (k+1)(t)=S(k+1)(t)n (k+1),
andS(k+1)satisfies
(2.29) S˙(k+1)=−(A˜(k))TS(k+1), S(k+1)(t )=I.
f
In addition, the multiplier associated with the terminal constraintis expressed as in (2.24).
Combining(2.28)with(2.3)givestheminimum-energycontrolatthe(k+1)th iteration,
(2.30) (u∗)(k+1)(t)=−R−1 B+(cid:229) n x(k+1)N TS(k+1)n (k+1).
j j
j=1
h i
NotethattheauxiliaryvariableP(k)(t)∈Rn×nateachiterationsatisfies(2.25),andthus
P(k+1)(0)=−F (t ,0)W(k+1)F T (t ,0),
A˜(k) f A˜(k) f
where F T (t ,t)=F (t,t ) is the transition matrix for the homogeneous equation
A˜(k) f −(A˜(k))T f
(2.29)and
W(k+1)= tf F (0,s )B˜(k)R−1(B˜(k))TF T (0,s )ds
A˜(k) A˜(k)
0
Z
isthecontrollabilityGramianforthetime-varyinglinearsystemasinProblem(P3),or,equiv-
alently,asin(2.9)and(2.10)withQ=0. Moreover,theclosed-loopexpressionin(2.30)is
consistentwiththeopen-loopexpressionoftheminimum-energycontrolintermsofthecon-
trollabilityGramian,thatis,
n
(2.31) (u∗)(k+1)=R−1 B+(cid:229) x(k+1)N TF T (0,t) W(k+1) −1x (k),
j j A˜(k)
j=1
(cid:2) (cid:3) (cid:0) (cid:1)
wherex (k)=F (0,t )x −x .
A˜(k) f f 0
3. ConvergenceoftheIterativeMethod. Followingtheiterativealgorithmdescribed
inSection2.2,weexpecttofindtheoptimalcontrolforProblem(P1),providedtheiterations
areconvergent.Inthissection,weshowthattheconvergenceofthisalgorithmispertinentto
thecontrollabilityofthelinearsystemconsideredateachiterationanddependsonthechoice
of the weightmatrix R. In Section 5, we will extendthis iterative methodto solve optimal
controlproblemsinvolvingbilinearensemblesystems.
Tofacilitatetheproof,weintroducethefollowingmathematicaltools. Consideringthe
. . .
Banach spaces, X =C([0,t ];Rn), Y =C([0,t ];Rn×n), and Z =C([0,t ];Rn) with the
f f f
FIXED-ENDPOINTCONTROLFORBILINEARENSEMBLES 7
norms
(3.1) kxka = sup kx(t)kexp(−a t) , for x∈X,
t∈[0,tf]
(cid:2) (cid:3)
(3.2) kyka = sup ky(t)kexp(−a (tf−t)) , for y∈Y,
t∈[0,tf]
(cid:2) (cid:3)
(3.3) kzka = sup kz(t)kexp(−a (tf−t)) , for z∈Z,
t∈[0,tf]
(cid:2) (cid:3)
in which kvk=(cid:229) n |v| for v∈Rn and kDk=max (cid:229) n |D | for D∈Rn×n, and the
i=1 i 1≤j≤n i=1 ij
parameter a serves as an additional degree of freedom to control the rate of convergence
[17], we define the operators T :X ×Y ×Z →X, T :X ×Y ×Z →Y, and T :
1 2 3
X ×Y ×Z →Z thatcharacterizethedynamicsofx∈X,K∈Y,andSn ∈Z asdescribed
inSection2.2,givenby
d
T [x,K,Sn ](t)=A˜(x(t),K(t),S(t)n )T [x,K,Sn ](t)−B˜(x(t))R−1B˜T(x(t))T [x,K,Sn ](t)
1 1 3
dt
(3.4) −B˜(x(t))R−1B˜T(x(t))T [x,K,Sn ](t)T [x,K,Sn ](t),
2 1
T [x,K,Sn ](0)=x
1 0
d
T [x,K,Sn ](t)=−Q+T [x,K,Sn ](t)B˜(x(t))R−1B˜T(x(t))T [x,K,Sn ](t)
2 2 2
dt
(3.5) −T [x,K,Sn ](t)A˜(x(t),K(t),S(t)n )−A˜T(x(t),K(t),S(t)n )T [x,K,Sn ](t)
2 2
T [x,K,Sn ](t )=0,
2 f
d
T [x,K,Sn ](t)=− A˜T(x(t),K(t),S(t)n )−T [x,K,Sn ](t)B˜(x(t))R−1B˜T(x(t)) ·
3 2
dt
(3.6) T [hx,K,Sn ](t), i
3
T [x,K,Sn ](t )=n (T [x,K,Sn ],T [x,K,Sn ],T [x,K,Sn ])
3 f 1 2 3
wheren (T [x,K,Sn ],T [x,K,Sn ],T [x,K,Sn ])isthemultipliersatisfying(2.24). Withthese
1 2 3
definitionsand the followinglemma, the convergenceof the iterativemethodcan be devel-
opedusingthefixed-pointtheorem.
LEMMA 3.1. The matrix P(k+1)(t) as in (2.25) is nonsingular over t ∈[0,tf] at each
iterationk ifandonlyifthetime-varyinglinearsysteminProblem(P2)iscontrollableover
[0,t ][19].
f
Proof: SeeAppendix8.1.3. (cid:3)
THEOREM3.2. Considertheiterativemethodwiththeiterationsevolvingaccordingto
(3.7) x(k+1)(t)=T [x(k),K(k),S(k)n (k)](t),
1
(3.8) K(k+1)(t)=T [x(k),K(k),S(k)n (k)](t),
2
(3.9) S(k+1)(t)n (k+1)=T [x(k),K(k),S(k)n (k)](t),
3
wheretheoperatorsT ,T ,andT aredefinedin(3.4),(3.5),and(3.6),respectively.Ifateach
1 2 3
iteration k the linear system as in (P2) is controllable, then T , T , and T are contractive.
1 2 3
Furthermore,startingwithatriple offeasibletrajectories(x(0),K(0),S(0)n (0)), theiteration
procedure is convergent, and the sequences x(k), K(k) and S(k)n (k) converge to the unique
fixedpoints,x∗,K∗,and(Sn )∗,respectively.
Proof:Becausethelinearsystemin(P2)iscontrollableateachiterationk,byLemma3.1
thematrixP(k+1) definedin(2.25)isinvertibleandhencethemultipliern (k+1) expressedin
8 S.WANGANDJ.-S.LI
(2.24)iswell-defined.Then,wehave,attimet ,S(k+1)(t )n (k+1)=T [x(k),K(k),S(k)n (k)](t )=
f f 3 f
n (k+1),sinceS(k+1)(t )=I.
f
From(2.19)and(2.20),foreachfixedt∈[0,t ],weobtainthebounds
f
kA˜(k+1)−A˜(k)k≤ (cid:229) n kGk2 1/2kl (k+1)−l (k)k
i
i=1
h i
+k (cid:229) n kH k2 1/2 kl (k+1)kkx(k+1)−x(k)k+kx(k)kkl (k+1)−l (k)k ,
ij
i,j=1
h i n o
kB˜(k+1)R−1(B˜(k+1))T−B˜(k)R−1(B˜(k))Tk≤k (cid:229) n kH k2 1/2k(x(k+1))2−(x(k))2k,
ij
i,j=1
h i
whereG =NR−1BT+BR−1NT andH =NR−1NT+N R−1NT,andfrom(2.21),wehave
i i i ij i j j i
Q˜(k+1) =Q˜(k) for all k=0,1,2,.... Substituting (2.18) into the aboveinequalities, we can
writetheseboundsintermsofkx(k+1)−x(k)k,kK(k+1)−K(k)kandkS(k+1)n (k+1)−S(k)n (k)k,
givenby
kA˜(k+1)−A˜(k)k≤ (cid:229) n kGk2 1/2+kx(k)k (cid:229) n kH k2 1/2 ·
i ij
i=1 i,j=1
nh i h i o
(3.10) kK(k+1)kkx(k+1)−x(k)k+kK(k+1)−K(k)kkx(k)k+kS(k+1)n (k+1)−S(k)n (k)k
(cid:8)+ (cid:229) n kH k2 1/2kK(k+1)kkx(k+1)k+kS(k+1)n (k+1)kkx(k+1)−x(k)k, (cid:9)
ij
i,j=1
h i
kB˜(k+1)R−1(B˜(k+1))T−B˜(k)R−1(B˜(k))Tk≤ (cid:229) n kH k2 1/2·
ij
i,j=1
h i
(3.11) kx(k+1)k+kx(k)k kx(k+1)−x(k)k.
n o
Inaddition,thesolutionto(2.23)isgivenby
S(k+1)(t)=F (t,t )S(k+1)(t )
−[(A˜(k))T−K(k+1)B˜(k)R−1(B˜(k))T] f f
(3.12) =F T (t ,t),
[A˜(k)−B˜(k)R−1(B˜(k))TK(k+1)] f
whereF denotesthetransitionmatrixassociatedwiththehomogeneoussystem(2.23)and
(.)
S(k+1)(t )=I. Then,wehave
f
T
kS(k+1)(t)−S(k)(t)k≤ k (S(k+1))T(t) −1kkS(k+1))T(s )k kA˜(k)(s )−A˜(k−1)(s )k
t
Z
(3.13) +kB˜(k)R(cid:2)−1(B˜(k))T(s (cid:3))−B˜(k−1)R−1(B˜(k−h1))T(s )kkK(k+1)(s )k
+kB˜(k−1)R−1(B˜(k−1))T(s )kkK(k+1)(s )−K(k)(s )k kS(k)(s )kds .
i
FromtheRiccatiequationforK(k) describedin(2.22),wecanwritethedifferentialequation
forthedifferenceK(k+1)−K(k)as
d
(K(k+1)−K(k))=−(K(k+1)−K(k)) A˜(k)−B˜(k)R−1(B˜(k))TK(k+1)
dt
h T i
(3.14) − A˜(k)−B˜(k)R−1(B˜(k))TK(k+1) (K(k+1)−K(k))
−Kh(k)(A˜(k)−A˜(k−1))−(A˜(k)−Ai˜(k−1))TK(k+1)
+K(k)(B˜(k)R−1(B˜(k))T−B˜(k−1)R−1(B˜(k−1))T)K(k+1),
FIXED-ENDPOINTCONTROLFORBILINEARENSEMBLES 9
with the terminal condition K(k+1)(t )−K(k)(t )=0. Applying the variation of constants
f f
formula,backwardintimefromt=t ,to(3.14)andemploying(3.12)yield
f
K(k+1)(t)−K(k)(t)=(S(k))T(t) tf (S(k))T(s ) −1 K(k)(s ) A˜(k)(s )−A˜(k−1)(s )
t
Z
+ A˜(k)(s )−A˜(k−1)n(s ) ThK(k+1)(s )−iK(hk)(s )· (cid:0) (cid:1)
−1
B˜(cid:0)(k)R−1(B˜(k))T−B˜(k−1(cid:1))R−1(B˜(k−1))T K(k+1)(s ) S(k+1)(s ) ds S(k+1)(t),
whichresult(cid:0)sin (cid:1) ih i o
kK(k+1)(t)−K(k)(t)k≤ tf b kA˜(k)(s )−A˜(k−1)(s )k
1
t
Z
h
(3.15) +b kB˜(k)R−1(B˜(k))T(s )−B˜(k−1)R−1(B˜(k−1))T(s )k ds ,
2
i
whereb andb arebothfinitetime-varyingcoefficients(seeAppendix8.2).
1 2
Similarly, from (2.17) and (2.18), we can write the differential equation for (x(k+1)−
x(k)),thatis,
d
(x(k+1)−x(k))= A˜(k)−B˜(k)R−1(B˜(k))TK(k+1) (x(k+1)−x(k))
dt
h i
+ (A˜(k)−A˜(k−1))− B˜(k)R−1(B˜(k))T−B˜(k−1)R−1(B˜(k−1))T K(k+1)
n (cid:16) (cid:17)
(3.16) −B˜(k−1)R−1(B˜(k−1))T(K(k+1)−K(k)) x(k)
o
− B˜(k)R−1(B˜(k))T−B˜(k−1)R−1(B˜(k−1))T S(k+1)n (k+1)
−B(cid:16)˜(k−1)R−1(B˜(k−1))T S(k+1)n (k+1)−S(k)(cid:17)n (k) ,
with the terminal condition x(k+1)(t )−x(cid:0)(k)(t )=0. Applying (cid:1)the variation of constants
f f
formulato(3.16)yields,
−1 t
x(k+1)(t)−x(k)(t)= (S(k+1))T(t) (S(k+1))T(s ) A˜(k)(s )−A˜(k−1)(s )
0
Z
− hB˜(k)R−1(B˜(k)i)T(s )−B˜(k−1)R−1(nB˜h(k(cid:0)−1))T(s ) K(k+1)(s )(cid:1)
−(cid:0)B˜(k−1)R−1(B˜(k−1))T(s ) K(k+1)(s )−K(k)(s )(cid:1)x(k)(s )
− B˜(k)R−1(B˜(k))T−B˜(k−(cid:0)1)R−1(B˜(k−1))T S(k+1(cid:1))in (k+1)
(cid:16) (cid:17)
−B˜(k−1)R−1(B˜(k−1))T(S(k+1)n (k+1)−S(k)n (k)) ds .
o
Itfollowsthat
t
kx(k+1)(t)−x(k)(t)k≤ b kA˜(k)(s )−A˜(k−1)(s )k+b kK(k+1)(s )−K(k)(s )k
3 4
0
Z
h
(3.17) +b kB˜(k)R−1(B˜(k))T(s )−B˜(k−1)R−1(B˜(k−1))T(s )k
5
+b kS(k+1)n (k+1)(s )−S(k)n (k)(s )k ds ,
6
i
whereb ,b , b andb areallfinitetime-varyingcoefficients(seeAppendix8.2). Further-
3 4 5 6
more,sincen (k+1) isaconstantwithineachiterationk,from(2.23)wecanwrite
d
(S(k+1)n (k+1))=− (A˜(k))T−K(k+1)B˜(k)R−1(B˜(k))T S(k+1)n (k+1),
dt
h i
10 S.WANGANDJ.-S.LI
withtheterminalconditionS(k+1)(t )n (k+1)=n (k+1). Thisallowsustowrite
f
d T
(S(k+1)n (k+1)−S(k)n (k))=− A˜(k)−B˜(k)R−1(B˜(k))TK(k+1) (S(k+1)n (k+1)−S(k)n (k))
dt
h i
− (A˜(k)−A˜(k−1))−B˜(k−1)R−1(B˜(k−1))T(K(k+1)−K(k))
n T
− B˜(k)R−1(B˜(k))T−B˜(k−1)R−1(B˜(k−1))T K(k+1) S(k)n (k),
(cid:16) (cid:17) o
withtheterminalconditionS(k+1)(t )n (k+1)−S(k)(t )n (k)=n (k+1)−n (k),andthen
f f
−1
S(k+1)(t)n (k+1)−S(k)(t)n (k)= (S(k+1))T(t) (n (k+1)−n (k))
− tf(S(k+1))T(hs ) (A˜(k)−A˜i(k−1n))−B˜(k−1)R−1(B˜(k−1))T(K(k+1)−K(k))
t
Z
h T
− B˜(k)R−1(B˜(k))T−B˜(k−1)R−1(B˜(k−1))T K(k+1) S(k)n (k)ds .
(cid:16) (cid:17) i o
From(2.24)wemayobtain
kn (k+1)−n (k)k≤k(P(k+1))−1kkP(k+1)(t)−P(k)(t)kkn (k)k
(3.18) +k(P(k+1))−1kkS′(k+1)(t)−S′(k)(t)kkx k,
0
inwhich,byevolving(2.25)backwardintimefromt=t ,thedifferenceP(k+1)(t)−P(k)(t)
f
satisfies
P(k+1)(t)−P(k)(t)=− tf S(k+1)+S(k) B˜(k−1)R−1(B˜(k−1))T S(k+1)(s )−S(k)(s )
t
Z (cid:20)
(cid:0) (cid:1) (cid:0) (cid:1)
(3.19) +S(k+1) B˜(k)R−1(B˜(k)(s ))T−B˜(k−1)R−1(B˜(k−1)(s ))T S(k) ds .
(cid:16) (cid:17) (cid:21)
Using(3.18)and(3.19),weobtain
kS(k+1)(t)n (k+1)−S(k)(t)n (k)k≤ tf b kA˜(k)(s )−A˜(k−1)(s )k
7
t
Z
h
(3.20) +b kB˜(k)R−1(B˜(k))T(s )−B˜(k−1)R−1(B˜(k−1))T(s )k
8
+b kK(k+1)(s )−K(k)(s )k ds ,
9
i
whereb ,b andb arefinitetime-varyingcoefficients(seeAppendix8.2).
7 8 9
Combining the bounds in (3.10), (3.11), (3.15), (3.17), and (3.20), and using the def-
initions of the operators T , T , and T in (3.4), (3.5) and (3.6), respectively, we reach the
1 2 3
inequalitythatholdscomponent-wise,givenby
kT1[x(k),K(k),S(k)n (k)]−T1[x(k−1),K(k−1),S(k−1)n (k−1)]ka
kT2[x(k),K(k),S(k)n (k)]−T2[x(k−1),K(k−1),S(k−1)n (k−1)]ka
 
kT3[x(k),K(k),S(k)n (k)]−T3[x(k−1),K(k−1),S(k−1)n (k−1)]ka
 kx(k)−x(k−1)ka 
(3.21) ≤M kK(k)−K(k−1)ka ,
 
kS(k)n (k)−S(k−1)n (k−1)ka
 