Two-Scale Stochastic Systems: Asymptotic Analysis and Control

274 Pages·2003·3.26 MB·English

by  Yuri Kabanov

StochasticMechanics Applications of
RandomMedia Mathematics
SignalProcessing StochasticModelling
andImageSynthesis a4nd9AppliedProbability
MathematicalEconomicsandFinance
StochasticOptimization
StochasticControl
StochasticModelsinLifeSciences
Editedby B.Rozovskii
M.Yor
AdvisoryBoard D.Dawson
D.Geman
G.Grimmett
I.Karatzas
F.Kelly
Y.LeJan
B.Øksendal
E.Pardoux
G.Papanicolaou
Springer-Verlag Berlin Heidelberg GmbH
ApplicationsofMathematics
1 Fleming/Rishel,DeterministicandStochasticOptimalControl(1975)
2 Marchuk,MethodsofNumericalMathematics1975,2nd.ed.1982)
3 Balakrishnan,AppliedFunctionalAnalysis(1976,2nd.ed.1981)
4 Borovkov,StochasticProcessesinQueueingTheory(1976)
5 Liptser/Shiryaev,StatisticsofRandomProcessesI:GeneralTheory(1977,2nd.ed.2001)
6 Liptser/Shiryaev,StatisticsofRandomProcessesII:Applications(1978,2nd.ed.2001)
7 Vorob’ev,GameTheory:LecturesforEconomistsandSystemsScientists(1977)
8 Shiryaev,OptimalStoppingRules(1978)
9 Ibragimov/Rozanov,GaussianRandomProcesses(1978)
10 Wonham,LinearMultivariableControl:AGeometricApproach(1979,2nd.ed.1985)
11 Hida,BrownianMotion(1980)
12 Hestenes,ConjugateDirectionMethodsinOptimization(1980)
13 Kallianpur,StochasticFilteringTheory(1980)
14 Krylov,ControlledDiffusionProcesses(1980)
15 Prabhu,StochasticStorageProcesses:Queues,InsuranceRisk,andDams(1980)
16 Ibragimov/Has’minskii,StatisticalEstimation:AsymptoticTheory(1981)
17 Cesari,Optimization:TheoryandApplications(1982)
18 Elliott,StochasticCalculusandApplications(1982)
19 Marchuk/Shaidourov,DifferenceMethodsandTheirExtrapolations(1983)
20 Hijab,StabilizationofControlSystems(1986)
21 Protter,StochasticIntegrationandDifferentialEquations(1990)
22 Benveniste/Me´tivier/Priouret,AdaptiveAlgorithmsandStochasticApproximations(1990)
23 Kloeden/Platen,NumericalSolutionofStochasticDifferentialEquations
(1992,corr.3rdprinting1999)
24 Kushner/Dupuis,NumericalMethodsforStochasticControlProblemsinContinuous
Time(1992)
25 Fleming/Soner,ControlledMarkovProcessesandViscositySolutions(1993)
26 Baccelli/Bre´maud,ElementsofQueueingTheory(1994)
27 Winkler,ImageAnalysis,RandomFieldsandDynamicMonteCarloMethods
(1995,2nd.ed.2003)
28 Kalpazidou,CycleRepresentationsofMarkovProcesses(1995)
29 Elliott/Aggoun/Moore,HiddenMarkovModels:EstimationandControl(1995)
30 Herna´ndez-Lerma/Lasserre,Discrete-TimeMarkovControlProcesses(1995)
31 Devroye/Gyo¨rfi/Lugosi,AProbabilisticTheoryofPatternRecognition(1996)
32 Maitra/Sudderth,DiscreteGamblingandStochasticGames(1996)
33 Embrechts/Klu¨ppelberg/Mikosch,ModellingExtremalEventsforInsuranceandFinance
(1997,corr.4thprinting2003)
34 Duflo,RandomIterativeModels(1997)
35 Kushner/Yin,StochasticApproximationAlgorithmsandApplications(1997)
36 Musiela/Rutkowski,MartingaleMethodsinFinancialModelling(1997)
37 Yin,Continuous-TimeMarkovChainsandApplications(1998)
38 Dembo/Zeitouni,LargeDeviationsTechniquesandApplications(1998)
39 Karatzas,MethodsofMathematicalFinance(1998)
40 Fayolle/Iasnogorodski/Malyshev,RandomWalksintheQuarter-Plane(1999)
41 Aven/Jensen,StochasticModelsinReliability(1999)
42 Hernandez-Lerma/Lasserre,FurtherTopicsonDiscrete-TimeMarkovControlProcesses
(1999)
43 Yong/Zhou,StochasticControls.HamiltonianSystemsandHJBEquations(1999)
44 Serfozo,IntroductiontoStochasticNetworks(1999)
45 Steele,StochasticCalculusandFinancialApplications(2001)
46 Chen/Yao,FundamentalsofQueuingNetworks:Performance,Asymptotics,
andOptimization(2001)
47 Kushner,HeavyTrafficAnalysisofControlledQueueingandCommunicationsNetworks
(2001)
48 Fernholz,StochasticPortfolioTheory(2002)
49 Kabanov/Pergamenshchikov,Two-ScaleStochasticSystems(2003)
50 Han,Information-SpectrumMethodsinInformationTheory(2003)
Yuri Kabanov
Sergei Pergamenshchikov
Two-Scale
Stochastic Systems
Asymptotic Analysis and Control
1 3
Authors
YuriKabanov
Universite´deFranche-Comte´
De´partementdeMathe´matiques
16routedeGray
25030BesançonCedex,France
e-mail:[email protected]
SergeiPergamenshchikov
Universite´deRouen
LIFAR,UFRSciencesetTechniques
76821MontSaintAignanCedex,France
e-mail:[email protected]
ManagingEditors
B.Rozovskii M.Yor
UniversityofSouthernCalifornia Universite´deParisVI
Center for Applied Mathematical LaboratoiredeProbabilite´s
Sciences etMode`lesAle´atoires
1042West36thPlace, 175,rueduChevaleret
DenneyResearchBuilding308 75013Paris,France
LosAngeles,CA90089,USA
MathematicsSubjectClassification(2000):93-02,60-02,49-02
CoverpatternbycourtesyofRickDurrett(CornellUniversity,Ithaca)
CoverillustrationbyMargaritaKabanova
LibraryofCongressCataloging-in-PublicationDataappliedfor
BibliographicinformationpublishedbyDieDeutscheBibliothek
DieDeutscheBibliothekliststhispublicationintheDeutscheNationalbibliografie;detailed
bibliographicdataisavailableintheInternetat<http://dnb.ddb.de>.
ISSN0172-4568
ISBN 978-3-642-08467-6 ISBN 978-3-662-13242-5 (eBook)
DOI 10.1007/978-3-662-13242-5
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthe
materialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,
recitation,broadcasting,reproductiononmicrofilmorinanyotherway,andstorageindata
banks.Duplicationofthispublicationorpartsthereofispermittedonlyundertheprovisions
oftheGermanCopyrightLawofSeptember9,1965,initscurrentversion,andpermission
forusemustalwaysbeobtainedfromSpringer-Verlag Berlin Heidelberg GmbH.
ViolationsareliableforprosecutionundertheGermanCopyrightLaw.
http://www.springer.de
©Springer-VerlagBerlinHeidelberg2003
Originally published by Springer-Verlag Berlin Heidelberg New York in 2003
Softcover reprint of the hardcover 1st edition 2003
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
Coverdesign:ErichKirchner,Heidelberg
TypesettingLe-TexJelonek,Schmidt&Vo¨cklerGbR,Leipzig
Printedonacid-freepaper SPIN:10702078 41/3142ck-543210
Table of Contents
Introduction.................................................. IX
0 Warm-up ................................................. 1
0.1 Processeswith Fast MarkovModulations .................. 1
0.1.1 Model Formulation ............................... 1
0.1.2 Asymptotic Behavior of Distributions ............... 2
0.2 The Li´enardOscillator Under Random Force............... 6
0.3 Filtering of Nearly ObservedProcesses .................... 9
0.4 Stochastic Approximation ............................... 11
1 Toolbox: Moment Bounds for Solutions of Stable SDEs... 19
1.1 Moment Bounds for Nonlinear Equations.................. 20
1.1.1 Key Lemma ..................................... 20
1.1.2 Bounds Efficient on Small Intervals ................. 22
1.1.3 Bounds Efficient on LargeIntervals ................. 24
1.2 Bounds for Linear Equations............................. 26
1.2.1 Assumption on the Fundamental Matrix............. 26
1.2.2 Differential Equationswith Random Coefficients ..... 27
1.2.3 The Continuity Theorem .......................... 29
1.2.4 Linear SDEs with Unbounded Coefficients........... 31
1.3 On the Growth Rate of the Maximal Function ............. 34
1.3.1 Lapeyre’s Inequality .............................. 34
1.3.2 Ornstein–Uhlenbeck Process ....................... 37
1.3.3 Sample PathGrowth ............................. 39
1.3.4 Fernique’s Lemma................................ 40
2 The Tikhonov Theory for SDEs........................... 43
2.1 The Stochastic TikhonovTheorem........................ 45
2.1.1 Setting.......................................... 45
2.1.2 BoundaryLayer Behavior ......................... 46
2.1.3 LargeScale Behavior ............................. 49
2.1.4 Concluding Step ................................. 54
VI TableofContents
2.2 The First-OrderAsymptotics for Fast Variables ............ 56
2.2.1 Basic Hypotheses................................. 56
2.2.2 The First-Order Correction........................ 57
2.2.3 The First-Order Approximationof the Rest Point .... 59
2.2.4 Normal ApproximationResult ..................... 62
2.3 Higher-OrderExpansions................................ 63
2.3.1 Formal Expansions ............................... 63
2.3.2 Convergenceof the Remainder ..................... 65
2.3.3 ExpansionAround the Rest Point .................. 69
2.4 Stochastic Approximation: Proofs ........................ 70
2.4.1 Asymptotic Expansionfor the Output Signal ........ 70
2.4.2 The Asymptotic Expansionat the Root ............. 78
2.4.3 Averaging ....................................... 80
2.4.4 Proofof Theorem 0.4.6............................ 83
3 Large Deviations ......................................... 87
3.1 Deviations in the Uniform Metric......................... 88
3.1.1 Formulation of the Result ......................... 88
3.1.2 A Lower Exponential Bound
for the Non-Exit Probability....................... 89
3.1.3 An Upper Bound for the Probabilityof Deviation
of a Trajectoryfrom the Lebesgue Sets of Sε ........ 91
T
3.1.4 Proofof Theorem 3.1.1............................ 99
3.1.5 Example: the Ornstein–UhlenbeckProcess........... 104
3.2 Deviations in the Metric of L2[0,T]....................... 105
4 Uniform Expansions for Two-Scale Systems............... 111
4.1 No Diffusion at the Fast Variable......................... 112
4.1.1 Formal Calculations .............................. 112
4.1.2 Integrabilityof Coefficients ........................ 119
4.1.3 The BoundaryLayer Function of Zero Order......... 120
4.1.4 BoundaryLayer Functions of Higher Order .......... 124
4.1.5 Proofof Theorem 4.1.1............................ 129
4.2 Expansionsfor the General Model ........................ 133
4.2.1 Formulations .................................... 133
4.2.2 Growth of Coefficients ............................ 135
4.2.3 Proofof Theorem 4.2.1............................ 136
4.3 Li´enardOscillator Driven by a Random Force.............. 140
TableofContents VII
5 Two-Scale Optimal Control Problems..................... 145
5.1 Semilinear ControlledSystem ............................ 146
5.1.1 The Model and Main Result ....................... 146
5.1.2 Proofof Proposition5.1.2 ......................... 148
5.1.3 Proofof Proposition5.1.3 ......................... 155
5.1.4 Proofof Theorem 5.1.1............................ 157
5.2 Structure of the Attainability Sets ........................ 158
5.2.1 Weak and Strong Solutions of SDEs ................ 158
5.2.2 Closed Loop ControlsVersus Open Loop ............ 160
5.2.3 “Tubes” and Attainability Sets for Feedback Controls. 164
5.2.4 Extreme Pointsof the Set of Attainable Densities .... 167
5.2.5 On the Existence of Optimal Control ............... 169
5.2.6 Comparisonof Attainability Sets ................... 171
5.3 Convergenceof the Attainability Sets, I ................... 175
5.3.1 The Dontchev–VeliovTheorem..................... 175
5.3.2 The First Stochastic Generalization................. 176
5.4 Convergenceof the Attainability Sets, II .................. 180
5.4.1 Formulation of the Result ......................... 180
5.4.2 The Fast Variable Model .......................... 182
5.4.3 General Case .................................... 185
5.4.4 Proofof Theorem 5.4.1............................ 187
6 Applications .............................................. 193
6.1 Applications to PDEs................................... 193
6.2 Fast MarkovModulations Revisited....................... 199
6.2.1 Main Result ..................................... 199
6.2.2 Preliminariesfrom Weak Convergence............... 200
6.2.3 Proofof Theorem 6.3.1............................ 202
6.2.4 Calculations and Estimates ........................ 203
6.2.5 Cox Processeswith Fast MarkovModulations........ 206
6.3 Accuracyof Approximate Filters ......................... 207
6.4 Signal Estimation ...................................... 208
6.5 Linear Regulator with Infinite Horizon .................... 213
6.5.1 Sensitive Probabilistic Criteria ..................... 213
6.5.2 Linear-QuadraticRegulator........................ 214
6.5.3 Preliminaries .................................... 216
6.5.4 Proofof Theorem 6.5.2............................ 218
6.5.5 Example ........................................ 220
VIII TableofContents
Appendix..................................................... 223
A.1 Basic Facts About SDEs ................................ 223
A.1.1 Existence and Uniqueness of Strong Solutions
for SDEs with Random Coefficients................. 223
A.1.2 Existence and Uniqueness with a LyapunovFunction . 224
A.1.3 Moment Bounds for Linear SDEs................... 225
A.1.4 The NovikovCondition ........................... 226
A.2 Exponential Bounds for Fundamental Matrices ............. 227
A.2.1 Uniform Bound in the Time-HomogeneousCase...... 227
A.2.2 NonhomogeneousCase............................ 229
A.2.3 Models with Singular Perturbations................. 230
A.3 Total Variation Distance and Hellinger Processes ........... 234
A.3.1 Total Variation Distance and Hellinger Integrals...... 234
A.3.2 The Hellinger Processes ........................... 235
A.3.3 Example: Diffusion-Type Processes ................. 238
A.4 Hausdorff Metric ....................................... 239
A.5 MeasurableSelection.................................... 240
A.5.1 Aumann Theorem ................................ 240
A.5.2 Filippov Implicit Function Lemma.................. 241
A.5.3 MeasurableVersion of the Carath´eodoryTheorem .... 241
A.6 Compact Sets in P(X) .................................. 243
A.6.1 Notations and Preliminaries ....................... 243
A.6.2 Integrationof Stochastic Kernels ................... 245
A.6.3 Distributions of Integrals.......................... 246
A.6.4 Compactness of the Limit of Attainability Sets....... 248
A.6.5 Supports of Conditional Distributions............... 250
A.7 The Koml´osTheorem .................................. 250
Historical Notes .............................................. 255
References.................................................... 259
Index......................................................... 265
Introduction
In many complex systems one can distinguish “fast” and “slow” processes
withradicallydifferentvelocities.Inmathematicalmodelsbasedondifferen-
tialequations,suchtwo-scalesystemscanbedescribedbyintroducingexplic-
itlyasmallparameterεontheleft-handsideofstateequationsforthe“fast”
variables,andtheseequationsarereferredtoassingularly perturbed.Surpris-
ingly, this kind of equation attracted attention relatively recently (the idea
of distinguishing “fast” and “slow” movements is, apparently, much older).
RobertO’Malley,incommentstohisbook,attributestheoriginofthewhole
historyofsingularperturbationstothecelebratedpaperofPrandtl[79].This
was an extremely short note, the text of his talk at the Third International
MathematicalCongressin 1904:the youngauthorbelievedthat ithadto be
literally identical with his ten-minute long oral presentation. In spite of its
length, it had a tremendous impact on the subsequent development. Many
famous mathematicians contributed to the discipline, having numerous and
important applications. We mention here only the name of A.N. Tikhonov,
whodevelopedattheendofthe1940sinhisdoctoralthesisabeautifultheory
for non-linear systems where the fast variables can almost reach their equi-
librium states while the slow variables still remain near their initial values:
the aerodynamics of a winged object like a plane or the “Katiusha” rocket
may servean example of such a system.
Itisgenerallyacceptedthat theprobabilisticmodeling ofreal-worldpro-
cesses is more adequate than the deterministic modeling. Needs of applica-
tions resulted in an increasing interest in the theory of two-scale stochastic
systems where an essential progress has been achieved during the last 25
years. However, in comparison with the classical theory, many vast areas
have never been explored and only a few research monographs have been
available.
The main subject of this book is a stochastic version of the Tikhonov
theory including some optimal control problems for systems with singular
perturbations. Of course, we do not pretend to cover all aspects of singular
perturbations: the absolute majority of the results presented here are our
own.The principalmodel wedealwith isgivenbythe stochasticdifferential
equations
dxε =f(t,xε,yε)dt+g(t,xε,yε)dwx, xε =xo,
t t t t t t 0
X Introduction
εdyε =F(t,xε,yε)dt+σ(ε)G(t,xε,yε)dwy, yε =yo,
t t t t t t 0
where wx and wy are independent Wiener processes. We avoid in this brief
introduction detailed discussions of needed assumptions on coefficients and
mentiononlythatforthelinearcase,whereF(t,x,y)=A x+B y,werequire
t t
the matrix-valued function A to be continuous and “exponentially stable”.
t
Sometimes, in problems where we feel that the full generality will make our
study unreasonably complicated, we constrain ourselves by considering the
model with only fast variables, given by a singularly perturbed SDE, which
is, in certain cases, important and interesting in itself.
The assumption on the behavior of the coefficient when ε ↓ 0 merits a
discussion. In the literature, one may observe a dominance of studies where
σ(ε)=ε1/2 thoughthereisanoticeablenumberofpaperswheremodelswith
σ = εδ, δ > 1/2, are also treated. No doubt, the case δ = 1/2, where the
typical random perturbation has an amplitude which behaves like a square
root of the velocity, is worthy of attention. In general, models of this type
fit the Bogoliubov averaging principle remarkably well. This principle pre-
scribes to do the following. To get a description of the limiting behavior of
the slowvariableone should “freeze” it in the equation for the fast variable,
i.e. consider the latter separately with a constant parameter x, representing
a fixed point of the state space and replacing varying xε, and calculate the
t
invariant measure (for time-dependent coefficients, “instantaneously invari-
ant”measure)oftheresultingdynamics.Thecoefficientsofthelimitingslow
dynamics are obtained by averaging (in the y variable) the corresponding
coefficients of the original prelimit equation with respect to these invariant
measures (depending on x). Obviously, the ergodicity of the fast motion for
the “frozen”slow variablecan be postulated explicitly or via some sufficient
conditions.Undervarioussetsofhypothesesonemayexpectvarioustypesof
convergence.Themostdevelopedtheoryconcernswiththeweakconvergence;
itissummarizedinthedeeptreatisebyHaroldKushner[57].Weavoidinter-
sectionswiththisexcellentbookbyconcentratingoureffortsonmodelswith
δ>1/2.Theyariseinanaturalwayandweprovideseveraltypicalexamples
widely discussed in the literature. At first glance, such models seem to be
muchsimpler thanthosewith δ=1/2since thediffusion inthe fastvariable
vanishes rapidly as ε tends to zero. However, the mathematics needed for
their analysisisnontrivial(recallthatthedeterministicsetting isjust apar-
ticularcase).Ifδ=1/2,thefastvariableusuallyexhibitsonlyconvergencein
distributionand,typically,aquestionposedtothemodelconcernseitherthe
convergenceoftheslowvariable(withamarginalinteresttothefastmotion)
or the convergence in distributions of the whole dynamics. We notice here
that the slow variable may converge almost surely also if δ = 0, though the
fastonedoesnotconvergeinanysense(recallearlierattemptsofphysiciststo
constructatheoryofBrownianmotionwithvelocity).Incontrasttothis,for
models with δ>1/2one canprovea uniformconvergencein both variables;
to getit for thefastmotiononthe wholetime intervaloneneeds“boundary