Ergodic Control of Diffusion Processes

342 Pages·2012·1.771 MB·English

by  Arapostathis A., Borkar V.S., Ghosh M.K.

ERGODIC CONTROL OF DIFFUSION PROCESSES
Thiscomprehensivevolumeonergodiccontrolofdiffusionshighlightsintuition
alongsidetechnicalarguments.AconciseaccountofMarkovprocesstheoryis
followedbyacompletedevelopmentofthefundamentalissuesandformalismsin
controlofdiffusions.Thisthenleadstoacomprehensivetreatmentofergodic
control,aproblemthatstraddlesstochasticcontrolandtheergodictheoryof
Markovprocesses.
Theinterplaybetweentheprobabilisticandergodic-theoreticaspectsofthe
problem,notablytheasymptoticsofempiricalmeasuresononehand,andthe
analyticaspectsleadingtoacharacterizationofoptimalityviatheassociated
Hamilton–Jacobi–Bellmanequationontheother,isclearlyrevealed.Themore
abstractcontrolledmartingaleproblemisalsopresented,inadditiontomanyother
relatedissuesandmodels.
Assumingonlygraduate-levelprobabilityandanalysis,theauthorsdevelopthe
theoryinamannerthatmakesitaccessibletousersinappliedmathematics,
engineering,financeandoperationsresearch.
EncyclopediaofMathematicsandItsApplications
Thisseriesisdevotedtosignificanttopicsorthemesthathavewideapplicationin
mathematicsormathematicalscienceandforwhichadetaileddevelopmentofthe
abstracttheoryislessimportantthanathoroughandconcreteexplorationofthe
implicationsandapplications.
BooksintheEncyclopediaofMathematicsandItsApplicationscovertheir
subjectscomprehensively.Lessimportantresultsmaybesummarizedasexercises
attheendsofchapters.Fortechnicalities,readerscanbereferredtothe
bibliography,whichisexpectedtobecomprehensive.Asaresult,volumesare
encyclopedicreferencesormanageableguidestomajorsubjects.
EncyclopediaofMathematicsanditsApplications
AllthetitleslistedbelowcanbeobtainedfromgoodbooksellersorfromCambridge
UniversityPress.Foracompleteserieslistingvisit
http://www.cambridge.org/mathematics
88 T.MoraSolvingPolynomialEquationSystemsI
89 K.BichtelerStochasticIntegrationwithJumps
90 M.LothaireAlgebraicCombinatoricsonWords
91 A.A.IvanovandS.V.ShpectorovGeometryofSporadicGroupsII
92 P.McMullenandE.SchulteAbstractRegularPolytopes
93 G.Gierzetal.ContinuousLatticesandDomains
94 S.R.FinchMathematicalConstants
95 Y.JabriTheMountainPassTheorem
96 G.GasperandM.RahmanBasicHypergeometricSeries,2ndedn
97 M.C.PedicchioandW.Tholen(eds.)CategoricalFoundations
98 M.E.H.IsmailClassicalandQuantumOrthogonalPolynomialsinOneVariable
99 T.MoraSolvingPolynomialEquationSystemsII
100 E.OlivieriandM.Eula´liaVaresLargeDeviationsandMetastability
101 A.Kushner,V.LychaginandV.RubtsovContactGeometryandNonlinearDifferentialEquations
102 L.W.BeinekeandR.J.Wilson(eds.)withP.J.CameronTopicsinAlgebraicGraphTheory
103 O.J.StaffansWell-PosedLinearSystems
104 J.M.Lewis,S.LakshmivarahanandS.K.DhallDynamicDataAssimilation
105 M.LothaireAppliedCombinatoricsonWords
106 A.MarkoeAnalyticTomography
107 P.A.MartinMultipleScattering
108 R.A.BrualdiCombinatorialMatrixClasses
109 J.M.BorweinandJ.D.VanderwerffConvexFunctions
110 M.-J.LaiandL.L.SchumakerSplineFunctionsonTriangulations
111 R.T.CurtisSymmetricGenerationofGroups
112 H.Salzmann,T.Grundho¨fer,H.Ha¨hlandR.Lo¨wenTheClassicalFields
113 S.PeszatandJ.ZabczykStochasticPartialDifferentialEquationswithLe´vyNoise
114 J.BeckCombinatorialGames
115 L.BarreiraandY.PesinNonuniformHyperbolicity
116 D.Z.ArovandH.DymJ-ContractiveMatrixValuedFunctionsandRelatedTopics
117 R.Glowinski,J.-L.LionsandJ.HeExactandApproximateControllabilityforDistributedParameter
Systems
118 A.A.BorovkovandK.A.BorovkovAsymptoticAnalysisofRandomWalks
119 M.DezaandM.DutourSikiric´GeometryofChemicalGraphs
120 T.NishiuraAbsoluteMeasurableSpaces
121 M.PrestPurity,SpectraandLocalisation
122 S.KhrushchevOrthogonalPolynomialsandContinuedFractions
123 H.NagamochiandT.IbarakiAlgorithmicAspectsofGraphConnectivity
124 F.W.KingHilbertTransformsI
125 F.W.KingHilbertTransformsII
126 O.CalinandD.-C.ChangSub-RiemannianGeometry
127 M.Grabischetal.AggregationFunctions
128 L.W.BeinekeandR.J.Wilson(eds.)withJ.L.GrossandT.W.TuckerTopicsinTopologicalGraph
Theory
129 J.Berstel,D.PerrinandC.ReutenauerCodesandAutomata
130 T.G.FaticoniModulesoverEndomorphismRings
131 H.MorimotoStochasticControlandMathematicalModeling
132 G.SchmidtRelationalMathematics
133 P.KornerupandD.W.MatulaFinitePrecisionNumberSystemsandArithmetic
134 Y.CramaandP.L.Hammer(eds.)BooleanFunctions
135 V.Berthe´andM.Rigo(eds.)Combinatorics,AutomataandNumberTheory
136 A.Krista´ly,V.D.Ra˘dulescuandC.VargaVariationalPrinciplesinMathematicalPhysics,Geometry
andEconomics
137 J.BerstelandC.ReutenauerNoncommutativeRationalSerieswithApplications
138 B.CourcelleGraphStructureandMonadicSecond-OrderLogic
139 M.FiedlerMatricesandGraphsinGeometry
140 N.VakilRealAnalysisthroughModernInfinitesimals
141 R.B.ParisHadamardExpansionsandHyperasymptoticEvaluation
142 Y.CramaandP.L.HammerBooleanFunctions
Encyclopedia of Mathematics and its Applications
Ergodic Control of
Diffusion Processes
ARI ARAPOSTATHIS
UniversityofTexas,Austin
VIVEK S. BORKAR
TataInstituteofFundamentalResearch,Mumbai
MRINAL K. GHOSH
IndianInstituteofScience,Bangalore
cambridge university press
Cambridge,NewYork,Melbourne,Madrid,CapeTown,
Singapore,Sa˜oPaulo,Delhi,Tokyo,MexicoCity
CambridgeUniversityPress
TheEdinburghBuilding,CambridgeCB28RU,UK
PublishedintheUnitedStatesofAmericabyCambridgeUniversityPress,NewYork
www.cambridge.org
Informationonthistitle:www.cambridge.org/9780521768405
(cid:1)C A.Arapostathis,V.S.BorkarandM.K.Ghosh2012
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2012
PrintedintheUnitedKingdomattheUniversityPress,Cambridge
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary
LibraryofCongressCataloguinginPublicationdata
Arapostathis,Ari,1954–
Ergodiccontrolofdiffusionprocesses/AriArapostathis,VivekS.Borkar,MrinalK.Ghosh.
p. cm.–(Encyclopediaofmathematicsanditsapplications;143)
Includesbibliographicalreferencesandindexes.
ISBN978-0-521-76840-5
1.Diffusionprocesses. 2.Ergodictheory. I.Borkar,VivekS. II.Ghosh,MrinalK.,1956–
III.Title. IV.Series.
QA274.75.A73 2011
519.2(cid:2)33–dc23 2011031547
ISBN978-0-521-76840-5Hardback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceor
accuracyofURLsforexternalorthird-partyinternetwebsitesreferredto
inthispublication,anddoesnotguaranteethatanycontentonsuch
websitesis,orwillremain,accurateorappropriate.
Dedicatedtoourparents:
TheodoreandHelen,
ShripadandSarita,
SudhirKumarandUma.

Contents
Preface pagexi
FrequentlyUsedNotation xv
1 MarkovProcessesandErgodicProperties 1
1.1 Introduction 1
1.2 Markovprocesses 4
1.3 SemigroupsassociatedwithMarkovprocesses 9
1.4 Themartingaleapproach 14
1.5 ErgodictheoryofMarkovprocesses 15
1.6 Diffusionprocesses 27
1.7 Bibliographicalnote 29
2 ControlledDiffusions 30
2.1 Introduction 30
2.2 Solutionconcepts 30
2.3 Relaxedcontrols 47
2.4 AtopologyforMarkovcontrols 57
2.5 Stabilityofcontrolleddiffusions 59
2.6 Stabilityofnondegeneratecontrolleddiffusions 64
2.7 Thediscountedandergodiccontrolproblems 82
2.8 Bibliographicalnote 85
3 NondegenerateControlledDiffusions 86
3.1 Introduction 86
3.2 Convexanalyticproperties 86
3.3 Uniformrecurrenceproperties 94
3.4 Existenceofoptimalcontrols 100
3.5 Thediscountedcontrolproblem 107
3.6 TheHJBequationunderanear-monotoneassumption 112
3.7 Thestablecase 124
3.8 One-dimensionalcontrolleddiffusions 147
3.9 Bibliographicalnote 153
viii Contents
4 VariousTopicsinNondegenerateDiffusions 154
4.1 Introduction 154
4.2 Multi-objectiveproblems 154
4.3 Singularperturbationsinergodiccontrol 160
4.4 Controloveraboundeddomain 172
4.5 Anapplication 174
4.6 Bibliographicalnote 193
5 ControlledSwitchingDiffusions 194
5.1 Themathematicalmodel 194
5.2 Cooperativeellipticsystems 199
5.3 Recurrenceandergodicityoffullycoupledswitchingdiffusions 204
5.4 Existenceofanoptimalcontrol 210
5.5 HJBequations 211
5.6 Thegeneralcase 212
5.7 Anexample 213
5.8 Bibliographicalnote 216
6 ControlledMartingaleProblems 217
6.1 Introduction 217
6.2 Thecontrolledmartingaleproblem 218
6.3 Ergodicoccupationmeasures 221
6.4 Extremalsolutions 231
6.5 Existenceresults 241
6.6 Thelinearprogrammingformulation 246
6.7 Krylov’sMarkovselection 248
6.8 Bibliographicalnote 252
7 DegenerateControlledDiffusions 253
7.1 Introduction 253
7.2 Controlledmartingaleformulation 253
7.3 Asymptoticallyflatcontrolleddiffusions 254
7.4 Partiallynondegeneratecontrolleddiffusions 263
7.5 Bibliographicalnote 278
8 ControlledDiffusionswithPartialObservations 279
8.1 Introduction 279
8.2 Controllednonlinearfilters 280
8.3 Theseparatedcontrolproblem 285
8.4 Splitchainandthepseudo-atom 288
8.5 Dynamicprogramming 293
8.6 Bibliographicalnote 299