Stochastic Reachability Analysis of Hybrid Systems

250 Pages·2012·3.143 MB·English

by  Luminita Manuela Bujorianu (auth.)

Communications and Control Engineering
SeriesEditors
(cid:2) (cid:2) (cid:2) (cid:2)
A.Isidori J.H.vanSchuppen E.D.Sontag M.Thoma M.Krstic
Forfurthervolumes:
www.springer.com/series/61
Luminita Manuela Bujorianu
Stochastic
Reachability
Analysis of Hybrid
Systems
LuminitaManuelaBujorianu
SchoolofMathematics
UniversityofManchester
Manchester
UK
ISSN0178-5354 CommunicationsandControlEngineering
ISBN978-1-4471-2794-9 e-ISBN978-1-4471-2795-6
DOI10.1007/978-1-4471-2795-6
SpringerLondonDordrechtHeidelbergNewYork
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
LibraryofCongressControlNumber:2012932743
MathematicsSubjectClassification(2010): 60J27,60J25,60J35,60J40,60J60,60K15,62F15,60G40,
32U20,93C30,34A38,49J40,49J55,49L20,49L25,35J99,47B34,45H05
©Springer-VerlagLondonLimited2012
Apartfromanyfairdealingforthepurposesofresearchorprivatestudy,orcriticismorreview,asper-
mittedundertheCopyright,DesignsandPatentsAct1988,thispublicationmayonlybereproduced,
storedortransmitted,inanyformorbyanymeans,withthepriorpermissioninwritingofthepublish-
ers,orinthecaseofreprographicreproductioninaccordancewiththetermsoflicensesissuedbythe
CopyrightLicensingAgency.Enquiriesconcerningreproductionoutsidethosetermsshouldbesentto
thepublishers.
Theuseofregisterednames,trademarks,etc.,inthispublicationdoesnotimply,evenintheabsenceofa
specificstatement,thatsuchnamesareexemptfromtherelevantlawsandregulationsandthereforefree
forgeneraluse.
Thepublishermakesnorepresentation,expressorimplied,withregardtotheaccuracyoftheinformation
containedinthisbookandcannotacceptanylegalresponsibilityorliabilityforanyerrorsoromissions
thatmaybemade.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
In memoryofElenaand Constanta
Preface
Theconceptofhybridsystemsisamathematicalmodelforreallifesystemswhose
behaviourinvolvesamixtureofdiscreteandcontinuousdynamics.Suchsystemsare
everywhere around us, from the simple bouncing ball to sophisticated cars, trains,
planes and robots. Basically, every form of life exhibits a hybrid dynamics. Most
physical and meteorological phenomena exhibit a hybrid dynamics. For scientists
andengineers,thesystemsofinterestarethoseforwhichthediscreteandcontinu-
ousdynamicsinteractorareconnected.Thisinteractioncantakemanyforms,but
themostcommononeoccurswhendiscrete/digitalcontrollersswitchbetweendif-
ferentcontinuousprocesses.Otherformsofinteractionincludediscretetransitions
thatdependoncontinuousevolutions,orotherappearasresultofadecisionprocess,
orbecauseoftheoccurrenceofcertainevents.Thesesystemsarenotgivenadistinct
name.Mostrecently,thetermcyber-physicalsystemshasbeenproposedtodenote
collectionsofsuchinteractingsystemsasnetworkedormulti-agenthybridsystems.
Although many engineered systems exhibit a mixture of discrete and continuous
dynamics,inmanycasestheunderlyingmathematicalmodelisnotahybridsystem.
Wehavetopointoutherethathybridsystemsdenoteafamilyofmathematicalmod-
els, but real life systems are traditionally developedusing either continuous-based
mathematical models (as in control engineering), or discrete-based mathematical
models(asincomputerscience).Methodsbasedonhybridsystemscanbeapplied
straightforwardlytopracticalsystems.Thisiswhysystemengineeringthatuseshy-
briddiscretecontinuousmodels(i.e.hybridsystems)hasbeensubjecttointensive
researchinthepasttwodecades.
Researchinhybridsystemsisusuallygoaloriented.Forexample,inlifesciences,
hybridsystemsareusedmainlyformodellingandanalysis.Incontrolengineering,
the main emphasis is on design, optimisation, analysis and control synthesis. In
computer science, the major focus is on modelling and formal verification. Con-
sequently, three major specialisations of the hybrid system concept arose: hybrid
dynamicalsystems,hybridcontrolsystemsandhybridautomata.
Traditionally, hybrid systems have been proposed as a modelling paradigm for
embeddedsystems.Embeddedsystemsrepresentanengineeringparadigmforelec-
tronic systems operating in physical, often harsh environments. The simplest way
vii
viii Preface
toobtainahybridmodelforanembeddedsystemistomodeltheelectricsystemas
adiscreteautomatonandtheenvironmentasaphysicalprocessgovernedbysome
differentialequations.Inmanycases,theelectronicsystemisahybridsystemitself
andthentheembeddedsystemismodelledasahybridsystemoperatinginaphys-
icalenvironment.Thisisthecaseformostautomotivesystems.Forexample,acar
oraplanehasmanyelectroniccontrollersthatcanswitchbetweendifferentmodes
ofoperationofaphysicaldeviceaccordingtotheenvironmentevolution.
Modernapplicationsofhybridsystemshavebecomeincreasinglycomplex.The
complexity is due to rich interactions, complicated dynamics, randomness of en-
vironment, uncertainty of measurements and tolerance to faults. Traditionally, the
way one reduces system complexity is by employing stochastics. Randomisation
hasbecomeastandardmethodinmodellingandanalysisofcomplexsystems.The
standardtheoryofstochasticprocesses hasthreemajoravenues:theabstractcase,
which is then specialised to the discrete case, and the continuous case. One could
easilyremarkthatthehybriddiscretecontinuouscaseismissinginmathematicalor
engineeringorientedtextbooks.Infact,hybridstochasticsstarteddevelopingonly
inthepastdecade.
Thepurposeofthisbookistopresentrecentdevelopmentsintheuseofhybrid
discretecontinuousstochasticmodellingfortheanalysisofembeddedsystems.The
book is based on an interdisciplinary perspective that can be used in control en-
gineering and computer and life sciences. The modelling aspect is now known as
thetheoryofstochastichybridsystems.Theanalysisaspectisknownasstochastic
reachability. Stochastic hybrid systems constitute a family of models that resulted
fromvarioustypesofrandomisationofhybridsystems.Therearemanywaystoin-
troducerandomnessinamodel.Thereareprobabilistichybridsystemswithproba-
bilitydistributionsassociatedonlywithdiscretetransitions.Otherstochasticmodels
considerthenoisethatperturbsthecontinuousevolutions.Inthemostgeneralform,
astochastichybridsystemconsidersprobabilitydistributionsforbothdiscreteand
continuoustransitions,andmoreoverthesedistributionscandependoneachother.
Thesimplestformofstochastichybridsystemswasintroducedincontrolengineer-
ing three decades ago, and in mathematics even earlier. The most general form of
stochastichybridsystemswasintroducedinthelastdecade,motivatedbyproblems
inairtrafficcontrol.Indeed,aturbulentenvironmentcanproduceasignificantdevi-
ationofaplanetrajectoryfromitsdesignatedflightplan.Inordertopreventserious
problems like sector intrusions or collisions, corrective actions have to be taken.
Suchsituationsaredescribedwithhighaccuracybystochastichybridsystems.
Theanalysisofstochastichybridsystemsisveryimportantsinceitisusedinall
activitiesrelatedtosystemengineering.Aspecifictypeofanalysisisdevelopedin
thiscontext.Itiscalledstochasticreachabilityanalysisanditadmitsaneatmathe-
maticalformulation:Evaluatetheprobabilitythatthesystemstartingfromagiven
initialstatecanreachatargetstateset.
Inthecaseofdiscretesystems,thismathematicalformulationdescribesthewell
established conceptof transient analysis of Markov chains (or probabilisticmodel
checking).Inthecontinuouscase,thisproblemisrelatedtotheclassicaltheoryof
hitting probabilities, or first passage times. In the hybrid case, the system exhibits
Preface ix
behavioursthatviolateallthepropertiesthatareknownindiscreteandcontinuous
cases. Consequently, none of the classical methods can be directly applicable to
hybridsystems.
The research goal in stochastic reachability analysis is twofold: to invent new
specificmethodsortodevelopwaystoapplytheexistingmethodsfromthediscrete
andcontinuouscases.Thisbodyofresearchisthesubjectofthisbook.Currently,
no book is dedicated to stochastic reachability for hybrid systems. This book is
ideal for postgraduate students, offering complete background, the state of the art
in research, case studies and many open problems. Practitioners (engineers) will
findthisbookusefulastheyattempttoconstructmorereliableflexibleandrobust
systems.Althoughthebookisintendedasasolidtheoreticalpresentation,thereare
manyindicationsonhowonemayobtaintoolsupport.Thetopictreatedinthisbook
isahighlyinterdisciplinaryarea,soknowledgeofmultipledisciplinesisrequired.
Thebookisself-containedanditaddressespostgraduatestudents,researchersand
practitionersfromcontrolengineering,computerscience,andappliedmathematics.
Iwouldliketothankmanypeoplewhomadethisbookpossible.First,Iexpress
mygratitudetoSpringereditors,OliverJacksonandKathyMcKenzie.Oliverspot-
tedtheneedforamonographtointroducestochasticreachabilityanalysistoawider
audience,andhechallengedmetowritesuchabook.Hispatienceandconstanten-
couragementhavebeeninvaluableforitscompletion.Kathyhassopatientlystud-
iedthemanuscriptastosuggestdozenofimprovements.Mostimportantly,Iwould
like to thank Professor John Lygeros for introducing me to the area of stochastic
hybrid systems. I am indebted to Savi Maharaj, Henk Blom, Rom Langerak, Hol-
gerHermanns,JianghaiHu,XenofonKoutsoukos,MariaPrandini,AlesandroGiua
and Joost-Pieter Katoen for fruitful collaboration and discussion. I also thank all
colleagues from the Hybridge project for their role in introducing the stochastic
reachabilityanalysisconcept.ProfessorDaveBroomheadmademeevaluatethisre-
searchinamultidisciplinarycontext,withsupportprovidedbytheCICADAproject
attheUniversityofManchester.Thankyou,Dave,forthreeyearsatManchesterand
contagiousenthusiasm!
Last,butnotleast,Ithankmyfamilyfortheirimmensesupport!
Manchester,UK LuminitaManuelaBujorianu
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 MarkovModels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 DiscreteSpaceMarkovModels . . . . . . . . . . . . . . . . . . . 5
2.2.1 DiscreteTimeMarkovChains . . . . . . . . . . . . . . . . 5
2.2.2 ContinuousTimeMarkovChains . . . . . . . . . . . . . . 7
2.3 ContinuousSpaceMarkovModels . . . . . . . . . . . . . . . . . 11
2.3.1 StrongMarkovProcesses . . . . . . . . . . . . . . . . . . 14
2.3.2 ContinuousProcesses . . . . . . . . . . . . . . . . . . . . 15
2.3.3 DiscontinuousProcesses. . . . . . . . . . . . . . . . . . . 18
2.4 MarkovProcessCharacterisations . . . . . . . . . . . . . . . . . . 21
2.4.1 OperatorMethods . . . . . . . . . . . . . . . . . . . . . . 21
2.4.2 KolmogorovEquations . . . . . . . . . . . . . . . . . . . 25
2.4.3 MartingaleProblemandExtendedGenerator . . . . . . . . 26
2.5 SomeRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3 HybridSystems:DeterministictoStochasticPerspectives . . . . . . 31
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 DeterministicHybridSystems . . . . . . . . . . . . . . . . . . . . 32
3.2.1 Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2.2 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.3 RandomnessIssueswhenModellingHybridSystems . . . . . . . 40
3.3.1 ProbabilisticHybridAutomata . . . . . . . . . . . . . . . 41
3.3.2 PiecewiseDeterministicMarkovProcesses . . . . . . . . . 42
3.3.3 StochasticHybridSystems . . . . . . . . . . . . . . . . . 45
3.3.4 SwitchingDiffusionProcesses . . . . . . . . . . . . . . . 47
3.3.5 GeneralSwitchingDiffusionProcesses . . . . . . . . . . . 49
3.3.6 AnalysisofStochasticHybridSystems . . . . . . . . . . . 51
3.4 SomeRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
xi
xii Contents
4 StochasticHybridSystems . . . . . . . . . . . . . . . . . . . . . . . . 55
4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.2 GeneralStochasticHybridSystems . . . . . . . . . . . . . . . . . 56
4.2.1 InformalPresentation . . . . . . . . . . . . . . . . . . . . 57
4.2.2 TheMathematicalModel . . . . . . . . . . . . . . . . . . 58
4.2.3 FormalDefinitions . . . . . . . . . . . . . . . . . . . . . . 61
4.3 MarkovString . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.3.1 InformalDescription. . . . . . . . . . . . . . . . . . . . . 63
4.3.2 TheComponents . . . . . . . . . . . . . . . . . . . . . . . 64
4.3.3 PiecingoutMarkovProcesses . . . . . . . . . . . . . . . . 69
4.3.4 BasicProperties . . . . . . . . . . . . . . . . . . . . . . . 73
4.4 StochasticHybridProcesses . . . . . . . . . . . . . . . . . . . . . 77
4.5 ExamplesofStochasticHybridSystems . . . . . . . . . . . . . . 81
4.5.1 Single-ServerQueues . . . . . . . . . . . . . . . . . . . . 81
4.5.2 AHybridManufacturingSystemModel . . . . . . . . . . 82
4.5.3 ASimplifiedModelofaTruckwithFlexibleTransmission 83
4.5.4 TheStochasticThermostat . . . . . . . . . . . . . . . . . 84
4.6 SomeRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5 StochasticReachabilityConcepts . . . . . . . . . . . . . . . . . . . . 87
5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.2 StochasticHybridControl . . . . . . . . . . . . . . . . . . . . . . 87
5.3 StochasticReachabilityProblem . . . . . . . . . . . . . . . . . . 89
5.3.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.3.2 MathematicalFormulation. . . . . . . . . . . . . . . . . . 91
5.4 MeasurabilityResults . . . . . . . . . . . . . . . . . . . . . . . . 92
5.5 DifferentMeasuresAssociatedwithStochasticReachability . . . . 93
5.5.1 HittingDistributions . . . . . . . . . . . . . . . . . . . . . 94
5.5.2 ExitDistributions . . . . . . . . . . . . . . . . . . . . . . 96
5.5.3 OccupationMeasure . . . . . . . . . . . . . . . . . . . . . 97
5.5.4 OccupationTimeDistribution . . . . . . . . . . . . . . . . 98
5.5.5 ImpreciseProbability . . . . . . . . . . . . . . . . . . . . 98
5.5.6 RéduiteandBalayage . . . . . . . . . . . . . . . . . . . . 101
5.6 SomeRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6 ProbabilisticMethodsforStochasticReachability . . . . . . . . . . . 105
6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.2 ReachabilityEstimationviaHittingTimes . . . . . . . . . . . . . 105
6.2.1 HittingDistributionsforMarkovChains . . . . . . . . . . 105
6.2.2 HittingDistributionsofGSHSs . . . . . . . . . . . . . . . 110
6.3 ReachabilityEstimationviaTransitionSemigroup . . . . . . . . . 111
6.3.1 TheCaseofaMarkovChain . . . . . . . . . . . . . . . . 112
6.3.2 TheCaseofaMarkovProcess . . . . . . . . . . . . . . . 113
6.3.3 TheCaseofaStochasticHybridProcess . . . . . . . . . . 114
6.4 ReachabilityEstimationviaMartinCapacities . . . . . . . . . . . 115
6.4.1 MartinCapacity . . . . . . . . . . . . . . . . . . . . . . . 116
Contents xiii
6.4.2 TheCaseofaMarkovChain . . . . . . . . . . . . . . . . 116
6.4.3 TheCaseofBrownianMotion. . . . . . . . . . . . . . . . 117
6.4.4 TheCaseofaMarkovProcess . . . . . . . . . . . . . . . 117
6.5 ReachabilityEstimationviaMartingaleTheory . . . . . . . . . . . 122
6.5.1 MethodBasedontheMartingaleProblem . . . . . . . . . 122
6.5.2 MethodBasedonMartingaleInequalities . . . . . . . . . . 124
6.5.3 MethodBasedontheBarrierCertificates . . . . . . . . . . 126
6.6 ReachabilityEstimationviaQuadraticForms . . . . . . . . . . . . 129
6.6.1 TargetSets . . . . . . . . . . . . . . . . . . . . . . . . . . 129
6.6.2 DirichletForms . . . . . . . . . . . . . . . . . . . . . . . 130
6.6.3 InducedDirichletForms . . . . . . . . . . . . . . . . . . . 130
6.6.4 UpperBoundsforReachSetProbabilities . . . . . . . . . 132
6.7 SomeRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
7 AnalyticMethodsforStochasticReachability . . . . . . . . . . . . . 135
7.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
7.2 RéduiteandOptimalStopping. . . . . . . . . . . . . . . . . . . . 135
7.3 StochasticReachabilityasanOptimalStoppingProblem. . . . . . 136
7.4 OptimalStoppingProblemforBorelRightProcesses . . . . . . . 137
7.5 MethodsBasedonExcessiveFunctionRepresentations . . . . . . 138
7.5.1 HowtheMethodWorksforStochasticHybridSystems . . 141
7.6 VariationalInequalitiesforStochasticReachability(I) . . . . . . . 145
7.7 OptimalStoppingProblemasanObstacleProblem . . . . . . . . . 147
7.8 EnergyForm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
7.8.1 EnergyFormAssociatedtoaBorelRightProcess . . . . . 150
7.9 VariationalInequalitiesonHilbertSpaces . . . . . . . . . . . . . . 151
7.10 VariationalInequalitiesforStochasticReachability(II). . . . . . . 154
7.10.1 ReachabilityforDiffusionProcesses . . . . . . . . . . . . 155
7.10.2 ReachabilityforJumpProcesses . . . . . . . . . . . . . . 156
7.11 Hamilton–Jacobi–BellmanEquations . . . . . . . . . . . . . . . . 157
7.12 SomeRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
8 StatisticalMethodstoStochasticReachability . . . . . . . . . . . . . 163
8.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
8.2 ImpreciseProbabilities. . . . . . . . . . . . . . . . . . . . . . . . 164
8.2.1 ChoquetIntegral . . . . . . . . . . . . . . . . . . . . . . . 165
8.2.2 Bayes’TheoremforCapacities . . . . . . . . . . . . . . . 165
8.3 CapacitiesandStochasticReachability . . . . . . . . . . . . . . . 167
8.3.1 CapacitiesAssociatedtoGSHSRealisations . . . . . . . . 167
8.3.2 TheGeneralisedBayesRuleforStochasticReachability . . 169
8.3.3 ComputingReachSetProbabilities . . . . . . . . . . . . . 170
8.4 SomeRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
9 StochasticReachabilityBasedonProbabilisticBisimulation . . . . . 173
9.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
9.2 StochasticEquivalence . . . . . . . . . . . . . . . . . . . . . . . 174
9.3 StochasticBisimulation . . . . . . . . . . . . . . . . . . . . . . . 176