Iterative learning control for multi-agent systems coordination

249 Pages·2017·13.211 MB·English

by  Li, Xuefang, Shen, Dong, Xu, Jian-Xin, Yang, Shiping

IterativeLearningControlforMulti-agentSystemsCoordination
Iterative Learning Control for Multi-agent
Systems Coordination
ShipingYang
Jian-XinXu
XuefangLi
NationalUniversityofSingapore
DongShen
BeijingUniversityofChemicalTechnology,P.R.China
Thiseditionfirstpublished2017
©2017JohnWiley&SonsSingaporePte.Ltd
RegisteredOffice
JohnWiley&SonsSingaporePte.Ltd,1FusionopolisWalk,#07-01SolarisSouthTower,Singapore138628.
Fordetailsofourglobaleditorialoffices,forcustomerservicesandforinformationabouthowtoapplyfor
permissiontoreusethecopyrightmaterialinthisbookpleaseseeourwebsiteatwww.wiley.com.
AllRightsReserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystemor
transmitted,inanyformorbyanymeans,electronic,mechanical,photocopying,recording,scanning,or
otherwise,exceptasexpresslypermittedbylaw,withouteitherthepriorwrittenpermissionofthePublisher,
orauthorizationthroughpaymentoftheappropriatephotocopyfeetotheCopyrightClearanceCenter.
RequestsforpermissionshouldbeaddressedtothePublisher,JohnWiley&SonsSingaporePte.Ltd.,
1FusionopolisWalk,#07-01SolarisSouthTower,Singapore138628,tel:[phone],fax:[phone],
email:[email protected].
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynot
beavailableinelectronicbooks.
Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.Allbrand
namesandproductnamesusedinthisbookaretradenames,servicemarks,trademarksorregistered
trademarksoftheirrespectiveowners.ThePublisherisnotassociatedwithanyproductorvendor
mentionedinthisbook.Thispublicationisdesignedtoprovideaccurateandauthoritativeinformationin
regardtothesubjectmattercovered.ItissoldontheunderstandingthatthePublisherisnotengagedin
renderingprofessionalservices.Ifprofessionaladviceorotherexpertassistanceisrequired,theservicesofa
competentprofessionalshouldbesought.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbesteffortsin
preparingthisbook,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyor
completenessofthecontentsofthisbookandspecificallydisclaimanyimpliedwarrantiesofmerchantability
orfitnessforaparticularpurpose.Itissoldontheunderstandingthatthepublisherisnotengagedin
renderingprofessionalservicesandneitherthepublishernortheauthorshallbeliablefordamagesarising
herefrom.Ifprofessionaladviceorotherexpertassistanceisrequired,theservicesofacompetent
professionalshouldbesought.
LibraryofCongressCataloging-in-PublicationData
Names:Yang,Shiping,1987–author.|Xu,Jian-Xin,author.|Li,Xuefang,
1985–author.|Shen,Dong,1982–author.
Title:Iterativelearningcontrolformulti-agentsystemscoordination/by
ShipingYang,Jian-XinXu,XuefangLi,DongShen.
Description:Singapore:JohnWiley&Sons,Inc.,2017.|Includesbibliographicalreferencesandindex.
Identifiers:LCCN2016052027(print)|LCCN2016056133(ebook)|ISBN9781119189046(hardback)|
ISBN9781119189060(pdf)|ISBN9781119189077(epub)
Subjects:LCSH:Intelligentcontrolsystems.|Multiagentsystems.|Machinelearning.|Iterativemethods
(Mathematics)|BISAC:TECHNOLOGY&ENGINEERING/Robotics.
Classification:LCCTJ217.5.Y362017(print)|LCCTJ217.5(ebook)|DDC629.8/9–dc23
LCrecordavailableathttps://lccn.loc.gov/2016052027
AcatalogrecordforthisbookisavailablefromtheBritishLibrary.
Setin10/12ptWarnockbySPiGlobal,Pondicherry,India
Coverdesign:Wiley
Coverimage:©MichaelMann/Gettyimages
10 9 8 7 6 5 4 3 2 1
v
Contents
Preface ix
1 Introduction 1
1.1 IntroductiontoIterativeLearningControl 1
1.1.1 Contraction-MappingApproach 3
1.1.2 CompositeEnergyFunctionApproach 4
1.2 IntroductiontoMASCoordination 5
1.3 MotivationandOverview 7
1.4 CommonNotationsinThisBook 9
2 OptimalIterativeLearningControlforMulti-agentConsensus
Tracking 11
2.1 Introduction 11
2.2 PreliminariesandProblemDescription 12
2.2.1 Preliminaries 12
2.2.2 ProblemDescription 13
2.3 MainResults 15
2.3.1 ControllerDesignforHomogeneousAgents 15
2.3.2 ControllerDesignforHeterogeneousAgents 20
2.4 OptimalLearningGainDesign 21
2.5 IllustrativeExample 23
2.6 Conclusion 26
3 IterativeLearningControlforMulti-agentCoordinationUnder
Iteration-VaryingGraph 27
3.1 Introduction 27
3.2 ProblemDescription 28
3.3 MainResults 29
3.3.1 FixedStronglyConnectedGraph 29
3.3.2 Iteration-VaryingStronglyConnectedGraph 32
3.3.3 UniformlyStronglyConnectedGraph 37
3.4 IllustrativeExample 38
3.5 Conclusion 40
vi Contents
4 IterativeLearningControlforMulti-agentCoordinationwithInitial
StateError 41
4.1 Introduction 41
4.2 ProblemDescription 42
4.3 MainResults 43
4.3.1 DistributedD-typeUpdatingRule 43
4.3.2 DistributedPD-typeUpdatingRule 48
4.4 IllustrativeExamples 49
4.5 Conclusion 50
5 Multi-agentConsensusTrackingwithInputSharingbyIterative
LearningControl 53
5.1 Introduction 53
5.2 ProblemFormulation 54
5.3 ControllerDesignandConvergenceAnalysis 54
5.3.1 ControllerDesignWithoutLeader’sInputSharing 55
5.3.2 OptimalDesignWithoutLeader’sInputSharing 58
5.3.3 ControllerDesignwithLeader’sInputSharing 59
5.4 ExtensiontoIteration-VaryingGraph 60
5.4.1 Iteration-VaryingGraphwithSpanningTrees 60
5.4.2 Iteration-VaryingStronglyConnectedGraph 60
5.4.3 UniformlyStronglyConnectedGraph 62
5.5 IllustrativeExamples 63
5.5.1 Example1:Iteration-InvariantCommunicationGraph 63
5.5.2 Example2:Iteration-VaryingCommunicationGraph 64
5.5.3 Example3:UniformlyStronglyConnectedGraph 66
5.6 Conclusion 68
6 AHOIM-BasedIterativeLearningControlSchemeforMulti-agent
Formation 69
6.1 Introduction 69
6.2 KinematicModelFormulation 70
6.3 HOIM-BasedILCforMulti-agentFormation 71
6.3.1 ControlLawforAgent1 72
6.3.2 ControlLawforAgent2 74
6.3.3 ControlLawforAgent3 75
6.3.4 SwitchingBetweenTwoStructures 78
6.4 IllustrativeExample 78
6.5 Conclusion 80
7 P-typeIterativeLearningforNon-parameterizedSystemswith
UncertainLocalLipschitzTerms 81
7.1 Introduction 81
7.2 MotivationandProblemDescription 82
7.2.1 Motivation 82
7.2.2 ProblemDescription 83
7.3 ConvergencePropertieswithLyapunovStabilityConditions 84
Contents vii
7.3.1 PreliminaryResults 84
7.3.2 LyapunovStableSystems 86
7.3.3 SystemswithStableLocalLipschitzTermsbutUnstableGlobal
LipschitzFactors 90
7.4 ConvergencePropertiesinthePresenceofBoundingConditions 92
7.4.1 SystemswithBoundedDriftTerm 92
7.4.2 SystemswithBoundedControlInput 94
7.5 ApplicationofP-typeRuleinMASwithLocalLipschitzUncertainties 97
7.6 Conclusion 99
8 SynchronizationforNonlinearMulti-agentSystemsbyAdaptive
IterativeLearningControl 101
8.1 Introduction 101
8.2 PreliminariesandProblemDescription 102
8.2.1 Preliminaries 102
8.2.2 ProblemDescriptionforFirst-OrderSystems 102
8.3 ControllerDesignforFirst-OrderMulti-agentSystems 105
8.3.1 MainResults 105
8.3.2 ExtensiontoAlignmentCondition 107
8.4 ExtensiontoHigh-OrderSystems 108
8.5 IllustrativeExample 113
8.5.1 First-OrderAgents 114
8.5.2 High-OrderAgents 115
8.6 Conclusion 118
9 DistributedAdaptiveIterativeLearningControlforNonlinear
Multi-agentSystemswithStateConstraints 123
9.1 Introduction 123
9.2 ProblemFormulation 124
9.3 MainResults 127
9.3.1 OriginalAlgorithms 127
9.3.2 ProjectionBasedAlgorithms 135
9.3.3 SmoothFunctionBasedAlgorithms 138
9.3.4 AlternativeSmoothFunctionBasedAlgorithms 141
9.3.5 PracticalDead-ZoneBasedAlgorithms 145
9.4 IllustrativeExample 147
9.5 Conclusion 174
10 SynchronizationforNetworkedLagrangianSystemsunder
DirectedGraphs 175
10.1 Introduction 175
10.2 ProblemDescription 176
10.3 ControllerDesignandPerformanceAnalysis 177
10.4 ExtensiontoAlignmentCondition 183
10.5 IllustrativeExample 184
10.6 Conclusion 188
viii Contents
11 GeneralizedIterativeLearningforEconomicDispatchProblemina
SmartGrid 189
11.1 Introduction 189
11.2 Preliminaries 190
11.2.1 In-NeighborandOut-Neighbor 190
11.2.2 Discrete-TimeConsensusAlgorithm 191
11.2.3 AnalyticSolutiontoEDPwithLossCalculation 192
11.3 MainResults 193
11.3.1 UpperLevel:EstimatingthePowerLoss 194
11.3.2 LowerLevel:SolvingEconomicDispatchDistributively 194
11.3.3 GeneralizationtotheConstrainedCase 197
11.4 LearningGainDesign 198
11.5 ApplicationExamples 200
11.5.1 CaseStudy1:ConvergenceTest 201
11.5.2 CaseStudy2:RobustnessofCommandNodeConnections 202
11.5.3 CaseStudy3:PlugandPlayTest 203
11.5.4 CaseStudy4:Time-VaryingDemand 205
11.5.5 CaseStudy5:ApplicationinLargeNetworks 207
11.5.6 CaseStudy6:RelationBetweenConvergenceSpeedandLearningGain 207
11.6 Conclusion 208
12 SummaryandFutureResearchDirections 209
12.1 Summary 209
12.2 FutureResearchDirections 210
12.2.1 OpenIssuesinMASControl 210
12.2.2 Applications 214
AppendixAGraphTheoryRevisit 223
AppendixBDetailedProofs 225
B.1 HOIMConstraintsDerivation 225
B.2 ProofofProposition2.1 226
B.3 ProofofLemma2.1 227
B.4 ProofofTheorem8.1 229
B.5 ProofofCorollary8.1 230
Bibliography 233
Index 245
ix
Preface
Thecoordinationandcontrolproblemsofmulti-agentsystems(MAS)havebeenexten-
sively studied by the control community due to the broad practical applications, for
example,theformationcontrolproblem,searchandrescuebymultipleaerialvehicles,
synchronization,sensorfusion,distributedoptimization,theeconomicdispatchprob-
lem in power systems, and so on. Meanwhile, many industry processes require both
repetitiveexecutionsandcoordinationamongseveralindependententities.Thisobser-
vation motivates the research of multi-agent coordination from an iterative learning
control(ILC)perspective.Thisbookisdedicatedtotheapplicationofiterativelearning
controltomulti-agentcoordinationproblems.
Inordertostudymulti-agentcoordinationbyILC,anextradimension,theiteration
domain,isintroducedintotheproblem.Achallengingissueincontrollingmulti-agent
systems by ILC is the non-perfect repeating characteristics of MAS. The inherent
nature of MAS such as heterogeneity, information sharing, sparse and intermittent
communication, imperfect initial conditions, and inconsistent target tracking trajec-
toriesincreasesthecomplexityoftheproblem.Duetothesefactors,controllerdesign
becomesachallengingproblem.Thisbookprovidesdetailedguidelinesforthedesignof
learningcontrollersundervariouscoordinationconditions,inasystematicmanner.The
maincontentcanbeclassifiedintotwopartsfollowingthetwomainframeworksofILC,
namely,contraction-mapping(CM)andcompositeenergyfunction(CEF)approaches.
Chapters 2–7 apply the CM approach, while Chapters 8–10 apply the CEF approach.
Eachchapterstudiesthecoordinationproblemundercertainconditions.Forexample,
Chapter 2 assumes a fixed communication topology, Chapter 3 assumes a switching
topology, and Chapter 4 addresses the initial state error problem in multi-agent
coordination.Inasense,eachchapteraddressesauniquecoordinationcontrolproblem
forMAS.Chapters2–10discusscontinuous-timesystems.InChapter11wepresenta
generalizediterativelearningalgorithmtosolveanoptimalpowerdispatchproblemin
asmartgridbyutilizingdiscrete-timeconsensusalgorithms.
This book is self contained and intensive. Prior knowledge of ILC and MAS is not
required.Chapter1providesarudimentaryintroductiontothetwoareas.Twominimal
examplesofILCarepresentedinChapter1,andashortreviewofsometerminologies
in graph theory is provided in AppendixA. Readers can skip the preliminary parts if
they are familiar with the domain. We present detailed convergence proofs for each
controlleraswebelievethatunderstandingthetheoreticalderivationscanbenefitread-
ers in two ways. On the one hand, it helps readers appreciate the controller design.
On the other hand, the control design and analysis techniques can be transferred to
x Preface
otherdomainstofacilitatefurtherexplorationinvariouscontrolapplications.Specifi-
callyforindustrialexpertsandpractitioners,weprovidedetailedillustrativeexamples
ineachchaptertoshowhowthosecontrolalgorithmsareimplemented.Theexamples
demonstratetheeffectivenessofthelearningcontrollersandcanbemodifiedtohandle
practicalproblems.
1
1
Introduction
1.1 IntroductiontoIterativeLearningControl
Iterativelearningcontrol(ILC),asaneffectivecontrolstrategy,isdesignedtoimprove
current control performance for unpredictable systems by fully utilizing past control
experience. Specifically, ILC is designed for systems that complete tasks over a fixed
time interval and perform them repeatedly. The underlying philosophy mimics the
human learning process that “practice makes perfect.” By synthesizing control inputs
from previous control inputs and tracking errors, the controller is able to learn from
pastexperienceandimprovecurrenttrackingperformance.ILCwasinitiallydeveloped
byArimotoetal.(1984),andhasbeenwidelyexploredbythecontrolcommunitysince
then(Moore,1993;BienandXu,1998;ChenandWen,1999;Longman,2000;Norrlof
andGunnarsson,2002;XuandTan,2003;Bristowetal.,2006;Mooreetal.,2006;Ahn
etal., 2007a; Rogers etal., 2007; Ahn etal., 2007b; Xu etal., 2008; Wang etal., 2009,
2014).
Figure1.1 shows the schematic diagram of an ILC system, where the subscript i
denotestheiterationindexandy denotesthereferencetrajectory.Basedontheinput
d
signal, u, at the ith iteration, as well as the tracking error e =y −y, the input u
i i d i i+1
for the next iteration, namely the (i+1)th iteration, is constructed. Meanwhile, the
inputsignalu willalsobestoredintomemoryforuseinthe(i+2)thiteration.Itis
i+1
importanttonotethatinFigure1.1,aclosedfeedbackloopisformedintheiteration
domain rather than the time domain. Compared to other control methods such as
proportional-integral-derivative (PID) control and sliding mode control, there are a
number of distinctive features about ILC. First, ILC is designed to handle repetitive
control tasks, while other control techniques don’t typically take advantage of task
repetition—under a repeatable control environment, repeating the same feedback
wouldyieldthesamecontrolperformance.Incontrast,byincorporatinglearning,ILC
isabletoimprovethecontrolperformanceiteratively.Second,thecontrolobjectiveis
different. ILC aims at achieving perfect tracking over the whole operational interval.
Most control methods aim to achieve asymptotic convergence in tracking accuracy
over time. Third, ILC is a feedforward control method if viewed in the time domain.
The plant shown in Figure1.1 is a generalized plant, that is, it can actually include a
feedbackloop.ILCcanbeusedtofurtherimprovetheperformanceofthegeneralized
plant.Assuch,thegeneralizedplantcouldbemadestableinthetimedomain,which
is helpful in guaranteeing transient response while learning takes place. Last but not
IterativeLearningControlforMulti-agentSystemsCoordination,FirstEdition.
ShipingYang,Jian-XinXu,XuefangLi,andDongShen.
©2017JohnWiley&SonsSingaporePte.Ltd.Published2017byJohnWiley&SonsSingaporePte.Ltd.