Advanced Optimal Control and Applications Involving Critic Intelligence PDF
Preview Advanced Optimal Control and Applications Involving Critic Intelligence
Intelligent Control and Learning Systems 6 Ding Wang · Mingming Ha · Mingming Zhao Advanced Optimal Control and Applications Involving Critic Intelligence Intelligent Control and Learning Systems Volume 6 SeriesEditor DongShen ,SchoolofMathematics,RenminUniversityofChina,Beijing, Beijing,China The Springer book series Intelligent Control and Learning Systems addresses the emergingadvancesinintelligentcontrolandlearningsystemsfrombothmathemat- ical theory and engineering application perspectives. It is a series of monographs andcontributedvolumesfocusingonthein-depthexplorationoflearningtheoryin controlsuchasiterativelearning,machinelearning,deeplearning,andotherssharing thelearningconcept,andtheircorrespondingintelligentsystemframeworksinengi- neeringapplications.Thisseriesisfeaturedbythecomprehensiveunderstandingand practicalapplicationoflearningmechanisms.Thisbookseriesinvolvesapplications inindustrialengineering,controlengineering,andmaterialengineering,etc. TheIntelligentControlandLearningSystembookseriespromotestheexchangeof emergingtheoryandtechnologyofintelligentcontrolandlearningsystemsbetween academia and industry. It aims to provide a timely reflection of the advances in intelligent control and learning systems. This book series is distinguished by the combination of the system theory and emerging topics such as machine learning, artificialintelligence,andbigdata.Asacollection,thisbookseriesprovidesvaluable resources to a wide audience in academia, the engineering research community, industry and anyone else looking to expand their knowledge in intelligent control andlearningsystems. · · Ding Wang Mingming Ha Mingming Zhao Advanced Optimal Control and Applications Involving Critic Intelligence DingWang MingmingHa FacultyofInformationTechnology SchoolofAutomationandElectrical BeijingUniversityofTechnology Engineering Beijing,China UniversityofScienceandTechnology Beijing MingmingZhao Beijing,China FacultyofInformationTechnology BeijingUniversityofTechnology Beijing,China ISSN 2662-5458 ISSN 2662-5466 (electronic) IntelligentControlandLearningSystems ISBN 978-981-19-7290-4 ISBN 978-981-19-7291-1 (eBook) https://doi.org/10.1007/978-981-19-7291-1 ©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNature SingaporePteLtd.2023 Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar ordissimilarmethodologynowknownorhereafterdeveloped. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional claimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore Preface Nowadays,wearegoingthroughaprofoundrevolutioninallwalksoflife,duetothe rapid development of artificial intelligence and intelligent techniques. Among the numerous amazing achievements, intelligent optimization methods are commonly applied,notonlytoscientificresearchbutalsotopracticalengineering.Thoseappli- cationareascovercybernetics,computerscience,computationalmathematics,and so on. Remarkably, the idea of optimization plays an important role in artificial- intelligence-based advanced control design and is significant to construct various intelligent systems. Unlike the ordinary linear case, nevertheless, the nonlinear optimalcontrolisoftendifficulttoaddress.Particularly,withthewidepopularityof networkedtechniquesandtheextensionofcomputercontrolscales,moreandmore dynamical systems are encountered with the difficulty of building mathematical modelsaccuratelyandareoperatedbasedonincreasingcommunicationresources. Forexample,theintelligentoptimalcontrolofwastewatertreatmentsystemsisan importantavenueofresourcescyclicutilizationwhencopingwiththemodernurban diseases.However,therealwaysexistobviousnonlinearitiesanduncertaintieswithin wastewatertreatmentprocesses.Becauseofthemoreandmorecommondynamics complexity,itisalwaysdifficulttoachievedirectoptimizationdesignandtherelated control efficiencies are often low. Therefore, it is necessary to establish advanced optimalcontrolstrategiesforcomplexdiscrete-timenonlinearsystems. Characterizedbyagent-environmentinteraction,reinforcementlearningisclosely relatedtodynamicprogrammingwhenconductingintelligentoptimizationdesign. Duringtheadaptivecriticframework,reinforcementlearningiscombinedwiththe neuralnetworkapproximatortocopewithcomplexoptimizationproblemsapproxi- mately.Inthelasttwodecades,theadaptivecriticmechanismhasbeenwidelyusedto solvecomplexoptimalcontrolproblemsandmanyexcellentresultshavebeendevel- opedinthesenseofadaptiveoptimalcontroldesign.Thisbookintendstoreportnew optimal control results with critic intelligence for complex discrete-time systems, whichcoversthenovelcontroltheory,advancedcontrolmethods,andtypicalappli- cations for wastewater treatment systems. Therein, combining with artificial intel- ligence techniques, such as neural networks and reinforcement learning, the novel intelligentcriticcontroltheoryaswellasaseriesofadvancedoptimalregulationand v vi Preface trajectory tracking strategies are established for discrete-time nonlinear systems, followed by application verifications to complex wastewater treatment processes. Consequently, developing such kind of critic intelligence approaches is of great significancefornonlinearoptimizationandwastewaterrecycling. Overall,tenchaptersareincludedinthisbook,focusedonbackgroundintroduc- tion (Chap. 1), optimal regulation (Chaps. 2–4), trajectory tracking (Chaps. 5–7), and industrial applications particularly wastewater treatment (Chaps. 8–10). Prof. DingWangcontributestoeachofthetenchapters.Dr.MingmingHacontributesto Chaps. 1–3, 5–8. Dr. Mingming Zhao contributes to Chaps. 1, 4, 9, and 10. They performthediscussion,revision,andimprovementforalltenchapters. InChap.1,consideringlearningapproximatorsandthereinforcementformula- tion,alearning-basedcontrolframeworkisestablishedandappliedtointelligentcritic learningandcontrolforcomplexnonlinearsystemsunderunknowndynamicswithin discrete-timedomain.Inaddition,thebases,thederivations,andrecentprogressesof criticintelligencefordiscrete-timeadvancedoptimalcontroldesignarepresented. Intermsofnormalregulationandtrajectorytracking,theadvancedoptimalcontrol methodsarealsoverifiedviasimulationexperimentsandwastewatertreatmentappli- cations,whicheffectivelyaddressunknownfactorsforcomplexnonlinearsystems, observablyenhancecontrolefficiencies,andreallyimproveintelligentoptimization performances. InChap.2,basedonaneffectiveheuristicdynamicprogrammingalgorithm,the adaptiveevent-triggeredcontrollerisdesignedforaclassofdiscrete-timenonlinear systemswithconstrainedinputs.First,toconquercontrolconstraints,anonquadratic performance index is introduced and the triggering threshold is provided with stabilityanalysisusingtheLyapunovtechnique.Second,threeneuralnetworksare constructed in the algorithm scheme and a novel weight initialization approach is developed to improve approximation accuracy of the model network. Simulation resultsfurtherdemonstratethevalidityoftheproposedstrategybycomparisonwith thetraditionalmethod. In Chap. 3, the event-based self-learning optimal regulation is developed for discrete-time nonlinear systems based on the iterative dual heuristic dynamic programmingalgorithm,whichsubstantiallydecreasesthecomputationcost.First, duringtheiterativeprocess,theconvergenceoftheevent-basedadaptivecriticalgo- rithmisdiscussed.Second,anappropriatetriggeringconditionisestablishedsoasto ensuretheinput-to-statestabilityoftheevent-basedsystem.Inaddition,themixed- driven control framework is clarified with data and event considerations. Simula- tionexamplesareconductedtodemonstratetheeffectivenessandsuperiorityofthe proposedapproachwhencomparedwiththetraditionaltechnique. InChap.4,aneffectivegeneralizedvalueiterationalgorithmisestablishedtodeal withthediscountednear-optimalcontrolissueforsystemswithcontrolconstraints. First,anonquadraticperformancefunctionisintroducedtoovercomesaturationand the initial cost function is selected as an arbitrary positive semidefinite function insteadofzero.Consideringconstrainedsystemswiththediscountfactor,themono- tonicityandconvergenceoftheiterativecostfunctionsequencearediscussed.Then, inordertoimplementtheproposedalgorithm,twoneuralnetworksareconstructed Preface vii toapproximatethecostfunctionandthecontrolpolicy.Additionally,twosimulation examplesareconductedtocertifythevalidityoftheproposedmethod. In Chap. 5, a novel neuro-optimal tracking controller is developed based on valueiterationfordiscrete-timenonlinearsystems.Theoptimaltrajectorytracking problem is transformed into the optimal regulation problem through constructing a new augmented system. Then, the convergence of the iterative cost function for the value-iteration-based tracking control algorithm is provided and the uniformly ultimately bounded stability of the closed-loop system is discussed. In addition, theheuristicdynamicprogrammingalgorithmisutilizedtoimplementtheproposed methodandtwosimulationsareconductedtotestifytheeffectivenessoftheproposed strategy. InChap.6,adata-basedoptimaltrackingcontroltechniqueisestablishedbased on the iterative dual heuristic dynamic programming algorithm for discrete-time nonaffine systems. In order to implement the proposed algorithm, three neural networksareconductedtoapproximatethesystemmodel,thecostatefunction,and the control strategy, respectively. In addition, when the model network is trained, biases are introduced to improve identification accuracy and the gradient descent algorithmisutilizedtoupdateweightsandbiasesofalllayers.ThroughtheLyapunov approach,theuniformlyultimatelyboundedstabilityisdiscussedandthesimulation iscarriedouttodemonstratethevalidityoftheproposedmethod. In Chap. 7, the discounted optimal control design is developed based on the adaptivecriticschemewithanovelperformanceindextosolvethetrackingcontrol problemforbothnonlinearandlinearsystems.First,thetrackingerrorscannotbe eliminatedcompletelyinpreviousmethodwithatraditionalperformanceindex.To dealwiththeproblem,anovelcostfunctionisintroduced.Second,thecostfunction inoptimaltrackingcontrolcannotbedeemedasaLyapunovfunction,andtherefore thenewstabilityanalysisisdiscussedtoensurethetrackingerrortendstozeroasthe numberoftimestepsincreases.Twonumericalsimulationsareperformedtoverify theeffectivenesswiththecomparationofthetrackingperformancefortheiterative adaptivecriticdesignsunderdifferentperformanceindexfunctions. InChap.8,adata-driveniterativeadaptivecriticschemeisdevelopedtodealwith thenonlinearoptimalfeedbackcontrolprobleminwastewatertreatmentsystems.In order to ensure the dissolved oxygen concentration and the nitrate level are main- tainedattheirdesiredsettingpoints,theiterativeadaptivecriticcontrolframework is established. In this way, faster response and less oscillation are obtained using theschemecomparedwiththeincrementalproportional–integral–derivativemethod andtheconvergenceisdiscussed.Inaddition,themixed-drivencontrolframework isalsointroducedandthenappliedtothewastewatertreatmentplant.Thesimulation demonstrates the effectiveness of the proposed intelligent controller for nonlinear optimizationandwastewaterrecycling. InChap.9,adata-driveniterativeadaptivetrackingcontrollerinvolvingthedual heuristicdynamicprogrammingstructureisestablishedtoimprovethecontrolperfor- manceofthedissolvedoxygenconcentrationandthenitratenitrogenconcentrationin theconstrainednonlinearplantofwastewatertreatment.First,toaddressasymmetric constraintsofthecontrolinput,anonquadraticperformancefunctionisintroduced. viii Preface Then,thesteadycontrolstrategyisobtainedandthenextsystemstateisevaluatedby themodelnetwork.Throughapplyingtothewastewatertreatmentplant,theproposed methodisverifiedtobefeasibleandefficient. In Chap. 10, based on the accelerated generalized value iteration algorithm, a hybridintelligenttrackingcontrolstrategyisdevelopedtoachieveoptimaltracking for a class of nonlinear discrete-time systems. Both offline and online training are utilized,wheretheformercanobtaintheadmissibletrackingcontrollawandthelatter canenhancethecontrolperformance.Inaddition,theaccelerationfactorisintroduced to improve the performance of value iteration and the admissible tracking control isobtained.Theinput–outputdataoftheunknownsystemiscollectedtoconstruct the model neural network, so as to attain the steady control and the approximate controlmatrix.Consideringapproximationerrorsofneuralnetworks,theuniformly ultimatelyboundedstabilityisdiscussedviatheLyapunovapproach.Twoexamples withindustrialapplicationbackgroundsareinvolvedtodemonstratetheavailability andeffectivenessoftheproposedmethod. Beijing,China DingWang Beijing,China MingmingHa Beijing,China MingmingZhao September2022 Acknowledgements TheauthorswouldliketothankProf.DerongLiu,Prof.JunfeiQiao,andProf.Long Cheng for providing valuable discussions when conducting related research. The authorsalsowouldliketothankJinRen,XinXu,andHuilingZhao,forpreparing somebasicmaterialsofthisbook.TheauthorsalsowouldliketothankNingGao, PengXin,LingzhiHu,JunlongWu,JiangyuWang,XinLi,ZihangZhou,Wenqian Fan, Haiming Huang, Ao Liu, Yuan Wang, and Hongyu Ma, for checking and improvingsomechaptersofthebook. TheauthorsareverygratefultoNationalKeyResearchandDevelopmentProgram of China (Grant 2021ZD0112302), the National Natural Science Foundation of China(Grant62222301,61773373),andBeijingNaturalScienceFoundation(Grant JQ19013), for providing necessary financial support to our research in the past 4 years. ix
The list of books you might like
