Stochastic Global Optimization and Its Applications with Fuzzy Adaptive Simulated Annealing

221 Pages·2012·2.95 MB·English

by  Hime Aguiar e Oliveira Junior

HimeAguiareOliveiraJunior,LesterIngber,AntonioPetraglia,
MarianeRemboldPetraglia,andMariaAugustaSoaresMachado
StochasticGlobalOptimizationandItsApplicationswithFuzzyAdaptive
SimulatedAnnealing
IntelligentSystemsReferenceLibrary,Volume35
Editors-in-Chief
Prof.JanuszKacprzyk Prof.LakhmiC.Jain
SystemsResearchInstitute UniversityofSouthAustralia
PolishAcademyofSciences Adelaide
ul.Newelska6 MawsonLakesCampus
01-447Warsaw SouthAustralia5095
Poland Australia
E-mail:[email protected] E-mail:[email protected]
Furthervolumesofthisseriescanbefoundonourhomepage:
springer.com
Vol.11.SamuliNiiranenandAndreRibeiro(Eds.) Vol.24.DawnE.HolmesandLakhmiC.Jain(Eds.)
InformationProcessingandBiologicalSystems,2011 DataMining:FoundationsandIntelligentParadigms,2011
ISBN978-3-642-19620-1 ISBN978-3-642-23240-4
Vol.12.FlorinGorunescu Vol.25.DawnE.HolmesandLakhmiC.Jain(Eds.)
DataMining,2011 DataMining:FoundationsandIntelligentParadigms,2011
ISBN978-3-642-19720-8 ISBN978-3-642-23150-6
Vol.13.WitoldPedryczandShyi-MingChen(Eds.) Vol.26.TauseefGulrezandAboulEllaHassanien(Eds.)
GranularComputingandIntelligentSystems,2011 AdvancesinRoboticsandVirtualReality,2011
ISBN978-3-642-19819-9 ISBN978-3-642-23362-3
Vol.14.GeorgeA.AnastassiouandOktayDuman Vol.27.CristinaUrdiales
TowardsIntelligentModeling:StatisticalApproximation CollaborativeAssistiveRobotforMobilityEnhancement
Theory,2011 (CARMEN),2011
ISBN978-3-642-19825-0 ISBN978-3-642-24901-3
Vol.15.AntoninoFrenoandEdmondoTrentin Vol.28.TatianaValentineGuy,MiroslavKa´rny´and
HybridRandomFields,2011 DavidH.Wolpert(Eds.)
ISBN978-3-642-20307-7 DecisionMakingwithImperfectDecisionMakers,2012
ISBN978-3-642-24646-3
Vol.16.AlexieiDingli
Vol.29.RoumenKountchevandKazumiNakamatsu(Eds.)
KnowledgeAnnotation:MakingImplicitKnowledge
AdvancesinReasoning-BasedImageProcessingIntelligent
Explicit,2011
Systems,2012
ISBN978-3-642-20322-0
ISBN978-3-642-24692-0
Vol.17.CrinaGrosanandAjithAbraham
Vol.30.MarinaV.SokolovaandAntonio
IntelligentSystems,2011
Ferna´ndez-Caballero
ISBN978-3-642-21003-7
DecisionMakinginComplexSystems,2012
Vol.18.AchimZielesny ISBN978-3-642-25543-4
FromCurveFittingtoMachineLearning,2011
Vol.31.LudomirM.Laudan´ski
ISBN978-3-642-21279-6
BetweenCertaintyandUncertainty,2012
Vol.19.GeorgeA.Anastassiou ISBN978-3-642-25696-7
IntelligentSystems:ApproximationbyArtificialNeural
Vol.32.Jose´J.PazosArias,AnaFerna´ndezVilas,
Networks,2011
andRebecaP.D´ıazRedondo
ISBN978-3-642-21430-1
RecommenderSystemsfortheSocialWeb,2012
Vol.20.LechPolkowski ISBN978-3-642-25693-6
ApproximateReasoningbyParts,2011
Vol.33.JieLu,LakhmiC.Jain,andGuangquanZhang
ISBN978-3-642-22278-8
HandbookonDecisionMaking,2012
Vol.21.IgorChikalov ISBN978-3-642-25754-4
AverageTimeComplexityofDecisionTrees,2011
Vol.34.ErnestoSanchez,GiovanniSquillero,
ISBN978-3-642-22660-1
andAlbertoTonda
Vol.22.Przemys(cid:2)lawRz˙ewski, IndustrialApplicationsofEvolutionaryAlgorithms,2012
EmmaKusztina,RyszardTadeusiewicz, ISBN978-3-642-27466-4
andOlegZaikin
Vol.35.HimeAguiareOliveiraJunior,LesterIngber,
IntelligentOpenLearningSystems,2011
AntonioPetraglia,MarianeRemboldPetraglia,
ISBN978-3-642-22666-3
andMariaAugustaSoaresMachado
Vol.23.DawnE.HolmesandLakhmiC.Jain(Eds.) StochasticGlobalOptimizationandItsApplicationswith
DataMining:FoundationsandIntelligentParadigms,2011 FuzzyAdaptiveSimulatedAnnealing,2012
ISBN978-3-642-23165-0 ISBN978-3-642-27478-7
Hime Aguiar e Oliveira Junior, Lester Ingber,
Antonio Petraglia, Mariane Rembold Petraglia,
and Maria Augusta Soares Machado
Stochastic Global Optimization
and Its Applications with Fuzzy
Adaptive Simulated Annealing
123
Authors
Dr.HimeAguiareOliveiraJunior Prof.MarianeRemboldPetraglia
RiodeJaneiro UniversidadeFederaldoRiodeJaneiro
Brazil RiodeJaneiro
Brazil
Prof.LesterIngber
LesterIngberResearch Prof.MariaAugustaSoaresMachado
Ashland IBMEC-RJ
USA RiodeJaneiro
Brazil
Prof.AntonioPetraglia
UniversidadeFederaldoRiodeJaneiro
RiodeJaneiro
Brazil
ISSN1868-4394 e-ISSN1868-4408
ISBN978-3-642-27478-7 e-ISBN978-3-642-27479-4
DOI10.1007/978-3-642-27479-4
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2011945156
(cid:2)c Springer-VerlagBerlinHeidelberg2012
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpub-
lication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforany
errorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespect
tothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Global optimization is a very important subject. It finds applications in Biology,
Statistics, Engineering, Mathematics itself, Medicine, Management Science, Eco-
nomicsandinvirtuallyeverythingyoucanimagine.Althoughmanyexcellentmeth-
ods have been developed so far, several of them assume certain conditions about
functionstobeprocessed,asconvexityordifferentiability,and,inpractice,weoften
getintosituationsinwhichobjectivefunctionstobeoptimized(minimizedormax-
imized)arenotdifferentiable,convexorevencontinuous.Insuchsettings,gradient-
basedmethodsarenotofmuchhelpanditisnecessaryto findmoregeneralways
togetgoodresults.Duringthelastdecadesanexpressivenumberofnewglobalop-
timizationmethodswereidealized,aimingtoreachthatmoregeneralobjectiveand
substantialpartofthembelongtothecategoryofmetaheuristicmethods,beingalso
known generically as metaheuristics. Their majority are of a probabilistic nature,
thatistosay,useprobabilitytheoryresultstogettotheirtarget,consequentlybeing
classified asstochasticmethodsaswell. Knowledgeofthe capabilitiesandlimita-
tionsofthesealgorithmsleadstoabetterunderstandingoftheirreachovervarious
applicationsandpointsusthewaytofutureresearchonimprovingandextendingal-
gorithms’theoreticalfoundationsandrespectiveimplementations.Ourgoalinthis
book is to present a description of some techniques for solving stochastic global
optimizationproblemsandto detailone,in particular-FuzzyAdaptiveSimulated
Annealing(FuzzyASA).Bypresentingseveral,detailedexamplesofitsapplication
we will try to stimulate the reader’sintuitionand makethe use of FuzzyASA (or
ASA)easiertoallwishingtosolvetheirproblemswiththistool.
Itisimportanttonotethat,inthisbook,allexampleprogramcodesarepresented
inthehopethattheywillbeusefulinthelearningprocess,butwithoutanywarranty
- withouteven the implied warrantyof fitness for a particularpurpose.Besides, it
isadvisableto highlightthatthearchitectureoftheexampleroutinesisnotneces-
sarilythemostefficientincomputationalterms,takingintoaccountthattheywere
constructedforpedagogicalreasonsonly.
Formal mathematical requirements are kept to a minimum and our focus will
be on continuousproblems,although ASA is able to handle discrete optimization
tasks as well. This book could be used in courses related to optimization as well
VI Preface
asbyresearchersandpractitionersinEngineeringandindustry,andisadequatefor
self-study,also.
Prerequisitesforreadingthisbookincludesome knowledgeof LinearAlgebra,
introductoryNumericAnalysisandbasicProbabilityTheory.
Theworkisdividedinthreeparts:
• Anintroductorysetofchapters,exposingbasicfactsaboutsomeimportantglobal
optimizationmethodsandtheiroverallstructure;
• A secondpartcontaininga detaileddescriptionofAdaptiveSimulatedAnneal-
ing(ASA)anditsfuzzycontrolledversion,howtomakeconstrainedanduncon-
strainedoptimizationwiththemandseveralillustrativeexamplesthat,wehope,
willbehelpfultothereader;
• AfinalpartcontainingchaptersthatdescribeapplicationsofFuzzyASAtosev-
eralareasofknowledgelikesignalprocessing,statisticalestimation,fuzzymod-
elingandnonlinearequationsolving.
Tocomplementthematerialcontainedinthetextandmakethelearningtimeshorter,
weinvitethereadertotrydoingsomeglobaloptimizationtasksbyhimorherself.
It suffices to download the publicly available code at www.ingber.com and start
coding.Incaseofdoubts,suggestionsoranythingelse,please,feelfreetocontact
us.Wehopeyouenjoythebookandthatitwillbeusefulinyourwork.
RiodeJaneiro,BRAZIL HimeAguiareOliveiraJunior
Ashland,USA LesterIngber
AntonioPetraglia
October2011 MarianeRemboldPetraglia
MariaAugustaSoaresMachado
Acknowledgements
We would like to thank Dr. Leontina Di Cecco (Publishing Editor), Mr. Hol-
ger Scha¨pe (Engineering Editorial) and all staff of Springer-Verlag for their kind
support.

Contents
PartI Fundamentals
1 Introduction.................................................. 3
1.1 WhytoOptimize?......................................... 3
1.2 KindsofOptimizationProblems............................. 5
1.3 HowtoOptimize?......................................... 6
References.................................................... 10
2 GlobalOptimizationandItsApplications........................ 11
2.1 Introduction .............................................. 11
2.2 StochasticorDeterministic? ................................ 12
2.3 ConsiderationsaboutGeneralGlobalOptimizationTasks ........ 13
2.4 SomePopularApproachesandFinalComments................ 18
References.................................................... 20
3 MetaheuristicMethods ........................................ 21
3.1 Introduction .............................................. 21
3.2 GeneticAlgorithms........................................ 23
3.3 ParticleSwarmOptimization................................ 24
3.4 DifferentialEvolution...................................... 25
3.5 Cross-EntropyMethod ..................................... 26
3.6 SimulatedAnnealing....................................... 27
References.................................................... 30
PartII ASA,FuzzyASAandTheirCharacteristics
4 AdaptiveSimulatedAnnealing.................................. 33
4.1 Introduction .............................................. 33
4.1.1 LICENSEandContributions ......................... 34
4.1.2 OrganizationofChapter ............................ 34
4.2 TheoreticalFoundationsofAdaptiveSimulatedAnnealing(ASA). 35