Theoretical Aspects of Evolutionary Computing PDF
Preview Theoretical Aspects of Evolutionary Computing
Natural Computing Series Series Editors: G.Rozenberg (Managing) Th. Bäck A.E.Eiben J.N.Kok H.P. Spaink LeidenCenterforNaturalComputing Advisory Board: S.Amari G.Brassard M.Conrad K.A. De Iong C.C.A.M. Gielen T.Head L.Kari L.Landweber T.Martinetz Z.Michalewicz M.C.Mozer E.Oja J.Reif H.Rubin A.Salomaa M.Schoenauer H.-P.Schwefel D.Whitley E.Winfree J.M.Zurada Springer-Verlag Berlin Heidelberg GmbH Leila Kallel Bart Naudts -Alex Rogers (Eds.) > Theoretical Aspects of Evolutionary Computing With 129Figures and 15Tables , Springer Editors SeriesEditors Dr.LeilaKallel G.Rozenberg(ManagingEditor) EcolePolytechnique Th.Bäck,A.E.Eiben,J.N.Kok, CMAP H.P.Spaink 91128PalaiseauCedex,France LeidenCenterforNaturalComputing [email protected] Leiden University Dr.BartNaudts NielsBohrweg1 UniversiteitAntwerpen(RUCA) 2333CALeiden DepartementWiskunde&Informatica The Netherlands Groenenborgerlaan171 [email protected] 2020Antwerpen,Belgium [email protected] AlexRogers UniversityofSouthampton DepartmentofElectronicsandComputerScience Highfield S017 lBJSouthampton,UK [email protected] LibraryofCongressCataloging-in-PublicationData Theoreticalaspectsofevolutionarycomputing/LeilaKallel,BartNaudts,AlexRogers(eds.). p.cm.- (Naturalcomputingseries) Basedonthelecturesand workshopcontributionsofthesecondEvoNetSummer SchoolonTheoreticalAspectsofEvolutionaryComputingheldattheMidde!heim campusoftheUniversityofAntwerp,Be!giumduringthefirstweekofSeptember1999. Includesbibliographicalreferencesand index. 1.Evolutionaryprogramming(Computerscience) I.Kallel,Leila,1972-11.Naudts, Bart,1973- III.Rogers,Alex,1970- IV.EvoNetSummerSchoolonTheoreticalAspects ofEvolutionaryComputing(2nd: 1999:UniversityofAntwerp,Belgium)V.Series. QA76.618.T472000 006.3-dc21 00-061910 ACM ComputingC1assification (1998): F.2.2,1.2.8,1.2.6,G.3, G.1.6, J.3,J.2 ISBN978-3-642-08676-2 ISBN978-3-662-04448-3(eBook) DOI 10.1007/978-3-662-04448-3 This workissubjectto copyright.Allrightsare reserved,whetherthe whole or partofthe materialisconcerned,specificallythe rightsoftranslation,reprinting,reuse ofillustrations, recitation,broadcasting,reproductiononmicrofilmorinanyotherway,andstorageindata banks. Duplication of this publication or parts thereof is perrnitted only under the provisionsofthe German CopyrightLawofSeptember9,1965,in its currentversion,and permissionfor use must alwaysbe obtainedfrom Springer-Verlag.Violationsare liable for prosecutionundertheGermanCopyrightLaw. Springer-VerlagBerlin HeidelbergNewYork, amemberofBertelsmannSpringerScience+BusinessMediaGmbH hup://www.springer.de ©Springer-VerlagBerlinHeide!berg2001 OriginallypublishedbySpringer-VerlagBerlinHeidelbergNewYorkin2001. SoftcoverreprintofthehardcoverIstedition200I The use ofgeneraldescriptive narnes, trademarks,etc.in this publication does not imply, even in the absenceofaspecific staternent,that such names are exemptfrom the relevant protectivelawsand regulationsand thereforefreeforgeneraluse. CoverDesign:KünkelLopka,Heidelberg Typesetting:Camerareadybytheeditors Printedonacid-freepaper SPIN10765204 45/3142SR- 543210 Preface During the first week of September 1999, the Second EvoNet Summer School on TheoreticalAspects of Evolutionary Computing was held at the Middelheimcam pus of the University of Antwerp, Belgium. Originally intended as a small get together of PhD students interested in the theory of evolutionary computing, the summerschool grew to become a successful combination of a four-day workshop withovertwentyresearchersinthefieldandatwo-day lectureseriesopentoawider audience. This book is based on the lectures and workshop contributions of this summer school. Its firstpart consistsoftutorial papers which introducethe readertoanum ber of importantdirections in the theory of evolutionary computing.The tutorials areatgraduate levelandassume onlyabasicbackgroundinmathematicsandcom puterscience. No priorknowledge ofevolutionarycomputing or its theory is nec essary. The second part of the book consists of technical papers, selected from the workshop contributions. A number of them build on the material of the tutorials, exploringthe theory to research level. Other technical papers may require a visit to the library. Thetheoryofevolutionisatthecrossroadsofmathernatics,computerscience,statis tics, and the theoretical sides of physics,chemistry, and biology.There isoften too little interaction between these disciplines. One of the summer school's goals was to get researchers from many disciplines together. As a result, this book contains papers from researchers with a backgroundincomplexity theory, neural networks, probability theory, population genetics, statistical physics, and mathematics. Only a fraction ofthe authors grew up in the evolutionarycomputing community itself. This isreflected inthebook, whichpresentsariehvarietyofapproachestothestudy ofthe behaviorofevolutionaryalgorithms. OverviewoftheThtorials The tutorial part ofthe book starts with ageneralintroductiontoevolutionarycom puting, fromthepointofviewofanexperiencedengineeringresearcher,A.J.Keane, VI Preface who usesevolutionaryalgorithmsasone ofhismain toolsinpractical optimization. Thisfirst tutorial hasbeen written incooperationwith A.Rogers. Beforeplunginginto the theory, we present asecond tutorialon evolutionaryalgo rithms,written by A. E. Eiben. He discusses the use ofevolutionaryalgorithmsfor constrainedsearchproblems. A first course on the microscopic approach to the dynamics ofthe simple genetic algorithm, as started by M. D. Vose in the early 1990s, is given by J. E. Rowe. Filling in the actual numbers for a concrete example, but also commenting on the latestdevelopment,thetutorial isallthereaderneeds togetstartedwith thisexciting mathematical framework. The statistical physics approach to the dynamics of evolutionary algorithms is a macroscopic one: instead ofmodeling the exact dynamics, only a few statistics of the dynamics are computed. A. Prügel-Bennett and J. L. Shapiro have written two complementarytutorialson this lineofresearch. Followingthe line ofmacroscopicdescriptionsisthe tutorial onevolution strategies by H.-G.Beyerand D.V.Arnold. Evolutionstrategiesoperateon real spaces.Their dynamicson a numberoffitnessmodelshave now been fully analyzed. Apartfrom beingsuccessful inpracticalapplications,optimization algorithmsbased onamodel ofthe fitnessdistributioncan beused toapproximategeneticalgorithms. H. Mühlenbeinand T.Mahnig introducethis approach tostudyingthe dynamicsof genetic algorithms,also showingthe linkwithpopulationgeneticsand learningwith Bayesian networks. Closing the gap betweentheoryand practiceisthe tutorial byL. Kallel,B.Naudts, and C. R. Reeves on propertiesof fitness landscapes and their impact on problem difficulty. The tutorialexploresissuessuchasWalsh coefficients,epistasis,correla tionsinthe fitnesslandscape,basinsofattraction,and interactionstructure. Overviewofthe TechnicalPapers In the first technical paper, A. Rogersand A. Prügel-Bennettbuild on the tutorials by Prügel-Bennettand Shapiro.They analyze the dynamics of a genetic algorithm with rankingselection,uniformcrossover,and mutationonthe 'basinwithabarrier' model, which contains one feature commonly found in hard search problems. A relationship betweenthe optimal mutation rate and population size isfound, which isindependentofthe problemsize. M. Dates and coworkersalsoexaminethe relationshipbetweenpopulationsize and mutation rate.Theirexhaustiveexperimentalwork onanumberofsearchproblems ispresented inthe secondtechnical paper. The third and last paper which deals with search in time-static problems is by D. V.Arnold,who presents the state ofthe art ofevolution strategies on noisy fitness models.Beyer and Arnold'stutorial on evolutionstrategiesserves asreference ma terial. Preface VII The quasispecies model from theoretical biology is studied by J. E. Rowe, and also by C. Ronnewinkel,C. O. Wilke, and T. Martinetz. Both use the microscopic model from Rowe'stutorialtostudy the quasispeciesadaptabilitytofitness function changes by computing errorthresholdsand optimal mutation rates.However,their approach to time-dependence is different. The former identifies cyclic attractors, whereas the latteraccumulate thetime-dependencyofthechangecycle. A. Bemy demonstrates the link between statistical leaming theory and combina torial optimization. Rather than searching in the space of bit strings, he demon strates how to search, using gradient techniques, in the space ofdistributions over bit strings. Thegeneticdriftcaused bymulti-parentscanningcrossover isrigorouslycalculated by C. A. Schippers. He finds that occurrence-based scanning produces a stronger than usual, self-amplifyinggenetic drift. Reference material for the next six papers is the tutorial by KalleI, Naudts, and Reeves. Instead of studying the dynamics of evolutionary algorithms, they focus onthe representationofsearch problemsand the fitness landscapes induced by the algorithm. When choosing representation and genetic operators, many implicit assumptions about the epistatic structure ofthe fitness function to be optimized are made. For example,one-pointcrossoverassumes that adjacent bit positions are highly corre lated. F.Burkowskiproposesnullifyingtheseassumptions,andlearningtheepistatic structure on-line. J. Garnier and L. Kallel develop statistical techniques to estimate the numberand sizes ofthe basinsofattractioninafitnesslandscape. Can fitness functions be c1assified according to the difficulty they present to a ge neticalgorithm?T.Jansen commentsonthisissue and detailssomeofthe flawsthat a number ofmeasures,introducedas predictorsofproblem difficulty,sufferfrom. The subject ofsymmetry in the representation ofthe search space has been stud ied in three independentcontributions.First, there is the paper of P. Stagge and C. Igel,which dealswith redundancycausedbyisomorphicstructures inthecontextof topologyoptimizationofneuralnetworks.Inasecondcontribution,A.Marinointro ducesspecificgeneticoperatorsforthegraphcoloringproblem,inorderto tosearch the highly symmetric solution space more efficiently. Finally, C. Van Hoyweghen introducesatechniquetodetectspin-flipsymmetry inarbitrarysearchspaces. The last contribution to this book, and also the largest in size, is by P. DeI Moral and L. Miclo. They first proposea model for GAs which generalizes Vose's model (introduced in the tutorial by Rowe) to continuous search spaces, inhomogeneous fitnessenvironments, and inhomogeneousmutationoperators.In this model a pop ulation is seen as a distribution (or measure) on the search space, and the genetic algorithmasameasure-valueddynamicalsystem.Then,theystudy theconvergence propertiesofthefinitepopulation processwhenthepopulationsize tendstoinfinity, andprovethat it convergesuniformlywithrespect to timeto the infinite population VIII Preface process.Finally, they show how their model of GAs can be used in the context of nonlinearfiltering. They prove the efficiency of finite population GAs to approxi mateanobserved signal inthepresence of noise. Acknowledgments Wewould like to thank all the participantsof the summer school for attending and sending invaluable contributionsafterwards.This book istheirwork.Wewerealso pleased with the fact that so many senior researchers were willing to write a 20+ page tutorial on theirsubject. Finding sponsorshiptoorganize asummerschool on atheoretical subject wasalot easier thanwethought. WewouldliketothanktheNetwork ofExcellenceinEvolu tionary Computing(EvoNet), theDoctoral StudyProgrammeScience-Pharmacyof the University of Antwerpen (DOCOP), the Foundation forEuropean Evolutionary Computation Conferences (FEECC), and the Leiden Institute for Advanced Com puter Science (LIACS) for their helpful cooperation.The editing of this book was done whileB.Naudts wasfundedasaPostdoctoral FellowoftheFundforScientific Research - Flanders, Belgium (FW.O.). Butaboveall,wearegreatly indebted tooursummerschoolco-organizers:Jennifer WilliesofEvoNet,JanovanHemertoftheLIACS,andIvesLandrieu, LukSchoofs, andClarissa VanHoyweghen oftheUniversity ofAntwerpen. Theydidagreatjob! Paris,Antwerpen,andSouthampton, L.Kallel Spring 2001 B. Naudts A.Rogers Table ofContents PartI:Tutorials IntroductiontoEvolutionaryComputing inDesign Search andOptimisation AoJo Keane 00 0 0 0000000.0 0 000 .00 0 00 0 0 0 0 00 0 0 0 0 000000 0 0 0 0 0 0 0 000 0 00 0 0 0 0 0 EvolutionaryAlgorithmsandConstraintSatisfaction: Definitions,Survey,Methodology,andResearch Directions AoEoEiben 13 000000 0 0 0 0000000 0 0 0 0 0 0 0 0 0 00 0 0 0 00 0 000 0 0 0 00000 0 0 0 0000 0 0 0 0 0 TheDynamicalSystemsModeloftheSimpleGeneticAlgorithm I, EoRowe 31 0 0 0 0 0 0 000 0 0 0 0 0 . 0 0 .0 0 0 . 0 0 . 0 0 0 . 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ModellingGeneticAlgorithmDynamics AoPrügel-BennettandA. Rogers 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 StatisticalMechanicsTheoryofGeneticAlgorithms Jo L. Shapiro 87 0 0 0 0 0 0 0 0 0 • 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ••• • 0 0 0 0 0 • 0 TheoryofEvolutionStrategies- ATutorial H.-GoBeyerandD. V.Arnold ..... 109 0 0 0 •••• ••• • • 0 • • • 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 EvolutionaryAlgorithms:FromRecombinationtoSearch Distributions H. MühlenbeinandT. Mahnig ... 135 0 .. 0 0 0 0 0 0 0 . 0 . . ..0. 0 0 000 0 000000000 00 PropertiesofFitnessFunctionsandSearchLandscapes L. Kallei,B.Naudts, and C.R.Reeves 175 0 0 •••0 ••• 0 0 •• 0 0 0 0 • 0 0 •• •••• ••••• Part11: TechnicalPapers ASolvableModelofaHardOptimisationProblem A.RogersandAo Prügel-Bennett o. 207 0 • 0 0 0 0 • • • • ••••• ••• 0 0 0 0 0 0 0 • •••• • • • • X TableofContents BimodalPerformanceProfile ofEvolutionarySearehand the Effeets ofCrossover M. Oates, J.Smedley, D. Corne,andR.Loader 223 EvolutionStrategiesinNoisy Environments- ASurveyofExisting Work D. V.Arnold 239 Cyclic Attraetorsand QuasispeeiesAdaptability J.E.Rowe 251 Genetie AigorithmsinTime-DependentEnvironments C.Ronnewinkel,C.O. Wilke,andT. Martinet: 261 Statistieal MaehineLeamingand CombinatorialOptimization A.Berny 287 Multi-ParentSeanningCrossoverand GenetieDrift C.A.Schippers 307 HarmonieReeombinationfor EvolutionaryComputation F. J. Burkowski 331 How toDeteetall Maximaof aFunetion J.GarnierandL.Kallel 343 On ClassifieationsofFitness Functions T. Jansen 371 GenetieSearehon Highly SymmetrieSolutionSpaees:PreliminaryResults A.Marino 387 StructureOptimizationand Isomorphisms P. StaggeandC.Igel 409 Deteeting Spin-FlipSymmetryinOptimizationProblems C.VanHoyweghen 423 AsymptotieResults forGenetie Aigorithmswith Applieationsto NonlinearEstimation P. Del MoralandL.Miclo 439 Index 495
The list of books you might like
