Rule-Based Evolutionary Online Learning Systems: A Principled Approach to LCS Analysis and Design PDF
Preview Rule-Based Evolutionary Online Learning Systems: A Principled Approach to LCS Analysis and Design
MartinV.Butz Rule-BasedEvolutionaryOnlineLearningSystems StudiesinFuzzinessandSoftComputing,Volume 191 Editor-in-chief Prof.JanuszKacprzyk SystemsResearchInstitute PolishAcademyofSciences ul.Newelska6 01-447Warsaw Poland E-mail:[email protected] Furthervolumesofthisseries Vol.183.LarryBull,TimKovacs(Eds.) canbefoundonourhomepage: FoundationsofLearningClassifierSystems, 2005 springeronline.com ISBN3-540-25073-5 Vol.184.BarryG.Silverman,AshleshaJain, Vol.175.AnnaMariaGil-Lafuente AjitaIchalkaranje,LakhmiC.Jain(Eds.) FuzzyLogicinFinancialAnalysis,2005 IntelligentParadigmsforHealthcare ISBN3-540-23213-3 Enterprises,2005 Vol.176.UdoSeiffert,LakhmiC.Jain, ISBN3-540-22903-5 PatricSchweizer(Eds.) Vol.185.SpirosSirmakessis(Ed.) BioinformaticsUsingComputational KnowledgeMining,2005 IntelligenceParadigms,2005 ISBN3-540-25070-0 ISBN3-540-22901-9 Vol.186.RadimBeˇlohlávek,Vilém Vol.177.LipoWang(Ed.) Vychodil SupportVectorMachines:Theoryand FuzzyEquationalLogic,2005 Applications,2005 ISBN3-540-26254-7 ISBN3-540-24388-7 Vol.187.ZhongLi,WolfgangA.Halang, Vol.178.ClaudeGhaoui,MituJain, GuanrongChen(Eds.) VivekBannore,LakhmiC.Jain(Eds.) IntegrationofFuzzyLogicandChaos Knowledge-BasedVirtualEducation,2005 Theory,2005 ISBN3-540-25045-X ISBN3-540-26899-5 Vol.179.MirceaNegoita, Vol.188.JamesJ.Buckley,LeonardJ. BerndReusch(Eds.) Jowers RealWorldApplicationsofComputational SimulatingContinuousFuzzySystems,2006 Intelligence,2005 ISBN3-540-28455-9 ISBN3-540-25006-9 Vol.189.Hans-WalterBandemer Vol.180.WesleyChu, MathematicsofUncertainty,2006 TsauYoungLin(Eds.) ISBN3-540-28457-5 FoundationsandAdvancesinDataMining, 2005 Vol.190.Ying-pingChen ISBN3-540-25057-3 ExtendingtheScalabilityofLinkage LearningGeneticAlgorithms,2006 Vol.181.NadiaNedjah, ISBN3-540-28459-1 LuizadeMacedoMourelle FuzzySystemsEngineering,2005 Vol.191.MartinV.Butz ISBN3-540-25322-X Rule-BasedEvolutionaryOnlineLearning Systems,2006 Vol.182.JohnN.Mordeson, ISBN3-540-25379-3 KiranR.Bhutani,AzrielRosenfeld FuzzyGroupTheory,2005 ISBN3-540-25072-7 Martin V. Butz Rule-Based Evolutionary Online Learning Systems A Principled Approach to LCS Analysis and Design ABC Dr.MartinV.Butz DepartmentofCognitivePsychology UniversityofWürzburg Röntgenring11 97070,Würzburg Germany E-mail:[email protected] LibraryofCongressControlNumber:2005932567 ISSNprintedition:1434-9922 ISSNelectronicedition:1860-0808 ISBN-10 3-540-25379-3SpringerBerlinHeidelbergNewYork ISBN-13 978-3-540-25379-2SpringerBerlinHeidelbergNewYork Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis concerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broadcasting, reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationofthispublication orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9, 1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violationsare liableforprosecutionundertheGermanCopyrightLaw. SpringerisapartofSpringerScience+BusinessMedia springeronline.com (cid:1)c Springer-VerlagBerlinHeidelberg2006 PrintedinTheNetherlands Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnotimply, evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotectivelaws andregulationsandthereforefreeforgeneraluse. Typesetting:bytheauthorandTechBooksusingaSpringerLATEXmacropackage Printedonacid-freepaper SPIN:11370642 89/TechBooks 543210 To my parents Susanne and Teja and my brother Christoph Preface Rule-basedevolutionaryonlinelearningsystems,oftenreferredtoasMichigan- style learning classifier systems (LCSs), were proposed nearly thirty years ago (Holland, 1976; Holland, 1977) originally calling them cognitive systems. LCSs combine the strength of reinforcement learning with the generaliza- tion capabilities of genetic algorithms promising a flexible, online generaliz- ing,solelyreinforcementdependentlearningsystem.However,despiteseveral initialsuccessfulapplicationsofLCSsandtheirinterestingrelationswithani- mallearningandcognition,understandingofthesystemsremainedsomewhat obscured. Questions concerning learning complexity or convergence remained unanswered.Performanceindifferentproblemtypes,problemstructures,con- ceptspaces,andhypothesisspacesstayednearlyunpredictable.Thisbookhas the following three major objectives: (1) to establish a facetwise theory ap- proachforLCSsthatpromotessystemanalysis,understanding,anddesign;(2) toanalyze,evaluate,andenhancetheXCSclassifiersystem(Wilson,1995)by themeansofthefacetwiseapproachestablishingafundamentalXCSlearning theory; (3) to identify both the major advantages of an LCS-based learning approachaswellasthemostpromisingpotentialapplicationareas.Achieving these three objectives leads to a rigorous understanding of LCS functioning that enables the successful application of LCSs to diverse problem types and problem domains. The quantitative analysis of XCS shows that the interac- tive, evolutionary-based online learning mechanism works machine learning competitivelyyieldingalow-orderpolynomiallearningcomplexity.Moreover, the facetwise analysis approach facilitates the successful design of more ad- vanced LCSs including Holland’s originally envisioned cognitive systems. Martin V. Butz Foreword I In 1979, in an MIT library looking for inspiration, I encountered John Hol- land’s(1978)bookchapter,writtenwithJudithReitman,containingthefirst implementationofwhatwenowcall“learningclassifiersystems”(LCS).That led me to the stacks of MIT’s Science Library and Holland’s (1975) magnif- icent book Adaptation in Natural and Artificial Systems (which had not yet been taken out!). Devouring the first, and sampling as deeply as I could the second, I became hooked. How can a system improve in on-going interaction with its environment? How can it create new structures that permit better adaptation?Howmightprogrammingjustspecifywhattodo,withoutsaying how?Hollandstatedandaimedhisworkatexactlysuchquestions,which,for understanding intelligence, I thought were exactly the right ones. From that point Holland’s classifier systems became my main intellectual passion. As is admirably related in Martin V. Butz’s Introduction, the early path followed by the small band of LCS researchers had its ups and downs but, fortunately,persistencepaidoffandthefieldisnowinrapiddevelopment,with over 700 published papers, several active research centers, and a prominent place in the major evolutionary computation conferences and journals. Dr. Butz’s book is in part a history and celebration of this progress, but much more importantly it is the first in-depth treatise available on learning classifier systems. He explains the relationships between LCS and its dual contexts of evolutionary computation and reinforcement learning . He dis- cusses the basic structure of LCS systems and how David Goldberg’s (2002) “facetwise” analysis can be applied to them. In the main part of the book, Butz presents in broad themes and solid detail the basic theory—much of which he originated—of the currently most-employed learning classifier sys- tem, XCS. At the same time, he provides a balanced representation of other LCS models and theories. InlaterchaptersDr.Butzappliesthetheory—whichisderivedinabinary setting—to multiple-valued problems, data mining, and reinforcement learn- ing.Hediscussesparameterandsub-systemarchitectureselection.Finally,he presents his thinking about LCS extensions to systems with greater cognitive X Foreword power that could include hierarchical structures and anticipatory behavior, among other properties. I am truly excited by Martin Butz’s great contribution and believe that thisbookwillbeturnedtoagainandagainforconcepts,theory,andnotleast, examples of high-quality research. Prediction Dynamics, Concord, MA Stewart W. Wilson [email protected] Foreword II Toward the end of 1998, I started to correspond with a German graduate student named Martin Butz who wanted to visit my lab at Illinois. He was doing interesting work in so-called anticipatory classifier systems, and my early-1980s flirtation with learning classifier systems (LCSs) was ripe for res- urrection. One thing led to another and Martin visited the Illinois Genetic AlgorithmsLaboratory,andIstillrememberourfirstmeeting.Heshowedme some cool work, and I asked him some interesting questions, and the result was a spark, a flame, and a delightful collaboration that is still ongoing. Martin’ssecondbook,RuleBasedEvolutionaryOnlineLearningSystems, is a (building?) blockbuster that should enrich the field, prolong the ongoing LCS renaissance, and inform the study and practice of genetics-based and other forms of machine learning. Monographs are like Isaiah Berlin’s famous hedgehog: they are supposed to do one big thing, and many are lucky if they do that, but Butz has succeeded in doing two big things. First, he has publishedaneffectivedesigntheoryoflearningclassifiersystems,andsecond, he has clearly demonstrated the value of linkage learning in an LCS context. The first of these things is important, because much of the theorizing about classifier systems has been qualitative and has avoided tough questions of complexityandconvergence.Butzlooksthesetwinantagonistsintheeyesand with bounding calculations useful in both analysis and design. The linkage learning work clearly shows the importance of building-block processing in genetics-basedmachinelearninginamannerthatisunassailable,andinaway that appears to lead to regular solutions of extraordinarily difficult problems on a regular basis. These two things would be enough, but like Babe Ruth pointing at the outfield,ButzfinishesupwithacallformorecognitiveLCSsandagameplan forgettingthere.Hisplansarebold;theyarealmostbrash,butIwouldn’tbet against Butz, his energy, his perseverance, or his intelligence. Martin’s book is a tale that is as yet unfinished, but I urge you to buy this book, read it, and stay tuned over the next ten years to see how it all turns out. University of Illinois at Urbana-Champaign, IL David E. Goldberg [email protected] Acknowledgments I am grateful to many people for supporting me not only intellectually but also mentally and socially in my work and life besides work. These acknowl- edgments can only give a glimpse on how much I benefited and learned from all my friends, family, and colleagues. Thank you so much to all of you. I am in debt to my previous adviser David E. Goldberg, who supplied me with invaluable advice and guidance throughout my time at the University ofIllinoisatUrbana-Champaign(UIUC)concerningmyresearch,writing,or- ganization, and life. He supported me in all my plans and encouraged me to pursue thoughts and ideas that may have partially sounded very unconven- tional at first. Thank you so much for trusting and believing in me. I am also very grateful to my other committee members during my PhD studies including Gerald DeJong, Sylvian Ray, and Dan Roth. Thank you for all the useful comments, suggestions, and additional work. I am also grate- ful to many other faculty members at UIUC including Gul Agha, Thomas Anastasio, Kathryn Bock, Gary Dell, Steven LaValle, and Edward Reingold. Thank you also for the support from the automated learning group at the national center for supercomputing applications (NCSA) and Michael Welge and Loretta Auvil, in particular. In the mean time, I am very grateful to Stewart W. Wilson, who was always available for additional advice, support, and help for XCS and also my writing. My visit to Boston in April 2000 was more than inspiring and certainlyshapedalargepartofthebasisofthisbookandthethoughtsbeyond it including the first complexity derivations for XCS. I am also very grateful for all the support I received during my studies from Joachim Hoffmann, head of the Department of Cognitive Psychology at the Universita¨t Wu¨rzburg. His cognitive perspective of things is always stimulating and many of the parts of this book related to cognition were shaped by the numerous discussions we had during my time in Germany. I am also grateful to Wolfgang Stolzmann, who made my involvement into learning classifier systems and anticipatory systems possible in the first
The list of books you might like
