Table Of Content

Markus Brameier and Wolfgang Banzhaf Linear Genetic Programming Genetic and Evolutionary Computation Series Series Editors David E. Goldberg Consulting Editor IlliGAL, Dept. of General Engineering University of Illinois at Urbana-Champaign Urbana, IL 61801 USA Email: [email protected] John R. Koza Consulting Editor Medical Informatics Stanford University Stanford, CA 94305-5479 USA Email: [email protected] Selected titles from this series: Nikolay Y. Nikolaev, Hitoshi Iba Adaptive Learning of Polynomial Networks, 2006 ISBN 978-0-387-31239-2 Tetsuya Higuchi, Yong Liu, Xin Yao Evolvable Hardware, 2006 ISBN 978-0-387-24386-3 David E. Goldberg The Design of Innovation: Lessons from and for Competent Genetic Algorithms, 2002 ISBN 978-1-4020-7098-3 John R. Koza, Martin A. Keane, Matthew J. Streeter, William Mydlowec, Jessen Yu, Guido Lanza Genetic Programming IV: Routine Human-Computer Machine Intelligence ISBN: 978-1-4020-7446-2 (hardcover), 2003; ISBN: 978-0-387-25067-0 (softcover), 2005 Carlos A. Coello Coello, David A. Van Veldhuizen, Gary B. Lamont Evolutionary Algorithms for Solving Multi-Objective Problems, 2002 ISBN: 978-0-306-46762-2 Lee Spector Automatic Quantum Computer Programming: A Genetic Programming Approach ISBN: 978-1-4020-7894-1 (hardcover), 2004; ISBN 978-0-387-36496-4 (softcover), 2007 William B. Langdon Genetic Programming and Data Structures: Genetic Programming + Data Structures = Automatic Programming! 1998 ISBN: 978-0-7923-8135-8 For a complete listing of books in this series, go to http://www.springer.com Markus Brameier Wolfgang Banzhaf Linear Genetic Programming 1 3 Markus F. Brameier Bioinformatics Research Center (BiRC) University of Aarhus Denmark [email protected] Wolfgang Banzhaf Department of Computer Science Memorial University of Newfoundland St. John’s, NL Canada [email protected] Library of Congress Control Number: 2006920909 ISBN-10: 0-387-31029-0 e-ISBN-10: 0-387-31030-4 ISBN-13: 978-0387-31029-9 e-ISBN-13: 978-0387-31030-5 © 2007 by Springer Science+Business Media, LLC All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science + Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America 9 8 7 6 5 4 3 2 1 springer.com We dedicate this book to our beloved parents. Contents Preface xi About the Authors xv 1. INTRODUCTION 1 1.1 Evolutionary Algorithms 1 1.2 Genetic Programming 3 1.3 Linear Genetic Programming 6 1.4 Motivation 8 Part I Fundamental Analysis 2. BASIC CONCEPTS OF LINEAR GENETIC PROGRAMMING 13 2.1 Representation of Programs 13 2.2 Execution of Programs 25 2.3 Evolution of Programs 29 3. CHARACTERISTICS OF THE LINEAR REPRESENTATION 35 3.1 Effective Code and Noneffective Code 35 3.2 Structural Introns and Semantic Introns 37 3.3 Graph Interpretation 47 3.4 Analysis of Program Structure 56 3.5 Graph Evolution 60 3.6 Summary and Conclusion 61 viii Contents 4. A COMPARISON WITH NEURAL NETWORKS 63 4.1 Medical Data Mining 63 4.2 Benchmark Data sets 64 4.3 Experimental Setup 65 4.4 Experiments and Comparison 69 4.5 Summary and Conclusion 74 Part II Method Design 5. SEGMENT VARIATIONS 77 5.1 Variation Effects 78 5.2 Effective Variation and Evaluation 79 5.3 Variation Step Size 80 5.4 Causality 82 5.5 Selection of Variation Points 86 5.6 Characteristics of Variation Operators 87 5.7 Segment Variation Operators 89 5.8 Experimental Setup 99 5.9 Experiments 102 5.10 Summary and Conclusion 118 6. INSTRUCTION MUTATIONS 119 6.1 Minimum Mutation Step Size 119 6.2 Instruction Mutation Operators 121 6.3 Experimental Setup 129 6.4 Experiments 131 6.5 Summary and Conclusion 148 7. ANALYSIS OF CONTROL PARAMETERS 149 7.1 Number of Registers 149 7.2 Number of Output Registers 156 7.3 Rate of Constants 157 Contents ix 7.4 Population Size 159 7.5 Maximum Program Length 162 7.6 Initialization of Linear Programs 164 7.7 Constant Program Length 169 7.8 Summary and Conclusion 170 8. A COMPARISON WITH TREE-BASED GP 173 8.1 Tree-Based Genetic Programming 173 8.2 Benchmark Problems 177 8.3 Experimental Setup 181 8.4 Experiments and Comparison 185 8.5 Discussion 190 8.6 Summary and Conclusion 191 Part III Advanced Techniques and Phenomena 9. CONTROL OF DIVERSITY AND VARIATION STEP SIZE 195 9.1 Introduction 195 9.2 Structural Program Distance 197 9.3 Semantic Program Distance 200 9.4 Control of Diversity 201 9.5 Control of Variation Step Size 203 9.6 Experimental Setup 205 9.7 Experiments 206 9.8 Alternative Selection Criteria 222 9.9 Summary and Conclusion 223 10. CODE GROWTH AND NEUTRAL VARIATIONS 225 10.1 Code Growth in GP 226 10.2 Proposed Causes of Code Growth 227 10.3 Influence of Variation Step Size 229 10.4 Neutral Variations 230 x Contents 10.5 Conditional Reproduction and Variation 232 10.6 Experimental Setup 233 10.7 Experiments 233 10.8 Control of Code Growth 249 10.9 Summary and Conclusion 259 11. EVOLUTION OF PROGRAM TEAMS 261 11.1 Introduction 261 11.2 Team Evolution 262 11.3 Combination of Multiple Predictors 265 11.4 Experimental Setup 273 11.5 Experiments 276 11.6 Combination of Multiple Program Outputs 286 11.7 Summary and Conclusion 287 Epilogue 289 References 291 Index 303 Preface This book is about linear genetic programming (LGP), a variant of GP that evolves computer programs as sequences of instructions of an imperative programming language. It is a comprehensive text with a strong experimental basis and an in-depth focus on structural aspects of the linear program representation. The three major objectives of this book are: (cid:2) To discuss linear genetic programming in a broader context and to contrast it with tree-based genetic programming. (cid:2) To develop advanced methods and efficient genetic operators for the imperative representation to produce both better and shorter program solutions. (cid:2) To give a better understanding for the intricate effects of operators on evolutionary processes and emergent phenomena in linear GP. PartIofthebookisdedicatedtolayingafoundationforbasicunderstand- ing as well as to providing methods for analysis. The first two chapters give an introduction to evolutionary computation, genetic programming, and linear GP. Chapter 3 presents efficient algorithms for analyzing the imperative and functional structure of linear genetic programs during runtime. The spe- cial program representation used in this book can be transformed into directed acyclic graphs (DAGs). So-called structurally noneffective code can be identified that is disconnected from the effective data flow and independent of program semantics. Other important parameters of linear programs include the number of effective registers at a certain program