Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation

397 Pages·2002·12.46 MB·English

by  J. A. Lozano (auth.), Pedro Larrañaga, Jose A. Lozano (eds.)

ESTIMATION OF
DISTRIBUTION ALGORITHMS
A New Tool for
Evolutionary Computation
Genetic Algorithms and
Evolutionary Computation
Consulting Editor, David E. Goldberg
Additional titles in the series:
Efficient and Accurate Parallel Genetic Algorithms, Erick Cantu-Paz
ISBN: 0-7923-7466-5
OmeGA: A Competent Genetic Algorithm for Solving Permutation and
Scheduling Problems, Dimitri Knjazew ISBN: 0-7923-7460-6
GE.f'JI\GENAG Il:
GENAGENAGENA
Genetic Algorithms and
Evolutionary Computation
http://www.wkap.nllseries.htmIGENA
ESTIMATION OF
DISTRIBUTION ALGORITHMS
A New Tool for
Evolutionary Computation
edited by
Pedro Larraiiaga
Jose A. Lozano
University of the Basque Country. Spain
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
ISBN 978-1-46 13-5604-2 ISBN 978-1-4615-1539-5 (eBook)
DOI 10.1007/978-1-4615-1539-5
Library ofCongres5 Cataloging-in-Publication Data
A C.I.P. Catalogue record for this book is available
from the Library of Congress.
Copyright <fl 2002 Springer Sciencc+Business Media New York
Originally published by Kluwer Academic Publisher.; in 2002
Softcover reprint ofthe hardcover lst edition 2002
AII rights reserved. No part of this publication may be reproduced, stored in a retrieval
systcm ar transmitted in any fonn ar by any means, mechanical, photo-copying, record ing, or
otherwise, without Ihe prior written pennission of the publisher.
Primed on acid-free paper.
Contents
List of Figures xi
List of Tables xvii
Preface xxiii
Contributing Authors xxvii
Series Foreword xxxiii
Part I Foundations
1
An Introduction to Evolutionary Algorithms 3
J.A. Lozano
1. Introduction 3
2. Genetic Algorithms 6
3. Evolution Strategies 14
4. Evolutionary Programming 19
5. Summary 20
2
An Introduction to Probabilistic Graphical Models 27
P. Larranaga
1. Introduction 27
2. Notation 28
3. Bayesian networks 31
4. Gaussian networks 44
5. Simulation 51
6. Summary 51
3
A Review on Estimation of Distribution Algorithms 57
P. Larranaga
vi Estimation of Distribution Algorithms
1. Introduction 57
2. EDA approaches to optimization 58
3. EDA approaches to combinatorial optimization 64
4. EDA approaches in continuous domains 80
5. Summary 90
4
Benefits of Data Clustering in Multimodal Function Optimization 101
via EDAs
J.M. Pena J.A. Lozano P. Larranaga
1. Introduction 101
2. Data clusterin(!; in evolutionary algorithms for multimodal nmc-
tion optimizatIOn 103
3. BNs and CGNs applied to data clustering 105
4. Further considerations about the EMDA 111
5. Experimental results 113
6. Conclusions 123
5
Parallel Estimation of Distribution Algorithms 129
J.A. Lozano R. Sagama P. Larranaga
1. Introduction 129
2. Sequential EBNABIc 130
3. Parallel EBNABIc 133
4. Numerical evaluation 138
5. Summary and conclusions 142
6
Mathematical Modeling of Discrete Estimation of Distribution Algorithms 147
C. Gonzalez J.A. Lozano P. Larranaga
1. Introduction 147
2. Using Markov chains to model EDAs 148
3. Dynamical systems in the modeling of some EDAs 155
4. Other approaches to modeling EDAs 159
5. Conclusions 161
Part II Optimization
7
An Empirical Comparison of Discrete Estimation of Distribution 167
Algorithms
R. Blanco J.A. Lozano
1. Introduction 167
2. Experimental framework 168
3. Sets of function test 169
4. Experimental results 173
5. Conclusions 177
8
Results in Function Optimization with EDAs in Continuous Domain 181
E. Bengoetxea T. Miquelez P. Larranaga J.A. Lozano
1. Introduction 181
2. Description of the optimization problems 182
Contents Vll
3. Algorithms to test 183
4. Brief description of the experiments 185
5. Conclusions 193
9
Solving the 0-1 Knapsack Problem with EDAs 195
R. Sagarna P. Larranaga
1. Introduction 195
2. The 0-1 knapsack problem 196
3. Binary representation 197
4. Representation based on permutations 202
5. Experimental results 203
6. Conclusions 208
10
Solving the Traveling Salesman Problem with ED As 211
V. Robles P. de Miguel P. Larranaga
1. Introduction 211
2. Review of algorithms for the TSP 212
3. A new approach: Solving the TSP with EDAs 217
4. Experimental results with EDAs 221
5. Conclusions 226
11
EDAs Applied to the Job Shop Scheduling Problem 231
J.A. Lozano A. Mendiburu
1. Introduction 231
2. EDAs in job shop scheduling problems 233
3. Hybridization 234
4. Experimental results 237
5. Conclusions 240
12
Solving Graph Matching with EDAs Using a Permutation-Based 243
Representation
E. Bengoetxea P. Larranaga 1. Bloch A. Perchant
1. Introduction 244
2. Graph matching as a combinatorial optimization problem with
constraints 245
3. Representing a matching as a permutation 247
4. Obtaining a permutation with discrete EDAs 254
5. Obtaining a permutation with continuous EDAs 256
6. Experimental results. The human brain example 257
7. Conclusions and further work 262
Part III Machine Learning
13
Feature Subset Selection by Estimation of Distribution Algorithms 269
1. Inza P. Larranaga B. Sierra
1. Introduction 269
2. Feature Subset Selection: Basic components 271
3. FSS by EDAs in small and medium scale domains 273
Vlll Estimation of Distribution Algorithms
4. FSS by EDAs in large scale domains 282
5. Conclusions and future work 289
14
Feature Weighting for Nearest Neighbor by EDAs 295
1. Inza P. Larranaga B. Sierra
1. Introduction 295
2. Related work 296
3. Learning weights by Bayesian and Gaussian networks 299
4. Experimental comparison 302
5. Summary and future work 308
15
Rule Induction by Estimation of Distribution Algorithms 313
B. Sierra E.A. Jimenez 1. Inza P. Larranaga J. Muruzabal
1. Introduction 313
2. A review of Classifier Systems 314
3. An approach to rule induction by means of EDAs 315
4. Empirical comparison 318
5. Conclusions and future work 320
16
Partial Abductive Inference in Ba~sian Networks: An Empirical 323
Comparison Between GAs and EDAs
L.M. de Campos J.A. Gamez P. Larranaga S. Moral T. Romero
1. Introduction 324
2. Query types in probabilistic expert systems 324
3. Solving queries 326
4. Tackling the problem with Genetic Algorithms 327
5. Tackling the problem with Estimation of Distribution Algo
rithms 330
6. Experimental evaluation 331
7. Concluding remarks 338
17
Comparing K-Means, GAs and EDAs in Partitional Clustering 343
J. Roure P. Larranaga R. Sanguesa
1. Introduction 343
2. Partitional clustering 345
3. Iterative algorithms 345
4. Genetic Algorithms in partitional clustering 347
5. Estimation of Distribution Algorithms in partitional clustering 351
6. Experimental results 352
7. Conclusions 355
18
Adjusting Weights in Artificial Neural 361
Networks using Evolutionary Algorithms
C. Cotta E. Alba R. Sagarna P. Larranaga
1. Introduction 362
2. An evolutionary approach to ANN training 363
3. Experimental results 368
4. Conclusions 373
Contents IX
Index 379
List of Figures
1.1 The modified Ackley function in two dimensions. 5
1.2 Pseudocode for the SGA. 7
1.3 One-point crossover. 8
1.4 The mutation operator. 8
1.5 An example of a four-point crossover operator. 10
1.6 Pseudocode for a general ES. 15
1.7 An example of recombination applied to four parents with
global discrete recombination in the search space and in-
termediary recombination in the strategy parameters. 17
2.1 Structure for a probabilistic graphical model defined over
=
X (XI, X2,X3, X4,XS,X6)· 29
2.2 Checking conditional independencies in a probabilistic gra-
phical model by means of the u-separation criterion for
undirected graphs. 30
2.3 Different degrees of complexity in the structure of proba-
bilistic graphical models. 31
2.4 Structure, local probabilities and resulting factorization for
a Bayesian network with four variables (Xl, X3 and X4
with two possible values, and X with three possible val-
2
ues). 33
2.5 Pseudocode for the PC algorithm. 36
2.6 The K2 algorithm. 41
2.7 The Chow and Liu MWST algorithm. 43
2.8 Pseudocode for the Probabilistic Logic Sampling method. 44
2.9 Structure, local densities, and resulting factorization for a
Gaussian network with four variables. 46
3.1 Illustration of the EDA approach to optimization. 63