Evolutionary Optimization Algorithms: Biologically-Inspired and Population-Based Approaches to Computer Intelligence

776 Pages·33.925 MB·English

by  Simon D.

EVOLUTIONARY
OPTIMIZATION
ALGORITHMS

EVOLUTIONARY
OPTIMIZATION
ALGORITHMS
Biologically-Inspired and
Population-Based Approaches to
Computer Intelligence
Dan Simon
Cleveland State University
WILEY
Copyright © 2013 by John Wiley & Sons, Inc. All rights reserved
Published by John Wiley & Sons, Inc., Hoboken, New Jersey
Published simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as
permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior
written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to
the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax
[phone], or on the web at www.copyright.com. Requests to the Publisher for permission should
be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ
07030, [phone], fax [phone], or online at http://www.wiley.com/go/permission.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in
preparing this book, they make no representations or warranties with respect to the accuracy or
completeness of the contents of this book and specifically disclaim any implied warranties of
merchantability or fitness for a particular purpose. No warranty may be created or extended by sales
representatives or written sales materials. The advice and strategies contained herein may not be
suitable for your situation. You should consult with a professional where appropriate. Neither the
publisher nor author shall be liable for any loss of profit or any other commercial damages, including
but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our
Customer Care Department within the United States at [phone], outside the United States at
[phone] or fax [phone].
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may
not be available in electronic formats. For more information about Wiley products, visit our web site at
www.wiley.com.
Library of Congress Cataloging-in-Publication Data is available.
ISBN 978-0-470-93741-9
Printed in the United States of America.
10 9 8 7 6 5 4 3 21
SHORT TABLE OF CONTENTS
Part I: Introduction to Evolutionary Optimization
1 Introduction 1
2 Optimization 11
Part II: Classic Evolutionary Algorithms
3 Genetic Algorithms 35
4 Mathematical Models of Genetic Algorithms 63
5 Evolutionary Programming 95
6 Evolution Strategies 117
7 Genetic Programming 141
8 Evolutionary Algorithm Variations 179
Part III: More Recent Evolutionary Algorithms
9 Simulated Annealing 223
10 Ant Colony Optimization 241
11 Particle Swarm Optimization 265
12 Differential Evolution 293
13 Estimation of Distribution Algorithms 313
14 Biogeography-Based Optimization 351
15 Cultural Algorithms 377
16 Opposition-Based Learning 397
17 Other Evolutionary Algorithms 421
Part IV: Special Types of Optimization Problems
18 Combinatorial Optimization 449
19 Constrained Optimization 481
20 Multi-Objective Optimization 517
21 Expensive, Noisy, and Dynamic Fitness Functions 563
Appendices
A Some Practical Advice 607
B The No Free Lunch Theorem and Performance Testing 613
C Benchmark Optimization Functions 641

DETAILED TABLE OF CONTENTS
Acknowledgments xxi
Acronyms xxiii
List of Algorithms xxvii
PART I INTRODUCTION TO EVOLUTIONARY OPTIMIZATION
1 Introduction 1
1.1 Terminology 2
1.2 Why Another Book on Evolutionary Algorithms? 4
1.3 Prerequisites 5
1.4 Homework Problems 5
1.5 Notation 6
1.6 Outline of the Book 7
1.7 A Course Based on This Book 8
2 Optimization 11
2.1 Unconstrained Optimization 12
2.2 Constrained Optimization 15
2.3 Multi-Objective Optimization 16
2.4 Multimodal Optimization 19
2.5 Combinatorial Optimization 20
vii
VÜi DETAILED TABLE OF CONTENTS
2.6 Hill Climbing 21
2.6.1 Biased Optimization Algorithms 25
2.6.2 The Importance of Monte Carlo Simulations 26
2.7 Intelligence 26
2.7.1 Adaptation 26
2.7.2 Randomness 27
2.7.3 Communication 27
2.7.4 Feedback 28
2.7.5 Exploration and Exploitation 28
2.8 Conclusion 29
Problems 30
PART II CLASSIC EVOLUTIONARY ALGORITHMS
Genetic Algorithms 35
3.1 The History of Genetics 36
3.1.1 Charles Darwin 36
3.1.2 Gregor Mendel 38
3.2 The Science of Genetics 39
3.3 The History of Genetic Algorithms 41
3.4 A Simple Binary Genetic Algorithm 44
3.4.1 A Genetic Algorithm for Robot Design 44
3.4.2 Selection and Crossover 45
3.4.3 Mutation 49
3.4.4 GA Summary 49
3.4.5 G A Tuning Parameters and Examples 49
3.5 A Simple Continuous Genetic Algorithm 55
3.6 Conclusion 59
Problems 60
Mathematical Models of Genetic Algorithms 63
4.1 Schema Theory 64
4.2 Markov Chains 68
4.3 Markov Model Notation for Evolutionary Algorithms 73
4.4 Markov Models of Genetic Algorithms 76
4.4.1 Selection 76
4.4.2 Mutation 77
4.4.3 Crossover 78
4.5 Dynamic System Models of Genetic Algorithms 82
4.5.1 Selection 82
4.5.2 Mutation 85
4.5.3 Crossover 87