ebook img

Iterative Computer Algorithms with Applications in Engineering: Solving Combinatorial Optimization Problems PDF

398 Pages·2000·2.079 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Iterative Computer Algorithms with Applications in Engineering: Solving Combinatorial Optimization Problems

Iterative Computer Algorithms with Applications in Engineering (Solving Combinatorial Optimization Problems ) Sadiq M. Sait and Habib Youssef 2 Sadiq M. Sait (S’87, M’90) obtained a Bachelor’s degree in Electronics from Bangalore University, India, in 1981, and Master’s and Ph.D degrees in Electrical Engineering from King Fahd University of Petroleum and Minerals (KFUPM),Dhahran,in1983and1987respectively. HeiscurrentlyaProfessor, intheDepartmentofComputerEngineeringofKFUPM.Saithasauthoredover 85 research papers in International Journals and Conferences. He is the co- author of the book ”VLSI Physical Design Automation: Theory and Practice”, published by McGraw-Hill Book Co., Europe, (also co-published by IEEE Press), January 1995. He has also contributed two chapters to a book entitled ‘Progress in VLSI Design’. He served on the editorial board of International Journal of Computer-Aided Design between 1988-1990. Currently he is the editor of ‘Arabian Journal for Science and Engineering’ for Computer Science & Engineering. His current areas of interest are in Digital Design Automation, VLSI System Design, High-Level Synthesis, and Iterative Algorithms. Email: [email protected]. Habib Youssef (S’86, M’90) received a Diplome d’Ingenieur en Informatique fromtheFacultedesSciencesdeTunis,Tunisiain1982andaPh.D.incomputer science from the University of Minnesota in 1990. He is currently an Associate Professor of Computer Engineering at King Fahd University of Petroleum and Minerals, Saudi Arabia. Habib Youssef has authored more than 45 journal and conference papers. He is the co-author of the book ”VLSI Physical Design Automation: Theory and Practice”, published by McGraw-Hill Book Co., Europe, (also co-published by IEEE Press), January 1995. His main research interests are CAD of VLSI, Computer networks, and performance evaluation of computer systems, and general stochastic and evolutionary algorithms. Email: [email protected]. 3 Dedication To Our wives Sumaiya and Leila Whose Care, Patience, & Understanding Made It Possible. Sadiq M. Sait & Habib Youssef This book represents a joint effort of both authors who closely cooperated on allchapters. Thenamesofauthorshavebeenlistedinalphabeticalorders. This does not imply any senior-junior relationship. 4 Contents 1 Introduction 3 1.1 Combinatorial Optimization . . . . . . . . . . . . . . . . . . . . . 3 1.1.1 Complexity of Algorithms . . . . . . . . . . . . . . . . . . 7 1.1.2 Hard Problems versus Easy Problems . . . . . . . . . . . 8 1.2 Optimization Methods . . . . . . . . . . . . . . . . . . . . . . . . 11 1.3 States, Moves, and Optimality . . . . . . . . . . . . . . . . . . . 13 1.4 Local Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.4.1 Deterministic and Stochastic Algorithms . . . . . . . . . . 16 1.5 Optimal versus Final Solution . . . . . . . . . . . . . . . . . . . . 17 1.6 Single versus Multi-criteria Constrained Optimization . . . . . . 18 1.7 Convergence Analysis of Iterative Algorithms . . . . . . . . . . . 20 1.7.1 Configuration Graph . . . . . . . . . . . . . . . . . . . . . 20 1.8 Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.8.1 Time-Homogeneous Markov Chains . . . . . . . . . . . . 22 1.8.2 Perturbation Probability. . . . . . . . . . . . . . . . . . . 23 1.8.3 Ergodic Markov Chains . . . . . . . . . . . . . . . . . . . 23 1.8.4 Acceptance Probability . . . . . . . . . . . . . . . . . . . 27 1.8.5 Transition Probability . . . . . . . . . . . . . . . . . . . . 27 1.9 Parallel Processing . . . . . . . . . . . . . . . . . . . . . . . . . . 28 1.10 Summary and Organization of the Book . . . . . . . . . . . . . . 34 2 Simulated Annealing (SA) 53 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . 54 2.2 Simulated Annealing Algorithm . . . . . . . . . . . . . . . . . . . 57 2.3 SA Convergence Aspects . . . . . . . . . . . . . . . . . . . . . . . 60 2.4 Parameters of the SA Algorithm . . . . . . . . . . . . . . . . . . 70 2.4.1 A Simple Cooling Schedule . . . . . . . . . . . . . . . . . 71 2.4.2 A Statistical Polynomial Time Cooling Schedule . . . . . 72 2.4.3 An Efficient General Cooling Schedule . . . . . . . . . . . 75 2.5 SA Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 2.6 SA Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 2.6.1 VLSI placement & TimberWolf3.2 . . . . . . . . . . . . . 79 2.6.2 A Brief Overview of Other Applications . . . . . . . . . . 84 5 6 CONTENTS 2.7 Parallelization of SA . . . . . . . . . . . . . . . . . . . . . . . . . 84 2.7.1 Parallel Annealing . . . . . . . . . . . . . . . . . . . . . . 85 2.7.2 Multiple-trials Parallelism . . . . . . . . . . . . . . . . . . 88 2.8 Conclusions and Recent Work . . . . . . . . . . . . . . . . . . . . 94 3 Genetic Algorithms (GAs) 111 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 3.1.1 GA Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 3.1.2 GA Terminology . . . . . . . . . . . . . . . . . . . . . . . 113 3.2 Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 120 3.3 Schema Theorem and Implicit Parallelism . . . . . . . . . . . . . 122 3.3.1 Schema Theorem . . . . . . . . . . . . . . . . . . . . . . . 123 3.3.2 Implicit Parallelism . . . . . . . . . . . . . . . . . . . . . 126 3.4 GA Convergence Aspects . . . . . . . . . . . . . . . . . . . . . . 127 3.4.1 GA without Mutation or Inversion . . . . . . . . . . . . . 128 3.4.2 GA with Crossover and Mutation . . . . . . . . . . . . . . 129 3.5 GA In Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 3.5.1 Genetic Operators . . . . . . . . . . . . . . . . . . . . . . 133 3.5.2 Scoring/Fitness function . . . . . . . . . . . . . . . . . . . 139 3.5.3 Scaling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 3.5.4 Selection of Parents . . . . . . . . . . . . . . . . . . . . . 141 3.6 GA Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 3.6.1 Population . . . . . . . . . . . . . . . . . . . . . . . . . . 143 3.6.2 Generation Gap . . . . . . . . . . . . . . . . . . . . . . . 144 3.6.3 Operator Probabilities . . . . . . . . . . . . . . . . . . . . 145 3.6.4 Other Issues. . . . . . . . . . . . . . . . . . . . . . . . . . 146 3.7 GA Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 3.7.1 Traveling Salesman Problem . . . . . . . . . . . . . . . . 148 3.7.2 Module Placement . . . . . . . . . . . . . . . . . . . . . . 152 3.7.3 Applications of GAs to Other Problems . . . . . . . . . . 155 3.8 Parallelization of GA . . . . . . . . . . . . . . . . . . . . . . . . 157 3.8.1 Island model . . . . . . . . . . . . . . . . . . . . . . . . . 158 3.8.2 Stepping stone model . . . . . . . . . . . . . . . . . . . . 158 3.8.3 Neighborhood or cellular model . . . . . . . . . . . . . . . 158 3.9 Other Issues and Recent Work . . . . . . . . . . . . . . . . . . . 160 3.9.1 Performance Measurement. . . . . . . . . . . . . . . . . . 160 3.9.2 Gray vs Binary Encoding . . . . . . . . . . . . . . . . . . 160 3.9.3 Niches, Crowding and Speciation . . . . . . . . . . . . . . 161 3.9.4 Constant vs Dynamically Decreasing Population . . . . . 163 3.9.5 Multi-Objective Optimization with GAs . . . . . . . . . . 164 3.9.6 Other Related Material . . . . . . . . . . . . . . . . . . . 164 3.10 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 CONTENTS 7 4 Tabu Search (TS) 185 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 4.2 Tabu Search Algorithm . . . . . . . . . . . . . . . . . . . . . . . 187 4.3 Implementation Related Issues . . . . . . . . . . . . . . . . . . . 195 4.3.1 Move Attributes . . . . . . . . . . . . . . . . . . . . . . . 195 4.3.2 Tabu List and Tabu Restrictions . . . . . . . . . . . . . . 196 4.3.3 Data Structure to Handle Tabu-lists . . . . . . . . . . . . 199 4.3.4 Other Aspiration Criteria . . . . . . . . . . . . . . . . . . 205 4.4 Limitations of Short-Term Memory . . . . . . . . . . . . . . . . . 207 4.4.1 Intermediate-Term Memory (Search Intensification) . . . 208 4.4.2 Long-Term Memory (Search Diversification) . . . . . . . . 209 4.5 Examples of Diversifying Search . . . . . . . . . . . . . . . . . . 212 4.5.1 Diversification in Quadratic Assignment (QAP) . . . . . . 212 4.5.2 Diversification in Multiprocessor Scheduling . . . . . . . 215 4.6 TS Convergence Aspects . . . . . . . . . . . . . . . . . . . . . . . 216 4.7 TS Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 4.7.1 QAP Problem . . . . . . . . . . . . . . . . . . . . . . . . 219 4.7.2 Bandwidth Packing Problem (BWP) . . . . . . . . . . . . 222 4.8 Parallelization of TS . . . . . . . . . . . . . . . . . . . . . . . . . 224 4.9 Other Issues and Related Work . . . . . . . . . . . . . . . . . . . 230 4.9.1 Neighborhood, Evaluator functions, Stopping Criteria . . 230 4.9.2 Target Analysis (TA) . . . . . . . . . . . . . . . . . . . . 232 4.9.3 Candidate List Strategies . . . . . . . . . . . . . . . . . . 236 4.9.4 Strategic Oscillation . . . . . . . . . . . . . . . . . . . . . 239 4.9.5 Path Relinking . . . . . . . . . . . . . . . . . . . . . . . . 241 4.10 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 5 Simulated Evolution (SimE) 255 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 5.2 Historical Background . . . . . . . . . . . . . . . . . . . . . . . . 256 5.3 Simulated Evolution Algorithm . . . . . . . . . . . . . . . . . . . 256 5.3.1 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 259 5.3.2 Selection. . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 5.3.3 Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 5.3.4 Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . 264 5.3.5 Initialization Phase . . . . . . . . . . . . . . . . . . . . . . 265 5.4 SimE Operators and Parameters . . . . . . . . . . . . . . . . . . 266 5.4.1 Effect of Selection . . . . . . . . . . . . . . . . . . . . . . 267 5.4.2 Effect of Allocation . . . . . . . . . . . . . . . . . . . . . 267 5.4.3 Effect of Number of Populations . . . . . . . . . . . . . . 268 5.4.4 Effect of Number of Iterations . . . . . . . . . . . . . . . 268 5.5 Comparison of SimE, SA, and GA . . . . . . . . . . . . . . . . . 268 5.6 SimE Convergence Aspects . . . . . . . . . . . . . . . . . . . . . 269 5.7 SimE Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 273 5.7.1 Standard-Cell Design Style . . . . . . . . . . . . . . . . . 273 5.7.2 Other Applications of SimE . . . . . . . . . . . . . . . . 284 8 CONTENTS 5.8 Parallelization of SimE . . . . . . . . . . . . . . . . . . . . . . . 285 5.8.1 Implementation of SimE on Vector Processors . . . . . . . 285 5.8.2 Implementation of SimE on MISD and MIMD machines . 287 5.9 Conclusion and Recent Work . . . . . . . . . . . . . . . . . . . . 289 6 Stochastic Evolution (StocE) 299 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 6.2 Historical Background . . . . . . . . . . . . . . . . . . . . . . . . 300 6.3 Stochastic Evolution Algorithm . . . . . . . . . . . . . . . . . . . 301 6.3.1 StocE Algorithm . . . . . . . . . . . . . . . . . . . . . . . 304 6.3.2 StocE versus SA . . . . . . . . . . . . . . . . . . . . . . . 307 6.3.3 StocE versus SimE . . . . . . . . . . . . . . . . . . . . . 308 6.4 StocE Convergence Aspects . . . . . . . . . . . . . . . . . . . . . 310 6.5 StocE Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 312 6.5.1 Network Bisection . . . . . . . . . . . . . . . . . . . . . . 312 6.5.2 Vertex Cover Problem . . . . . . . . . . . . . . . . . . . . 313 6.5.3 Hamiltonian Circuit Problem . . . . . . . . . . . . . . . . 315 6.5.4 Traveling Salesman Problem . . . . . . . . . . . . . . . . 319 6.5.5 StocE based Technology Mapping of FPGAs . . . . . . . 320 6.6 Parallelization of StocE . . . . . . . . . . . . . . . . . . . . . . . 331 6.7 Conclusion and Recent Work . . . . . . . . . . . . . . . . . . . . 337 7 Hybrids and Other Issues 347 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 7.2 Overview of Algorithms . . . . . . . . . . . . . . . . . . . . . . . 348 7.3 Hybridization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350 7.3.1 SA/TS Hybrid . . . . . . . . . . . . . . . . . . . . . . . . 350 7.3.2 GA/SA Hybrid . . . . . . . . . . . . . . . . . . . . . . . . 351 7.3.3 Others hybrids . . . . . . . . . . . . . . . . . . . . . . . . 353 7.4 GA and Multi-objective Optimization . . . . . . . . . . . . . . . 354 7.4.1 Pareto Optimality . . . . . . . . . . . . . . . . . . . . . . 355 7.4.2 VEGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356 7.4.3 MOGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356 7.5 Fuzzy Logic for Multi-objective Optimization . . . . . . . . . . . 358 7.5.1 Fuzzy Set Theory. . . . . . . . . . . . . . . . . . . . . . . 358 7.5.2 Fuzzy Operators . . . . . . . . . . . . . . . . . . . . . . . 360 7.5.3 Fuzzy Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 361 7.5.4 Example of Fuzzy Multi-objective Optimization . . . . . . 361 7.5.5 Fuzzy Goal Directed Optimization . . . . . . . . . . . . . 363 7.6 Artificial Neural Networks . . . . . . . . . . . . . . . . . . . . . 365 7.7 Solution Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 7.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 Preface Motivation Combinatorial optimization problems are encountered everywhere, in science, engineering, as well as in industrial management, economics, etc. Most Engineering and business schools offer several courses in algorithms and optimization. The advent of the digital computer is credited for all of this explosion in the amount of algorithmic solutions to combinatorial optimization problems. Such solution techniques were unthinkable before this magnificent invention. This book in concerned with one class of combinatorial optimization algorithms: general iterative non-deterministic algorithms. The growing interest in this class of algorithms is attributed to their generality, ease of implementation, and mainly, the many success stories reporting very positive results. We shall limit ourselves to five dominant iterative non-deterministic algorithms, which, in order of popularity are: (1) Simulated Annealing (SA), (2) Genetic Algorithm (GA), (3) Tabu Search (TS), (4) Simulated Evolution, and (5) Stochastic Evolution. All five search heuristics have several important properties in common. 1. They are blind, in that they do not know when they reached the optimal solution. Therefore they must be told when to stop. 2. Theyareapproximationalgorithms,thatis,theydonotguaranteefinding an optimal solution. 3. Theyhave‘hillclimbing’property, thatis, theyoccasionallyacceptuphill (bad) moves. 4. They are easy to implement. All that is required is to have a suitable solution representation, a cost function, and a mechanism to traverse the search space. 5. They are all ‘general’. Practically they can be applied to solve any combinatorial optimization problem. 9 10 CONTENTS 6. They all strive to exploit domain specific heuristic knowledge to bias the searchtoward“good”solutionsubspace. Thequalityofsubspacesearched depends to a large extent on the amount of heuristic knowledge used. 7. Although they asymptotically converge to an optimal solution, the rate of convergence is heavily dependent on the adequate choice of several parameters. The last two properties are the hidden bone in the five combinatorial optimization strategies. Our main goal in this book is to address this aspect by compiling in a single source the extensive research work related to the problem of intelligently setting the required parameters of these five heuristics. Mostbooksoncomputeralgorithmsmainlyaddressdeterministicheuristics. Recently, due to the increase in size and complexity of a large number of combinatorial optimization problems, there has been a growing interest in general iterative non-deterministic algorithms. There are several books dedicated to one particular iterative algorithm. For example, there are at least three books that introduce the theory and concepts of simulated annealing. They are rigorous in mathematics and go far beyond the level of concepts required by engineers (and scientists). There is at least one excellent text book that introduces the theory and concepts of genetic algorithms. The field of genetic algorithms is relatively new and the technique has only been recently attempted to solve several NP-hard problem. Other non-deterministic techniquessuchastabusearch,simulatedevolution,andstochasticevolutionare now gaining ground and applications of such techniques have begun to appear inscientificliterature. Tabusearchhasrecentlybeenthesubjectofanexcellent bookbyF.Glover(KluwerPublishers,1997). Currently,toourknowledgethere arenobooksthataddressthelasttwotechniques. And,nobookisavailablethat contains an integrated and up-to-date description of all the above techniques, with case studies and examples. Allfiveheuristicsdescribedinthisbookconstituteverygeneralandeffective optimizationtechniques. Recently,SA,GA,andTShasbeendesignatedbythe CommitteeoftheNextDecadeofOperationsResearchas‘extremelypromising’ forthefuturetreatmentofpracticalapplications1. ItisourbeliefthatSimulated Evolution and Stochastic Evolution are equally promising techniques to a wide array of combinatorial optimization problems. Organization of the Book The book is organized into 7 chapters. The introductory chapter motivates the student towards a study of iterative algorithms. The definition of NP-hard and NP-complete is introduced, and some “hard” problems are illustrated with examples. 1F. Glover, E. Taillard, and D. de Werra. A user’s guide to tabu search. Annals of Operations Research,41:3–28,1993.

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.