Computational Mathematical Programming NATO ASI Series Advanced Science Institutes Series A series presenting the results of activities sponsored by the NA TO Science Committee, which aims at the dissemination of advanced scientific and technological knowledge, with a view to strengthening links between scientific communities. The Series is published by an international board of publishers in conjunction with the NATO Scientific Affairs Division A Life Sciences Plenum Publishing Corporation B Physics London and New York C Mathematical and D. Reidel Publishing Company Physical Sciences Dordrecht, Boston and Lancaster D Behavioural and Martinus Nijhoff Publishers Social Sciences Boston, The Hague, Dordrecht and Lancaster E Applied Sciences F Computer and Springer-Verlag Systems Sciences Berlin Heidelberg New York Tokyo G Ecological Sciences Series F: Computer and Systems Sciences Vol. 15 Computational Mathematical Programming Edited by Klaus Schittkowski Institut fur Informatik, Universitat Stuttgart, AzenbergstraBe 12 7000 Stuttgart 1, Federal Republic of Germany Springer-Verlag Berlin Heidelberg New York Tokyo Published in cooperation with NATO Scientific Affairs Division Proceedings of the NATO Advanced Study Institute on Computational Mathematical Programming held at Bad Windsheim, Federal Republic of Germany, July 23-August 2, 1984 ISBN-13: 978-3-642-82452-4 e-ISBN-13: 978-3-642-82450-0 001: 10.1007/978-3-642-82450-0 Library of Congress Cataloging in Publication Data. NATO Advanced Study Institute on Computational Mathematical Programming (1984 : Bad Windsheim, Germany) Computational mathematical programming. (NATO ASI series. Series F, Computer and systems sciences; vol. 15) "Proceedings of the NATO Advanced Study Institute on Computational Mathematical Programming held at Bad Windsheim. Federal Republic of Germany. July 23-August 2, 1984"-T. p. verso. "Published in cooperation with NATO Scientific Affairs Division." 1. Programming (Mathematics)-Data processing-Congresses.1. Schittkowski, Klaus. 1946-.11. North Atlantic Treaty Organization. Scientific Affairs Division. III. Series: NATOASI series. Series F, Computer and system sciences; no. 15. QA402.5.N3651984 519.7 85-8055 This work is subject to copyright. All rights are reserved, whether the Whole or part of the material is concerned, specifically those of translating. reprinting, re-use of illustrations. broadcastings, reproduction by photocopying machine or similar means, and storage in data banks. Under § 54 of the Gerrnan Copyright Law where copies are made for other than private use. a fee is payable to "Verwertungsgesellschaft Wort", Munich. © Springer-Verlag Berlin Heidelberg 1985 Softcover reprint of the hardcover 1s t edition 1985 Printing: Beltz Offsetdruck, Hemsbach; Bookbinding: J. Schaffer OHG, Grunstadt 2145/3140-54321 0 Preface This book contains the written versions of main lectures presented at the Advanced Study Institute (ASI) on Computational Mathematical Programming, which was held in Bad Windsheim, Germany F. R., from July 23 to August 2, 1984, under the sponsorship of NATO. The ASI was organized by the Committee on Algorithms (COAL) of the Mathematical Programming Society. Co-directors were Karla Hoffmann (National Bureau of Standards, Washington, U.S.A.) and Jan Teigen (Rabobank Nederland, Zeist, The Netherlands). Ninety participants coming from about 20 different countries attended the ASI and contributed their efforts to achieve a highly interesting and stimulating meeting. Since 1947 when the first linear programming technique was developed, the importance of optimization models and their mathematical solution methods has steadily increased, and now plays a leading role in applied research areas. The basic idea of optimization theory is to minimize (or maximize) a function of several variables subject to certain restrictions. This general mathematical concept covers a broad class of possible practical applications arising in mechanical, electrical, or chemical engineering, physics, economics, medicine, biology, etc. There are both industrial applications (e.g. design of mechanical structures, production plans) and applications in the natural, engineering, and social sciences (e.g. chemical equilibrium problems, christollography problems). In most cases, a theoretical foundation which predicts the numerical performance of a mathematical programming algorithm, does not exist Nor can one at the present time prove theoretically that one algorithm is best for a given application. As a consequence, the performance of optimization methods must be evaluated by experiments. By defining a sample, i. e. by searching for suitable test problems, and by designing a computational experiment, the efficiency, robustness, and reliability of an algorithm can be investigated. Such experimental optimization will become even more important in future years as modeling efforts attempt to solve problems related to evermore complex systems and consequently, the mathematical algorithms used to solve these problems become more and more sophisticated. The need to disseminate information about computational mathematical programming procedures was the main impulse for organizing the Advanced Study Institute. Consequently, the general purpose of the Advanced Study Institute was to link new algorithmic developments for optimization models with practical applications. From the organizational point of view, the ASI consisted of tutorilas, research seminars, and a software fair. The tutorials covered all major types of optimization models currently used in practice. Leading experts gave an introduction to the subject, provided any mathematical background, presented suitable algorithms stressing information about their computer implementations, usage and numerical performance. The topics of the tutorials ranged from linear and integer programming, networks, and various aspects of nonlinear programming to optimal control, stochastic and nonsmooth optimization. They were accomplished by about 60 research seminars of other participants leading to a more detailed insight in special individual research projects. Moreover, information material about existing optimization software was displayed at the ASI e. g. in form of user's guides or code descriptions. The purpose of this book is to compose the tutorials and to offer them to the mathematical programming community. It is expected that only minor knowledge of mathematical theory and terminology is required to understand the most important opinions of the individual authors. In addition, the book contains the main results of the software fair, i. e. a collection of about 50 code descriptions, with the intention to disseminate information about existing optimization programs. The organizer of the ASI is indepted to the NATO Science Committee for the generous financial support. Other sponsors were the National Bureau of Standards of the U. S. Department of Commerce, and the Deutscher Akademischer Austauschdienst. The success of the meeting is based on the quality of the tutorials held at the ASI. Without any exception, the organizer acknowledges the extensive amount of time the invited lecturers had to spend for preparing the tutorials and, in particular, for preparing the manuscripts submitted for publication. Other individuals who deserve special thanks, are the co directors Karla Hoffmann and Jan Teigen, furthermore Marlis and Jochem Zowe for their extensive assistance concerning all local organizational arrangements. K. Schittkowski Contents Integer Programming EML.Beale . ............................................. . Model Building in Linear and Integer Programming H. P. Williams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 LP-Based Combinatorial Problem Solving K Hoffman, M Padberg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Network Optimization R.R.Meyer . .............................................. . 125 Reflections on Geometric Programming M J. Rijckaert E J. C. Walraven. . .. ................................ 141 Principles of Sequential Quadratic Programming Methods for Solving Nonlinear Programs J. Stoer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Model Building and Practical Aspects of Nonlinear Programming P. E Gill, W Murray, M A. Saunders, M H. Wright . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Comparative Performance Evaluation, Experimental Design, and Generation of Test Problems in Nonlinear Optimization FA. Lootsma. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 On Converting Optimal Control Problems into Nonlinear Programming Problems 0. Kraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 A Stochastic Approach to Global Optimization A. H. G. Rinnooy Kan, C. G. E Boender, G. Th. Timmer . . . . . . . . . . . . . . . . . . . . . . . . 281 Algorithmic Procedures for Stochastic Optimization R. J. B. Wets .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 Nondifferentiable Optimization J.Zowe . 323 Parallel Computing in Optimization R. B Schnabel . 357 Software for Mathematical Programming K. Schittkowski . 383 INTEGER PROGRAMMING E.M.L. Beale Scicon Limited Brick Close, Kiln Farm, Milton Keynes, Great Britain. SUMMARY Following an introduction, which discusses the motivation for studying integer programming, the relevance of computational complexity and the relative merits of integer and dynamic programming, the branch and bound method is introduced in general terms. Various types of global entity to which it can be applied are introduced. These are integer variables, semicontinuous variables, special ordered sets and chains of linked ordered sets. A discussion of the algorithmic details follows. Finally, various approaches to automatic model reformulation are discussed: this seems to be the most important current area of integer programming research 1. INTRODUCTION The requirement that the variables must take integer values is a natural mathematical extension of linear programming, which is known as Pure Integer Programming. Alternatively, one may think of planning problems in which some questions require qualitative answers - Yes or No, while others require quantitative answers - How much. The Yes-or-No decisions can be coded as zero-one (or binary) variables. And many of these problems can be formulated as linear programming problems with the additional restriction that some variables must be either zero or one. This is the problem of Mixed Integer Programming if there are other variables that can take continuous values. And it is sometimes useful to give some integer variables upper bounds greater than one. In any integer programming problem, a local optimum may not be a global optimum. This property is shared by nonlinear problems in which either the objective function to be maximized is not concave or the feasible region is not convex or both. Markowitz and Manne (1957) showed that some of these problems can be formulated as mixed integer programming problems. But it has been found better to introduce other types of global entity as described in Section 3 below, rather than to reduce these problems explicitly to mixed integer programming. So the topiC of integer programming can be considered as including global optimization of other models using the methodology developed for integer programming. NATOASISeri~ Vol.F15 Computational Mathematical Programming Edited by K Schittkowski © Springer-Verlag Berlin Heidelberg 1985 2 Integer programming is one approach to problems of combinatorial optimization. Elementary considerations can teach some important lessons, notably that it is unrealistic to expect to solve all problems by waiting for computing to become cheaper or quicker. Any method that requires the enumeration of all 2n possible combinations of Yes and No decisions on n items may well work for,up to 10 or even 20 items. But an increase of computer speed by a factor of 1000 only allows n to increase by 10 with this approach. The situation is even worse if one has to enumerate all n! permutations of n items. A value of n around 10 may then be acceptable, since 10! is only about 3.6 x 106, but each further increase in n by a single unit multiplies the amount of work by a factor of more than 10. Similar remarks apply to methods that improve on complete enumeration but save "only" 99.9% of the work. These facts also illustrate two facts about model building. If the essence of a problem is to make a selection from n items, or to find an optimum ordering of them, then it may be important to simplify the representation of each individual item, even if this requires a fair amount of computing. And it may also be important to suppress minor details from a combinatorial model designed to suggest an overall strategy. Most integer programming problems are such that the amount of work needed to produce a guaranteed optimum solution may increase exponentially with the problem size. The significance of this proposition is limited by two facts: we usually wish to solve a particular class of problems of a given size, and we can also often accept a "good" solution that is "probably nearly optimum" if a guaranteed optimum is unavailable. We should remember that many practical problems with thousands of integer variables can be completely solved in a few minutes. But the result about exponential growth warns us that some problems can become much harder when the problem size increases by an apparently modest amount. We have already noted that complete enumeration of all 2n combinations requires 1000 times as much work when n increases from 20 to 30. Before studying integer programming in detail, it is worth considering dynamic programming as an alternative approach to some problems. For most practical purposes, dynamic programming can be considered as the process of formulating and solving problems as shortest-route problems. Its scope is much more restricted than