Lecture Notes ni Computer Science Edited by G. Goos and .J Hartmanis 862 .M.P soladraP J.B. nesoR deniartsnoC labolG :noitazimitpO smhtiroglA dna snoitacilppA # galreV-regnirpS Berlin Heidelberg NewYork London Paris oykoT Editorial Board D. Barstow W. Brauer .P Brinch Hansen D. Cries D. Luckham C. Moler A. Pnueli G. Seegm(Jller .J Stoer N. Wirth Authors Panos M. Pardalos Computer Science Department, The Pennsylvania State University University Park, PA 16802, USA .J Ben Rosen Computer Science Department, University of Minnesota MinneapoLis, MN 55455, USA CR Subject Classification (1987): G.1.6, G.1.2, E2.1 ISBN 3-540-18095-8 Springer-Verlag Berlin Heidelberg New York ISBN 0-38?-18095-8 Springer-Verlag New York Berlin Heidelberg sihT work si tcejbus to .thgirypoc llA sthgir era ,devreser rehtehw the whole or part of eht lairetam si ,denrecnoc yllacificeps the sthgir of ,noitalsnart ,gnitnirper esu-er of ,snoitartsulli ,noitaticer ,gnitsacdaorb noitcudorper no smliforcim or ni other ,syaw dna egarots ni data .sknab noitacilpuD of this noitacilbup or parts foereht si only dettimrep rednu eht snoisivorp of eht namreG thgirypoC waL of rebmetpeS ,9 1965, ni sti noisrev fo enuJ ,42 ,5891 dna a thgirypoc fee tsum syawla eb .diap snoitaloiV fall under eht noitucesorp tca of eht namreG thgirypoC .waL (cid:14)9 galreV-regnirpS Berlin grebledieH 7891 detnirP ni ynamreG gnitnirP dna :gnidnib suahkcurD ,ztleB .rtsgreB/hcabsmeH 2145/3140-543210 Preface Global optimization is concemed with the characterization and computation of global minima or maxima of nonlinear functions. The general constrained global minimization problem has the following form: Given: K ff.R n compact set, f R~---K: continuous function Find: x* e K, f*=f (x*) such that f* <f (x ) for all x eK Such problems are widespread in mathematical modeling of real world systems for a very broad range of applications. Such applications include economies of scale, fixed charges, allocation and location problems, quadratic assignment and a number of other combinatorial optimization problems. More recently it has been shown that certain aspects of VLSI chip design and database problems can be for- mulated as constrained global optimization problems with a quadratic objective function. Although standard nonlinear programming algorithms will usually obtain a local minimum to the problem, such a local minimum will only be global when certain conditions are satisfied (such as f and K being convex). In general several local minima may exist and the corresponding function values may differ substan- tially. The problem of designing algorithms that obtain global solutions is very difficult, since in general, there is no local criterion for deciding whether a local solution is global. Active research during the past two decades has produced a variety of methods for finding constrained global solutions to nonlinear optimization problems. In this monograph we consider deterministic methods which include those concerned with enumerative techniques, cutting planes, branch and bound, bilinear programming, general approximate algorithms and large-scale approaches. There has been a significant recent increase in research activity on the subject of constrained global optimization and related computational algorithms. This IV monograph summarizes much of this recent work and contains an extensive list of references to papers on constrained global optimization, deterministic solution methods, and applications. We hope that this work wiU be valuable for other researchers in global optimization. We wish to express our appreciation and thanks to Andrew T. Phillips and Nainan Kovoor who carefully read an earlier version of this monograph and sug- gested a number of valuable improvements. The authors' research described in this monograph was supported in part by the National Science Foundation Grant DCR8405489. June 1987 P.M. Pardalos, J.B. Rosen Table of Contents Chapter 1 Convex sets and functions ..................................................................... 1 1.1 Convex sets ................................................................................................................ 1 1.2 Linear and affine spaces ........................................................................................... 2 1.3 Convex hull ................................................................................................................ 3 1.4 Convex and concave functions ............................................................................... 4 1.5 Convex envelopes ..................................................................................................... 6 1.6 Exercises ..................................................................................................................... l0 1.7 References ................................................................................................................... 11 Chapter 2 Optimality conditions in nonlinear programming ........................... 13 2.1 Quadratic problems solvable in polynomially bounded time ............................ 16 2.2 Some general remarks .................................... .......................................................... 19 2.3 Exercises ....... . .............................................................................................................. 20 2.4 References 21 Chapter 3 Combinatorial optimization problems that can be formu- lated as nonconvex quadratic problems .................................................................. 24 3.1 Integer programming ................................................................................................ 24 3.2 Quadratic 0-1 programming .................................................................................... 25 3.3 Quadratic assignment problem ................................................................................ 26 3.4 The 3-dimensional assignment problem ................................................................ 27 3.5 Bilinear programming ............................................................................................... 28 3.6 Linear complementarity problems .......................................................................... 28 3.7 Max-rain problems .................................................................................................... 30 3.8 Exercises ..................................................................................................................... 30 3.9 References ................................................ . .................................................................. 32 Chapter 4 Enumerative methods in nonconvex programming ........................ 34 4.1 Global concave minimization by ranking the extreme points .......................... 34 4.2 Construction of linear underestimating functions ............................................... 37 4.3 An algorithm for.the indefmite quadratic problem ............................................. 39 4.4 Concave cost network flow problems ................................................................... 43 VI 4.5 Exercises ..................................................................................................................... 44 4.6 References ................................................................................................................... 46 Chapter 5 Cutting plane methods ............................................................................. 49 5.1 Tuy's "cone splitting" method ................................................................................ 49 5.2 General class of algorithms ..................................................................................... 52 5.3 Outer and inner approximation methods ..... , ......................................................... 54 5.4 Exercises ..................................................................................................................... 55 5.5 References ................................................................................................................... 56 Chapter 6 Branch and bound methods ................................................................... 58 6.1 Rectangular partitions and convex envelopes ...................................................... 58 6.2 Simplex partitions ...................................................................................................... 61 6.3 A successive underestimation method ................................................................... 64 6.4 Techniques using special ordered sets ~ . ................................................................. 67 6.5 Factorable nonconvex programming ...................................................................... 69 6.6 Exercises .................. . .................................................................................................. 71 6.7 References ................................................................................................................... 72 Chapter 7 Bilinear programming methods for nonconvex quadratic problems ............................................................................................................................ 75 7.1 Introduction ................................................................................................................ 75 7.2 Maximization of convex quadratic functions ....................................................... 77 7.3 Jointly constrained bilinear problems .................................................................... 79 7.4 Exercises ..................................................................................................................... 81 7.5 References ................................................................................................................... 82 Chapter $ Large scale problems ................................................................................ 84 quadratic 8.1 Concave problems ................................................................................... 84 8.2 Separable formulation ............................................................................................... 85 8.3 Linear underestimator and error bounds ............................................................... 86 8.4 Piecewise linear approximation and zero-one integer formulation .................. 90 8.5 Computational algorithm and analysis .................................................................. 94 8.6 Exercises ..................................................................................................................... 98 8.7 References ................................................................................................................... 99 Chapter 9 Global minimization of indefinite quadratic problems .................. 101 9.1 Introduction ................................................................................................................ 101 9.2 Partition of the objective function .......................................... . ............................... 104 9.3 Approximate methods and error bounds ............................................................... 105 vl! 914 Branch and bound bisection techniques ................................................................ 107 9.5 Piecewise linear approximation .............................................................................. 110 9.6 Algorithm and computational results ..................................................................... 111 9.7 Concluding remarks .................................................................................................. 113 9.8 Exercises ..................................................................................................................... 114 9.9 References ................................................................................................................... 115 Chapter 10 Test problems for global nonconvex quadratic program- ming algorithms .............................................................................................................. 118 10.1 Introduction .............................................................................................................. 118 10.2 Global minimum concave quadratic test problems ........................................... 118 10.3 Generation of test problems with nonconvex (concave or indefinite) quadratic objective function ........................................................................................... 120 10.4 Exercises ................................................................................................................... 121 10.5 References ................................................................................................................ 122 General references for constrained global optimization ..................................... 123 Chapter 1 Convex sets and functions Convex sets and functions play a dominant role in optimization and some of their properties are essential in the study of several algorithms. In this introductory chapter we start with summary of some of the most important properties that we are going to use. 1.1 Convex sets A subset C of the Euclidean space R n is said to be convex if for every x 1, x2eC and ,~ real, 0 < ,~ < 1, the point LXl+(1-~.)x2e C. The geometric interpretation of this definition is clear. For any two points of C, the line segment joining these two points lies entirely in C. m Given the vectors xt ..... x,,, in R" and real numbers i,~ > 0 with ~,i = ,1 i=1 the vector sum ~,1x1+ (cid:12)9 (cid:12)9 (cid:12)9 +)~,,,x,,, is called a convex combination of these points. Some properties of convex sets are summarized in the next theorems. Theorem 1.1.1: A subset of R n is convex iff it contains all the convex combi- nations of its elements. Proof: Let C be a convex set in R n. Ifxi~C and i~) > 0, i=1 ..... k, such k k that ~,i = ,1 prove by induction on k that the convex combination ~.~,iXi E C. i=1 i=1 Theorem 1.1.2: Let F be a family of convex sets. Then the intersection C is also a convex set. CEF However, it is easy to see that the union of convex sets need not be convex. Some other algebraic set operations that preserve convexity are defined below. Theorem 1.1.3: 1) Let C be a convex set in R n and a a real number. Then the set aC = {x: x = ay, y e C } is also convex. 2) Let C ,1 C 2 be convex sets in R n. Then the set C 1 -I- C 2 -- {x: x -- Xl-l'x ,2 xIEC ,1 x2EC2} is alSO convex. 1.2 Linear and affine spaces n For any x, yeR n the inner product xTy is the real number ~xiY i. The i=1 n Euclidean norm is defined to be IIx II = (~xi2) 1/2. Other notations and terminology i=1 not defined here are the standard ones used in the literature. A hyperplane H in R" is a set of the form H = {x~Rn:crx =b}. for some vector c e R n and b e R. Similarly we define the closed half spaces H 1 = {x~Rn: cTx -> b}, H 2= {x~Rn: cTx -< b). It is very easy to see that H, H ,1 H 2 are aU convex sets. A nonempty subset V of R n is caUed a (linear) subspace if the foUowing con- ditions are true: i) if x, y~V then x+y~V, ii) if x ~ V and r ~ R then rx ~ V. Next we define the structure of linear subspaces. Let S={v 1 ..... v m } be a set Ecivim , of vectors in V. We say that S spans V if for every vector vE V, v where = i=1 the ci's are real numbers. The set S is said to be linearly independent if we cannot find constants c 1 ..... cm not all zero such that c iv 1+ (cid:12)9 (cid:12)9 (cid:12)9 +Crn mV = 0 (otherwise S is called linearly dependent). 3 If the set S spans V and is linearly independent we call it a basis of V. The dimension of the subspace V, dim(V), is defined to be the number of vectors in some basis S. To get more insight into the geometric and algebraic nature of a subspace we equivalently define a linear subspace to be the set V = {xeRn: cilxl+ " " " +cinxn = O, i = 1 ..... m }, that is, V is the solution set of the homogeneous system of linear equations Cx = 0 where C is the m xn matrix of the coefficients clj. Here the dimension of V is equal to n-rank(C) where the rank of the matrix is the maximum number of linearly independent columns (or rows) of the matrix. An affine subspace A of Rn is a linear subspace V translated by some vector u, that is A = {xeR't: x = u+v, veV}. Also dim(A) = dim(V). Equivalently we can define A = {xeRn:ci:xl+'''+cinx~ =b,i = 1 ..... m}, that is, A is the solution set of the (nonhomogeneous) linear system Cx = b. From the above discussion it is clear that a hyperplane in R ~ is an affine sub- space of dimension n-1. 1.3 Convex hull Another important concept in convexity is that of forming the smallest convex set containing a given subset S in R". The convex hull of S is the set Co (S) = n{ C" C convex in R" and C D S .} The convex hull of a finite set of points is called a convex polytope. It is clear that a convex polytope is always bounded. A convex polytope that contains all its boundary points is closed. A point x on the boundary of S is called an extreme point (or vertex) if there are no distinct points xl, x2eS such that x = kxl+(1-~,)x ,2 0 < ,~ < .1 For example in the plane a triangle has 3 extreme points, and the sphere has all its boundary points as extreme points. The following