Nonlinear Programming. Proceedings of a Symposium Conducted by the Mathematics Research Center, the University of Wisconsin–Madison, May 4–6, 1970

491 Pages·1970·18.703 MB·English

by  J. B. Rosen, O. L. Mangasarian, K. Ritter

Nonlinear
Programming
Edited by J.B.Rosen
O.LMangasarian
K. Ritter
Proceedings of a Symposium
Conducted by the Mathematics Research Center,
The University of Wisconsin, Madison
May 4-6,1970
®
Academic Press
New York • London 1970
COPYRIGHT © 1970, BY ACADEMIC PRESS, INC.
ALL RIGHTS RESERVED
NO PART OF THIS BOOK MAY BE REPRODUCED IN ANY FORM,
BY PHOTOSTAT, MICROFILM, RETRIEVAL SYSTEM, OR ANY
OTHER MEANS, WITHOUT WRITTEN PERMISSION FROM
THE PUBLISHERS.
ACADEMIC PRESS, INC.
Ill Fifth Avenue, New York, New York 10003
United Kingdom Edition published by
ACADEMIC PRESS, INC. (LONDON) LTD.
Berkeley Square House, London W1X 6BA
LIBRARY OF CONGRESS CATALOG CARD NUMBER: 75-132012
PRINTED IN THE UNITED STATES OF AMERICA
Foreword
This volume contains the proceedings of a Symposium on Nonlinear Program­
ming held in Madison, Wisconsin, on May 4-6, 1970, and sponsored by the
Mathematics Research Center, University of Wisconsin.
The organizing committee and editors of these proceedings consisted (in
addition to myself as chairman) of my colleagues here in Madison, Olvi
Mangasarian and Klaus Ritter. The Symposium consisted of five sessions.
Sessions were chaired by R. R. Hughes and D. L. Russell, also both at
Madison, in addition to the three of us on the organizing committee.
The Symposium was attended by 213 registrants. Its success was due in
large measure to the previous experience, hard work, and improvisation,
when needed, of Mrs. Gladys Moran as Symposium secretary and Steve
Robinson of the MRC staff. The prompt publication of these proceedings
is due largely to the fast and accurate typing of the manuscripts by Mrs.
Carol Chase.
In view of the unusual circumstances during, and subsequent to, this Sympo­
sium, I feel that some additional remarks are in order. These remarks, of
course, represent my own views.
The difficulties in holding a scientific conference (or any other intellectual
activity) on many university campuses during May, 1970, were made painfully
clear to many who attended this Symposium. These difficulties strongly sug­
gest that the era of sheltered pursuit of academic research interests has ended
and that (justified or not) a scientist is now likely to be held accountable for
any ultimate use to which his research is put.
Violence on the Madison campus reached its climax in the early morning of
August 24, when the bombing of Sterling Hall was carried out with the ap­
parent intention of destroying the Mathematics Research Center. The well-
known results were tragic for the faculty and graduate students of the Physics
Department who occupied the basement and first floor of Sterling Hall. The
final typed manuscripts for these proceedings were in the secretarial office on
IX
FOREWORD
the second floor just above the blast area. Fortunately, they were all recovered
essentially intact, as were all other manuscripts and research reports at the
Mathematics Research Center. In this instance, as in all too many others, it
seems that resort to violence, while rarely achieving its intended objective,
almost always leads to tragedy.
J. B. Rosen
October, 1970
x
Preface
It was the intention of the organizing and editorial committee for this Sym­
posium on Nonlinear Programming to emphasize those algorithms and related
theory which lead to efficient computational methods for solving nonlinear
programming problems. Therefore one of the main purposes of this Sympo­
sium was to further strengthen the existing relationship between theory and
computational aspects of this subject. I hope the reader will agree that the
17 papers in these Proceedings are sufficient evidence that we have been
successful.
In view of this it is difficult to classify the papers here with regard to their
theoretical or computational emphasis. However, for convenience we have
attempted to present them according to three general groupings. The first
nine papers are concerned primarily with computational algorithms.1 The
second four papers are devoted to theoretical aspects of nonlinear program­
ming, while in the papers of Duffin, Krafft, Barrodale, and Meyer, certain ap­
plications of nonlinear programming are considered. The word applications
is being used in a somewhat limited sense in connection with these four
papers. They all represent applications to other basic areas (physics, statistics,
approximation) which in turn may be used to solve more applied problems.
An application of nonlinear programming (or for that matter any other math­
ematical or computational method) is usually interpreted in a broader sense
to mean the use of the method to solve some specific scientific, economic, or
even sociological problem. In principle, any such problem which can be
formulated in terms of an objective function to be minimized or maximized,
subject to various conditions or constraints, is one to which mathematical
programming methods can be applied. Clearly the scope of such problems is
very large and includes many of the technological problems facing our society
today. In particular, many of the environmental problems can be formulated
in terms of satisfying stated conditions at a minimum cost. The methods
presented in these Proceedings can then be applied directly to solve these
problems. An excellent survey of such applications in the broader sense, is
given in the recent paper by Van Dyne, Frayer, and Bledsoe.2
xi
PREFACE
Among the more active areas of research covered in these papers are algorithms
for nonlinear constraint problems, investigation of convergence rates, and the
use of nonlinear programming for approximation.3 Computational results are
included in several of the papers and played an important role in motivating
many of the others. I believe that experimental computing remains an essential
part of this field in developing new algorithms and comparing the performance
of known methods on different types of problems. Areas in which significant
work still remains to be done include a unified (and possibly simpler) theory
of convergence rates for many of the existing algorithms, and the application
of mathematical programming to generalized approximation, including bound­
ary value problems.
J. B. Rosen
lrThe paper by A. M. Geoffrion titled "Generalized Benders' decomposition"
which was presented at this Symposium will be published in the Journal of
Optimization Theory and Applications.
2G. M. Van Dyne, W. E. Frayer, and L. J. Bledsoe, "Some optimization
techniques and problems in the natural resource sciences," Studies in
Optimization 7, Soc. Indust. Appl. Math., Philadelphia, 1970, pp. 95-124.
3An application of mathematical programming to a generalized approximation
problem is illustrated by the jacket design. This shows an approximate solu­
tion to the Navier-Stokes equations on a square domain obtained by mini­
mizing the maximum error in the differential equation over the domain. For
details see J. B. Rosen, "Approximate solution to transient Navier-Stokes
cavity convection problems," Computer Sciences Dept. Tech. Rep. No. 32,
Univ. of Wisconsin, Nov., 1970.
xii
A Method of Centers by Upper-Bounding
Functions with Applications
P. HUARD
ABSTRACT
The convergence of the method of centers is obtained
with approximate centres, if the corresponding errors tend to
zero. A very general procedure is developed by using an
upper-bound of the F-distance. Different applications of
this procedure lead back to classical methods such as those
of Zoutendijk, Frank-Wolfe, and Rosen.
1
P. HUARD
Introduction
Many methods of solving nonlinear programming
problems have been proposed. Although some of these meth­
ods are basically similar, it is not easy to classify them into
a small number of families. Zangwill (13), Polak (7, 8),
Chevassus (2), Topkis and Veinott (11), Roode (9), and the
present author (5) have proposed very general algorithms
each englobing a number of particular methods. The question
is whether this synthesis work is of interest and, if so, why.
For the mathematician, this work is interesting in
that it brings about unification in the theory; it becomes,
sooner or later, something which he cannot dispense with.
On the other hand, it is not very useful, from a practical
standpoint, to establish simpler proofs of the convergence in
a particular method: it is indeed easier to prove the con­
vergence in a particular rather than general context. In this
respect, it should be noted that while the proofs relating to
the general algorithms are very often short, they rely on a
number of necessary hypotheses, and when applying them to
a particular method it usually requires a considerable effort
to establish that these hypotheses are satisfied.
However, still from a practical standpoint, the theory
of a general algorithm can be of great interest in discussing
the conditions of convergence of a particular method, and in
specifying what is necessary to this convergence and what
is not. Thus, a method can be modified, with a view to
accelerate its convergence, on a heuristic or practical basis
2
BOUNDED METHOD OF CENTERS
(e.g. reduction of the accuracy of some computations),
while making sure that the conditions for obtaining the opti­
mal solution are not violated.
The subject of this article lies somewhere between
both extremes: starting from the method of centers (1) a
very general algorithm, a particular variant is derived.
In the method of centers, one has to determine at
each step a feasible interior solution, or "center", by
maximizing a somewhat arbitrary function, called "F-
distance", which characterizes the distance from the boundary
of the domain. The determination of such a center can be
made approximately, provided that the error thus induced
tends to zero during the iterative procedure. This possibility
is used here, taking a particular F-distance and making the
computations with simpler upper bounding functions.
The variant so obtained is still a fairly general
algorithm, since by particularizing it still further we
get at well-known methods such as those of Zoutendijk
(14) (method of feasible directions), of Frank and Wolfe
(4)j and of Rosen (10) (gradient projection method).
In particular an "anti-zigzag" process slightly
different from that proposed by Zoutendijk is obtained in a
natural manner under our algorithm. It is interesting to
note that between the method of feasible directions and the
linearized method of centers (described in (6)) the difference
lies only in the value of a scalar parameter.
For Rosen's method of projected gradient, a fairly
simple procedure for ensuring the convergence of the method
is found.
Notation
R Set of real numbers
Rn n-dimensional euclidian space
P(Rn) set of all subsets of Rn
N set of positive integers
3
P. HUARD
If x € Rn, x. i-th component of x
If A c Rn, Fr (A) boundary of A
o
A interior of A
If f : Rn —*R differentiable at x, Vf(x) value of the
gradient of f at x
1. The Method of Centers: A Summary with Modifications
1.1. F-distance, center, e-center
Let 6cP(Rn) be a set whose elements are subsets
of Rn, and let d: Rn X £ —R be a real function.
1.1.1. Definition: d is called an F-distance on
Rn X e if it satisfies:
(i) d(x, E) = 0, VE€£, Vx€Fr(E)
(ii) d(x, E) > 0, VE € 6, VX<EE
(iii) V E € 6 , VE1 e e: E cE';VxeE, there exists
a scalar p(x) > 0 such that:
d(x, E) < p(x) • d(x, E")
Remark: If we take p(x) = c- = 1 in property (iii)
we find the definition already given in (1) . This weakening
of the definition of an F-distance has been given by
Tremolieres in (12).
1.1. 2. Definition. An F-distance d is said to be
regular if it satisfies in addition:
(iv) V sequences {E^. € £ | keN} and
{x€Rn | k cN} such that:
4