Computational Optimization: A Tribute to Olvi Mangasarian Volume I

265 Pages·1999·5.843 MB·English

by  Jong-Shi Pang (auth.), Jong-Shi Pang (eds.)

COMPUTATIONAL OPTIMIZATION
ATribute to Olvi Mangasarian
Volume 1
edited by
Jong-Shi Pang
The Johns Hopkins University, U.S.A.
A Special Issue of
COMPUTATIONAL OPTIMIZATION AND
APPLICATIONS
An International Journal
Volume 12, Nos. 1/2/3 (1999)
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
COMPUTATIONAL
OPTIMIZATION
AND
APPLICATIONS
An International Jo urnal
Volume 12, Nas. 1/2/3, January 1999
Special Issue: COMPUTATIONAL OPTIMIZATION-A Tribute to Olvi Mangasarian, Part 1
Guest Editor: Jong-Shi Pang
Guest Editorial .......................................................... JonR-Sili PmlR 5
Solving Euelidean Distanee Matrix Completion Problems Via Semidetinite Programming .......... .
· ...................................... Alu/o Y AI(akiil. Amir Khandalli and Helln- Wt)lkol\'ic~ 13
A Logarithmic-Quadratie Proximal Method for Variational Inequalities ......................... .
· ........................................ AUied Al/sleI/de!: Mare Tehoulle and Sami Ben-Tiba 31
A Note on Error Bounds for Convex and Nonconvex Programs . . . . . .. ....... Dimitri P Bertsekas 41
Multicategory Classitieation by Support Vector Machines ................................... .
· .............................................. Erin 1. Bredensteiner and Kristin P Bel/nett 53
Quartic Barriers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. Richan/ W Omle 81
A Partitioned f-Relaxation Algnrithm for Separable Convex Network Flow Problems ............. .
· ................................... Rellillo De Leone. Robert R. Mever al/d Armal/d Zakariall 107
On a Primal-Dual Analystic Center Cutting Plane Method for Variationallnequalities ............. .
· ........................................................... M. Denault and J.-L. Coffin 127
A Shifted-Barrier Primal-Dual Aigorithm Model for Linearly Constrained Optimization Problems ....
· ... , ...................................... Cial/Ili Di Pillo. StefmlO Lueidi and Laura PahlRi 157
Arithmetic Continuation of Regular Roots of Formal Parametric Polynomial Systems ............. .
· .................................................. B. ClIrti,\' Em'es and Uriel C. Rothhlum 189
Interfaces to PATH 3.0: Design. Implementation and Usage .................................. .
· .................................................. Micilarl C. Ferris and Todd S. Munsoll 207
Existence and Limiting Behavior of Trajectories Associated with Pn-equations ................... .
· ................................................. M. Seetharama Cowda and M.A. Tawhid 229
Stabilized Sequential Quadratic Programming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. William W HaRer 253
Computational optimization : atribute to Olvi Mangasarian / edited by
Jong-Shi Pang.
p. em.
"A special issue of Computational optimization and applieations,
an international journal, volume 12 nos. 1/2/3 (1999) and volume
13, nos. 1/2/3 (1999)."
Includes bibliographical referenees.
ISBN 978-1-4613-7367-4 ISBN 978-1-4615-5197-3 (eBook)
DOI 10.1007/978-1-4615-5197-3
1. Mathematical optimization--Data proeessing. 2. Nonlinear
programming--Data processing. 1. Mangasarian, Olvi L., 1934-
II. Pang, Jong-Shi.
QA402.5.C5587 1999
519.3 '0285--dc21 99-13159
CIP
Copyright © 1999 Springer Science+Business Media New York
Originally published by Kluwer Academic Publishers, New York in 1999
Softcover reprint ofthe hardcover Ist edition 1999
AII rights reserved. No part of this publication may be reproduced, stored in a
retrieval system or transmitted in any form or by any means, mechanieal, photo
copying, recording, or otherwise, without the prior written permission of the
publisher, Springer Science+Business Media, LLC.
Printed on acid-free paper.
., Computational Optimization and Applications 12. 5-12 ( 1999)
'I11III" © l'l'l'l Kluwer Academic Publishers. Manufactured in The Netherlands.
Guest Editorial
The two dozen papers in this collection were submitted by friends, former students, col
leagues, and admirers of Professor Olvi L. Mangasarian in celebration of his sixty-fifth
birthday on January 12, 1999. These papers were authored by forty-five international ex
perts in the field of computational optimization who, directly and indirectly, have all been
influenced and touched by Professor Mangasarian through his scholarly work. teaching
and mentoring, and invaluable friendship. In appreciation of the fundamental and sustained
contributions of this beloved and esteemed pioneer in our field, and the kindness and gene
rosity of this wonderful friend and colleague, it is our great pleasure and honor to dedicate
our works to you, Professor Mangasarian.
Four decades of contributions
The topics addressed by the papers in this collection cover many facets of Professor
Mangasarian's forty years of research contributions. From the biography that follows this
preface, we learn that Professor Mangasarian started his career as a doctoral student in Ap
plied Mathematics at Harvard University. Several of his pioneering papers in 1960s paved
the way for the fruitful development of the nonlinear programming field; others provided
the foundation for his most recent entry into the fast growing areas of machine learning
and data mining. Form his output in nonlinear programming, one must mention his joint
work with Stan Fromovitz and the outstanding book titled Nonlinear Programming. The
former introduced the famous Mangasarian-Fromovitz constraint qualification; the latter
has served the field for thirty years and was recently recognized by the Society of Industrial
and Applied Mathematics as a classic in applied mathematics. Another important work
from this era is the paper on pseudo-convex functions which have played a fundamental
role in the theory of generalized convexity.
In 1970s, Professor Mangasarian 's research interests turned to algorithmic developments
in nonlinear programming. Through his own work and that of his students, he was in
strumental for the development of many well-known algorithms such as the tremendously
successful family of sequential quadratic programming algorithms and the exact penalty
approach to solving nonlinear programs. In the same period, he also began to publish his
highly influential papers on the linear complementarity problem. In 1970, together with Ben
Rosen and Klaus Ritter, Professor Mangasarian organized a major international conference
on nonlinear programming that later turned out to be the first of four highly acclaimed
conferences of this kind, with the latter three co-organized with Bob Meyer and Steve
Robinson. As a young researcher, the Guest Editor was honored to be invited to speak at
the fourth and last such conference held in 1980.
6 PANG
While maintaining his leading role in nonlinear programming and complementarity re
search, Professor Mangasarian broke ground, in the mid-1980s, on two new research fron
tiers: parallel optimization and error bounds for inequality systems. As with his previous
works, he published some of the first papers on the latter two subjects. Today, the theory of
error bounds has developed into a very fruitful topic within mathematical programming with
vast connections to many interesting ideas and great potential for important application.
At the beginning this decade, Professor Mangasarian developed a renewed research in
terest that has occupied much of his time and energy till the present. This began with the
subject of pattern classification and has led him to the novel areas of machine learning and
data mining which, before his involvement, were virtually unknown to most researchers in
the optimization community. Building on his early work in the mid-1960s, he has much
expanded the domain of application and introduced new optimization methodology to deal
with practical problems such as breast cancer diagnosis and prognosis. With the collab
oration of his former and present students, he has been a strong advocate for applying
optimization methods to important problems in data mining.
In addition to being a pioneer of our field, Professor Mangasarian has been a much loved
teacher and mentor to several generations of young researchers. Many of his academic off
spring are now established leaders in their respective areas of expertise. Today, he continues
to teach and nurture the next generations of scholars and leaders. We all wish him good
health and continued success in the next sixty-five years. Happy birthday, Olvi!
A personal tribute
As one rapidly gathers from the tributes in this collection of papers, many researchers have
learned and benefited greatly from Professor Mangasarian. The Guest Editor is no exception.
As a fresh Ph.D. graduate from Stanford University (supervised by Richard Cottle), I had the
good fortune of first meeting Professor Mangasarian in Fall 1976 when I was an Assistant
Research Scientist at the (then-called) Mathematics Research Center at the University of
Wisconsin at Madison (I also thank Steve Robinson who was instrumental in bringing
me to Madison). Since that time, I have always considered Professor Mangasarian as my
second academic father. I have received invaluable support, encouragement, and inspiration
from Professor Mangasarian throughout my career. When Bill Hager first suggested to me
the idea of a special project in honor of Professor Mangasarian, I immediately embraced
the suggestion and took on the task of Guest Editorship of these special issues of the
journal COAP. My hope is to use this responsibility as an opportunity to express my deep
appreciation to a great leader, a trusted mentor, an esteemed colleague, and a dear friend.
Thank you, Olvi!
The papers in this festchrift
The collected papers cover a wide spectrum of computational optimization topics; these
represent a good mix of familiar nonlinear programming topics and such novel paradigms
as semidefinite programming and complementarity constrained nonlinear programs. Many
GUEST EDITORIAL 7
new results are presented in these papers which are bound to inspire further research and
generate new avenues for applications. An informal categorization of the papers is as follows.
Algorithmic advancesfor special classes ofc onstrained optimization problems. De Leone,
Meyer, and Zakarian present an E relaxation algorithm for separable convex network flow
problems; Di Pillo, Lucidi, and Palagi introduce a shifted-barrier primal-dual algorithm
model for linearly constrained problems; Hager establishes new convergence results for
stabilized sequential quadratic programming algorithms; Polak, Qi, and Sun describe first
order algorithms for semi-infinite min-max problems; Vanderbei and Shanno present ex
tensive computational results on an interior pont algorithm for nonconvex programs; and
Yamakawa and Fukushima report the results of testing parallel variable transformations.
Analysis of linear and nonlinear programs. Bertsekas presents an interesting approach
to obtain error bounds for convex and nonconvex programs; Klatte and Kummer study
generalized Kojima-functions and Lipschitz stability of critical points; Luo and Zhang
obtain some extensions of the classical Frank-Wolfe Theorem to quadratically constrained
quadratic programs; and Tseng analyzes the convergence and establishes new error bound
results for linear programs under nonlinear perturbations.
Several papers deal with variational inequalities and the class of optimization problems
with equilibrium constraints.
Algorithmic advances. Auslender, Teboulle, and Ben-Tiba describe a logarithmic-quadratic
proximal method for variational inequalities and establish its global convergence under a
very weak condition; Denault and Goffin report extensive computational results on a primal
dual analytic center cutting plane method for variational inequalities; Ferris and Munson
describe the design, implementation, and usage of interfaces to the award-winning software
PATH for solving mixed complementarity problems; Solodov presents some equivalent op
timization reformations of the extended linear complementarity problem; Sun and Qi focus
on various merit functions for the nonlinear complementarity problem; and Jiang and Ralph
describe a MATLAB generator of test problems for mathematical programs with quadratic
objectives and affine variational inequality constraints.
B-stationary points of mathematical programs with equilibrium constraints. Pang and
Fukushima introduce some novel complementarity constraint qualifications under which
verification for B-stationarity of this class of difficult mathematical programs can be sim
plified.
The remaining papers address various mathematical and applied topics related to com
putational optimization.
Applications of optimization. Alfakih, Khandani, and Wolkowicz apply the methodology
of semidefinite programming to Euclidean distance matrix completion problems; Bennett
and Bredensteiner explain a structured quadratic programming approach to multicategory
classification by support vector machines; Powell discusses an interesting geometric prob
lem and how it can be solved by a special optimization method.
8 PANG
Some mathematical topics. Cottle surveys some propositions arising from the literature of
pure and applied mathematics whose validity is restricted by an associated natural number
not exceeding four; Hiriart-Urruty and Lewis present computational formulas for the Clarke
and Michel-Penot subdifferentials of the eigenvalues of a symmetric matrix.
Systems of nonlinear equations. Eaves and Rothblum discuss the continuation of regular
points of formal parametric polynomial systems; and Gowda and Tawhid establish the
existence and analyze the limiting behavior of trajectories associated with equations with
Po defining functions.
Acknowledgments
The Editor-in-Chief of COAP, Bill Hager, suggested the idea of devoting a special issue of
the journal dedicated to Olvi Mangasarian. We are thankful to all the authors for contribut
ing their interesting papers to make this festchrift a success. Many of them have also acted
as reviewers for others' papers. We have also relied on additional individuals to serve as
referees for the papers published herein. Their efforts were particularly commendable; the
editors and the authors are very grateful for their help.
Jong-Shi Pang
Baltimore, Maryland
GUEST EDITORIAL 9
10 PANG
Professor Olvi L. Mangasarian's Biography
Olvi Mangasarian was born in Baghdad, Iraq on January 12, 1934. His first two years of
college, were at the American University of Beirut in Lebanon followed by three years
at Princeton University where he obtained his BSE and MSE in 1954 and 1955. Af
ter Princeton, Olvi went to Harvard University where he completed his Ph.D. in Applied
Mathematics in 1958 in the Division of Engineering and Applied Physics and stayed there
for another year as a Research Fellow. In 1959 he joined the Applied Mathematics De
partment of Shell Development Company in Emeryville, California. In 1967 Olvi joined
the Computer Sciences Department at the University of Wisconsin at Madison where he
has been the John von Neumann Professor of Mathematics and Computer Science since
1982. During the period 1965-67 Olvi held an adjunct appointment in the Electrical and
Industrial Engineering Departments at the University of California at Berkeley where he
taught a course on nonlinear programming. The material taught in that course constituted
the basis for his book, "Nonlinear Programming", published by McGaw-Hill in 1969 and
re-published by SIAM in 1994.
In 1959 Olvi married Claire Garabedian of Worcester, Massachusetts, a graduate of Tufts
University. They have three children. Leon, a journalist, and his wife Tatjana live in Bonn,
Germany. Jeffrey, a computer scientist, his wife Cheryl and their son Tarrant live in EI
Granada, California. Aram, a biologist, lives currently in Geneva, Switzerland.
Olvi enjoys lap swimming on workdays. On weekends he likes to work in the garden when
the temperature is above freezing. He also enjoys classical music, photography, historical
biographies and discovering unusual World Wide Web sites.
Selected publications of Professor Olvi L. Mangasarian
1. "Linear and nonlinear separation of patterns by linear programming," Operations Re
search, vol. 13, pp. 444-452,1965.
2. "Pseudo-convex functions," SIAM Journal on Control, vol. 3, pp. 281-290, 1965.
3. (with S. Fromovitz) "The Fritz John necessary optimality conditions in the presence
of equality constraints," Journal of Mathematical Analysis and Applications, vol. 17,
pp. 34-47, 1967.
4. "Multi-surface method of pattern separation," IEEE Transaction on Information Theory,
vol. IT-14, pp. 801-807, 1968.
5. (with U.M. Garcia-Palomares) "Superlinearly convergent quasi-Newton algorithms for
nonlinearly constrained optimization problems," Mathematical Programming, vol. 11,
pp. 1-13, 1976.
6. "Equivalence of the complementarity problem to a system of nonlinear equations,"
SIAM Journal on Applied Mathematics, vol. 31, pp. 89-92,1976.
7. "Solution of symmetric linear complementarity problems by iterative methods," Journal
of Optimization Theory and Applications, vol. 22, pp. 465-485, 1977.
8. (with S.P. Han) "Exact penalty functions in nonlinear programming," Mathematical
Programming, vol. 17, pp. 251-169,1979.
GUEST EDITORIAL II
9. "Locally unique solutions of quadratic programs, linear and nonlinear complementarity
problems," Mathematical Programming, vol. 19, pp. 200-212, 1980.
10. "A condition number for linear inequalities and linear programs," in "Methods of Oper
ations Research," Proceedings of 6th Symposium on Operations Research, Augsburg,
September 1981, G. Bamberg and O. Opitz (Eds.), Verlagsgruppe AthenaumlHainl
Scriptor/Hanstein: Konigstein, 1981, pp. 3-15.
II. "A condition number for differentiable convex inequalities," Mathematics of Operations
Research, vol. 10, pp. 175-179, 1985.
12. (with T.H. Shiau) "A variable-complexity norm maximization problem," SIAM Journal
on Algebraic and Discrete Methods, vol. 7, pp. 455-461, 1986.
13. (with T.H. Shiau) "Error bounds for monotone linear complementarity problems,"
Mathematical Programming, vol. 36, pp. 81-89, 1986.
14. "A simple characterization of solution sets of convex programs," Operations Research
Letters, vol. 7, pp. 21-26,1988.
15. (with R. De Leone) "Serial and parallel solution of large scale linear programs by
augmented Lagrangian successive overrelaxation," in Optimization, Parallel Processing
and Applications, A. Kurzhanski, K. Neumann, and D. Pallaschke (Eds.), Lecture Notes
in Economics and Mathematical Systems, vol. 304, Springer-Verlag, Berlin, 1988,
pp. 103-124.
16. (with R. Setiono and W.H. Wolberg) "Pattern recognition via linear programming:
Theory and application to medical diagnosis," in Large-Scale Numerical Optimization,
T.F. Coleman and Y. Li (Eds.), SIAM Publications,: Philadelphia, 1990, pp. 22-30.
17. (with M.e. Ferris) "Minimum principle sufficiency," Mathematical Programming,
voI.B57,pp. 1-14, 1992.
18. (with K.P. Bennett) "Neural network training via linear programming," in Advances in
Optimization and Parallel Computing, P.M. Pardalos (Ed.), North Holland: Amsterdam,
1992, pp. 56-67.
19. (with K.P. Bennett) "Robust linear programming discrimination of two linearly insep
arable sets," Optimization Methods and Software, vol. 1, pp. 23-34, 1992.
20. (with M.V. Solodov) "Nonlinear complementarity as unconstrained and constrained
minimization," Mathematical Programming, Series B, vol. 62, pp. 277-297, 1993.
21. (with M.Y. Solodov) "Serial and parallel backpropagation convergence via nonmono
tone perturbed minimization," Optimization Methods and Software, vol. 4, pp. 103-
116, 1994.
22. (with M.e. Ferris) "Parallel variable distribution," SIAM Journal on Optimization,
vol. 4, pp. 815-832,1994.
23. "Misclassification minimization," Journal of Global Optimization, vol. 5, pp. 309-323,
1994.
24. (with J.S. Pang) "The extended linear complementarity problem," SIAM Journal on
Matrix Analysis and Applications, vol. 16, pp. 359-368, 1995.
25. (with W.H. Wolberg and W.N. Street) "Breast cancer diagnosis and prognosis via linear
programming," Operations Research, vol. 43, pp. 570-577, 1995.
26. (with e. Chen) "A class of smoothing functions for nonlinear and mixed complementar
ity problems," Computational Optimization and Applications, vol. 5, pp. 97-138, 1996.