Documenta Mathematica Optimization Stories - ZIB

466 Pages·2012·6.29 MB·English

Documenta Mathematica
Journal der
Deutschen Mathematiker-Vereinigung
Gegru¨ndet 1996
Extra Volume
Optimization Stories
21st International Symposium on Mathematical Programming
Berlin, August 19–24, 2012
Editor:
Martin Gro¨tschel
Documenta Mathematica, Journal der Deutschen Mathematiker-Vereinigung,
ver¨offentlicht Forschungsarbeiten aus allen mathematischen Gebieten und wird in
traditionellerWeisereferiert. EswirdindiziertdurchMathematicalReviews,Science
Citation Index Expanded, Zentralblatt fu¨r Mathematik.
Artikel k¨onnen als TEX-Dateien per E-Mail bei einem der Herausgeber eingereicht
werden. Hinweisefu¨rdieVorbereitungderArtikelk¨onnenunterderuntenangegebe-
nen WWW-Adresse gefunden werden.
Documenta Mathematica, Journal der Deutschen Mathematiker-Vereinigung,
publishes research manuscripts out of all mathematical fields and is refereed in the
traditional manner. It is indexed in Mathematical Reviews, Science Citation Index
Expanded, Zentralblatt fu¨r Mathematik.
Manuscriptsshould besubmitted asTEX -files bye-mail toone oftheeditors. Hints
for manuscript preparation can be found under the following web address.
http://www.math.uni-bielefeld.de/documenta
Gescha¨ftsfu¨hrende Herausgeber / Managing Editors:
Alfred K. Louis, Saarbru¨cken [email protected]
Ulf Rehmann (techn.), Bielefeld [email protected]
Peter Schneider, Mu¨nster [email protected]
Herausgeber / Editors:
Christian B¨ar, Potsdam [email protected]
Don Blasius, Los Angeles [email protected]
Joachim Cuntz, Mu¨nster [email protected]
Patrick Delorme, Marseille [email protected]
Gavril Farkas, Berlin (HU) [email protected]
Edward Frenkel, Berkeley [email protected]
Friedrich G¨otze, Bielefeld [email protected]
Ursula Hamenst¨adt, Bonn [email protected]
Lars Hesselholt, Cambridge, MA (MIT) [email protected]
Max Karoubi, Paris [email protected]
Stephen Lichtenbaum Stephen [email protected]
Eckhard Meinrenken, Toronto [email protected]
Alexander S. Merkurjev, Los Angeles [email protected]
Anil Nerode, Ithaca [email protected]
Thomas Peternell, Bayreuth [email protected]
Takeshi Saito, Tokyo [email protected]
Stefan Schwede, Bonn [email protected]
Heinz Siedentop, Mu¨nchen (LMU) [email protected]
Wolfgang Soergel, Freiburg [email protected]
ISBN 978-3-936609-58-5 ISSN 1431-0635 (Print) ISSN 1431-0643 (Internet)
SPARC Documenta MathematicaisaLeadingEdgePartnerofSPARC,
Leading Edge theScholarlyPublishingandAcademicResourceCoalitionoftheAs-
sociationofResearchLibraries(ARL),WashingtonDC,USA.
Address of Technical Managing Editor: Ulf Rehmann, Fakult¨at fu¨r Mathematik, Universit¨at
Bielefeld, Postfach 100131, D-33501 Bielefeld, Copyright (cid:13)c 2010 for Layout: Ulf Rehmann.
TypesettinginTEX.
Documenta Mathematica
Extra Volume: Optimization Stories, 2012
Preface 1
Introduction 3
Stories about the Old Masters of Optimization 7
Ya-xiang Yuan
Jiu Zhang Suan Shu and
the Gauss Algorithm for Linear Equations 9–14
Eberhard Knobloch
Leibniz and the Brachistochrone 15–18
Eberhard Knobloch
Leibniz and the Infinite 19–23
Peter Deuflhard
A Short History of Newton’s Method 25–30
Eberhard Knobloch
Euler and Infinite Speed 31–35
Eberhard Knobloch
Euler and Variations 37–42
Martin Gro¨tschel and Ya-xiang Yuan
Euler, Mei-Ko Kwan, Ko¨nigsberg,
and a Chinese Postman 43–50
Linear Programming Stories 51
David Shanno
Who Invented the Interior-Point Method? 55–64
George L. Nemhauser
Column Generation for Linear and Integer Programming 65–73
Gu¨nter M. Ziegler
Who Solved the Hirsch Conjecture? 75–85
Friedrich Eisenbrand
Pope Gregory, the Calendar,
and Continued Fractions 87–93
iii
Martin Henk
Lo¨wner–John Ellipsoids 95–106
Robert E. Bixby
A Brief History of Linear and
Mixed-Integer Programming Computation 107–121
Discrete Optimization Stories 123
Jaroslav Neˇsetrˇil and Helena Neˇsetrˇilova´
The Origins of
Minimal Spanning Tree Algorithms –
Bor˚uvka and Jarn´ik 127–141
William H. Cunningham
The Coming of the Matroids 143–153
Alexander Schrijver
On the History of the Shortest Path Problem 155–167
Alexander Schrijver
On the History of the Transportation
and Maximum Flow Problems 169–180
William R. Pulleyblank
Edmonds, Matching
and the Birth of Polyhedral Combinatorics 181–197
Thomas L. Gertzen and Martin Gro¨tschel
Flinders Petrie, the Travelling Salesman Problem,
and the Beginning of Mathematical Modeling
in Archaeology 199–210
Rolf H. Mo¨hring
D. Ray Fulkerson and Project Scheduling 211–219
Ge´rard Cornue´jols
The Ongoing Story of Gomory Cuts 221–226
William Cook
Markowitz and Manne + Eastman + Land and Doig
= Branch and Bound 227–238
Susanne Albers
Ronald Graham:
Laying the Foundations of Online Optimization 239–245
Continuous Optimization Stories 247
Claude Lemare´chal
Cauchy and the Gradient Method 251–254
iv
Richard W. Cottle
William Karush and the KKT Theorem 255–269
Margaret H. Wright
Nelder, Mead, and the Other Simplex Method 271–276
Jean-Louis Goffin
Subgradient Optimization in
Nonsmooth Optimization
(including the Soviet Revolution) 277–290
Robert Mifflin and Claudia Sagastiza´bal
A Science Fiction Story in Nonsmooth Optimization
Originating at IIASA 291–300
Andreas Griewank
Broyden Updating, the Good and the Bad! 301–315
Hans Josef Pesch
Carathe´odory on the Road to the Maximum Principle 317–329
Hans Josef Pesch and Michael Plail
The Cold War and
the Maximum Principle of Optimal Control 331–343
Hans Josef Pesch
The Princess and Infinite-Dimensional Optimization 345–356
Computing Stories 357
David S. Johnson
A Brief History of NP-Completeness, 1954–2012 359–376
Robert Fourer
On the Evolution of Optimization Modeling Systems 377–388
Andreas Griewank
Who Invented the Reverse Mode of Differentiation? 389–400
Rau´l Rojas
Gordon Moore and His Law:
Numerical Methods to the Rescue 401–415
More Optimization Stories 417
Thomas M. Liebling and Lionel Pournin
Voronoi Diagrams and Delaunay Triangulations:
Ubiquitous Siamese Twins 419–431
Konrad Schmu¨dgen
Around Hilbert’s 17th Problem 433–438
Michael Joswig
From Kepler to Hales, and Back to Hilbert 439–446
v
Matthias Ehrgott
Vilfredo Pareto and Multi-objective Optimization 447–453
Walter Schachermayer
Optimisation and Utility Functions 455–460
vi
Documenta Math. 1
Preface
Whenindangerofturningthisprefaceintoanessayaboutwhyitisimportant
to know the history of optimization, I remembered my favorite Antoine de
Saint-Exupery quote: “If you want to build a ship, don’t drum up the men to
gather wood, divide the work and give orders. Instead, teach them to yearn
for the vast and endless sea.” Optimization history is not just important; it is
simply fascinating, thrilling, funny, and full of surprises. This book makes an
attempt to get this view of history across by asking questions such as:
• Did Newton create the Newton method?
• Has Gauss imported Gauss elimination from China?
• Who invented interior point methods?
• Was the Kuhn-Tucker theorem of 1951 already proved in 1939?
• Did the Hungarian algorithm originate in Budapest, Princeton or Berlin?
• Who built the first program-controlled computing machine in the world?
• Was the term NP-complete created by a vote initiated by Don Knuth?
• Did the Cold War have an influence on the maximum principle?
• Was the discovery of the max-flow min-cut theorem a result of the Second
World War?
• Did Voronoi invent Voronoi diagrams?
• Were regular matroids characterized by a code breaking chemist?
• Did an archaeologist invent the Hamming distance and the TSP?
• What has the Kepler conjecture to do with “mathematical philosophy”?
• Have you ever heard of an Italian named Wilfried Fritz, born in France and
deceased in Switzerland?
• What does the electrification of South-Moravia have to do with spanning
trees?
• Did Euler cheat Russia and Prussia concerning stolen horses?
• And why did Omar Khayyam compute the third convergent of a continued
fraction?
Interested? Howmanyofthesequestionscanyouanswer? Someofthemtouch
fundamentalissuesofoptimization, othersappearanecdotalorevensomewhat
obscure, but there may be more behind them than you think. The forty-one
articlesinthisbookandmyintroductionstothesectionsprovidesomefulland
some partial answers. Just glance through the book, and I hope you will get
stuck and start reading.
·
Documenta Mathematica Extra Volume ISMP (2012) 1–2
2 Martin Gro¨tschel
Why is the book not entitled Optimization History? Well, this would have
put in a serious claim that I did not want to meet. This book is not intended
to compete with scholarly historical research. A few articles, though, get close
tothat. Noarticleisfiction;allarebasedonsolidhistoricalinformation. ButI
have asked the authors to present also their particular views, and if something
is historically not clear, to present their own opinions. Most of all, I requested
to write in an entertaining way addressing the casual reader.
The articles in this book are not meant for the rare quiet moments in a
study. You can read them on a train or plane ride; and I do hope that you get
excitedaboutsomeofthetopicspresentedandstartinvestigatingtheirhistory
bydiggingdeeperintothesubject. Thereferencesinthearticlesshowyouhow
to do that.
Berlin, August 2012 Martin Gr¨otschel
·
Documenta Mathematica Extra Volume ISMP (2012) 1–2
Documenta Math. 3
Introduction
WhenanInternationalSymposiumonMathematicalProgrammingishostedin
Berlin and when Leonhard Euler is one of the local (and global) mathematical
heroes, one cannot resist the temptation to begin the introduction by quoting
an Euler statement from 1744 that every optimizer loves:
CumenimmundiuniversifabricasitperfectissimaatqueaCreatore
sapientissimo absoluta, nihil omnino in mundo contingit, in quo
non maximi minimive ratio quaepiam eluceat; quamobrem dubium
prorsus est nullum, quin omnes mundi effectus ex causis finalibus
ope methodi maximorum et minimorum aeque feliciter determinari
queant, atque ex ipsis causis efficientibus.
Briefly and very freely translated: Nothing in the world takes place without
optimization, and there is no doubt that all aspects of the world that have a
rational basis can be explained by optimization methods. It is not so bad to
hear such a statement from one the greatest mathematicians of all time.
Optimization is a mathematical discipline that differs considerably from
other areas of mathematics. Practical problems, more generally, classes of
problems, usually arising in fields outside of mathematics, are in the center,
and mathematical models are invented that somehow grasp the essence of the
problems. Thenmathematicaltheoryisdevelopedtounderstandthestructure
of the models. And here, every branch of mathematics that helps provide in-
sightiswelcometosupporttheinvestigations. Optimizationis,thus,influenced
inmanywaysfrommanysourcesandhasnounifiedtheory, althoughthereex-
ist “core technologies” such as linear, nonlinear, combinatorial and stochastic
optimization, each with a rich body of results. But it is not unusual that all of
a sudden, methods, appearing far removed at first sight, start playing impor-
tant roles. The ultimate goal of optimization is not just a good understanding
of models; the research has to yield algorithms that efficiently solve the prob-
lems one has started from. And this ties optimization with the computational
sciences.
One can infer from these introductory remarks that the historic roots of op-
timization are manifold and widespread and that there is no straight line of
development. And this makes the history of optimization even more interest-
ing. Most optimizers I know are not so keen on really thorough and scholarly
·
Documenta Mathematica Extra Volume ISMP (2012) 3–5
4 Martin Gro¨tschel
historical articles. That is why I thought that the best way to popularize the
history of optimization is by presenting brief entertaining and easy to read
articles with a clear and narrow focus.
The articles in this book areof three types. The firsttype, and the majority
of articles belongs to this group, is about a person (usually a famous mathe-
matician,orsometimesanotsowell-knownpersonwhodeservestocometothe
fore)andaboutonemajorachievement(e.g.,Cauchyandthegradientmethod,
Flinders Petrie and the TSP, or Karush and the KKT theorem). Such articles
contain a brief CV of the person (unless he is too well known like Euler or
Leibniz) and then discuss the particular result, algorithm, or achievement that
is important for the history of optimization. I have asked the authors to also
add “personal flavor”, for instance, in cases where the authors had personal
encounters with or have private information about the colleague portrayed.
The second type of articles is of the sort “Who invented ...?”. In many
cases it is not really obvious who did what first, and thus, the task of this
article type is to explore the contributions and come to a conclusion. And a
few articles survey certain developments such as Moore’s Law, the history of
column generation or of NP-completeness.
I wrote to the authors on February 22, 2012 when the serious work on this
book began:
Iamnotrequestingacompletelythoroughaccountofthehistoryof
acertainoptimizationsubjectoraperfectCVofagreatoptimizer.
I would like the articles to be appetizers. They should show, in
particulartheyoungercolleagues,thatoptimizationisafascinating
human endeavor and that there are lots of interesting stories that
happen in the development of our field. There can be surprises,
funny and even tragic stories. There has to be serious and correct
information,ofcourse,butsomehumantouchand/orhumorshould
show.
In my opinion almost all authors have achieved this goal.
My initial favorite for the book title was “Short Optimization Histories”. I
wanted to have short articles on focused aspects of the history of optimization
that should present good stories and should have the flavor of a short story
in fiction literature. I considered this title a nice play with words but was
defeated by my colleagues. After long discussions, even including a vote, the
current title was selected. I hope it carries the desired connotations.
I am happy to mention that this book has a predecessor. For the ISMP in
Amsterdamin1991,J.K.Lenstra,A.RinnoyKan,andA.Schrijvereditedthe
book History of Mathematical Programming: A Collection of Personal Remi-
niscences (CWIandNorth-Holland, 1991). Thisbookcontainsanoutstanding
collection of articles by the pioneers of optimization themselves on their own
achievements. Greatreading,trytogetacopyofthisbook! Thispresentbook
complements the ISMP 1991 volume; it is broader in scope and provides an
outside view.
·
Documenta Mathematica Extra Volume ISMP (2012) 3–5