Undergraduate convexity. From Fourier and Motzkin to Kuhn and Tucker

291 Pages·2013·2.511 MB·English

by  Niels Lauritzen

y.
nl
o
e
s
u
mal
on
co
entific.or pers
dsci5. F
worl11/1
w.1/
wwon 0
m O
oG
ed frDIE UNDERGRADUATE
adN
oA
nlS CONVEXITY
w@
o
DA
xity RNI From Fourier and Motzkin to Kuhn and Tucker
veO
nF
CoLI
uate F CA
dO
graY
UnderVERSIT
NI
U
y
b
8527_9789814412513_tp.indd 1 24/1/13 4:19 PM
TThhiiss ppaaggee iinntteennttiioonnaallllyy lleefftt bbllaannkk
y.
nl
o
e
s
u
mal
on
co
entific.or pers
dsci5. F
worl11/1
w.1/
wwon 0
m O
oG
ed frDIE
adN
oA
nlS
w@
o
DA
xity RNI
veO
nF
CoLI
uate F CA
dO
graY
UnderVERSIT
NI
U
y
b
y.
nl
o
e
s
u
mal
on
co
entific.or pers
dsci5. F
worl11/1
w.1/
wwon 0
m O
oG
ed frDIE
adN
oA UNDERGRADUATE
nlS
w@
o
DA
xity RNI CONVEXITY
veO
nF
CoLI
uate F CA
gradY O From Fourier and Motzkin to Kuhn and Tucker
UnderVERSIT
NI
U
y
b Niels Lauritzen
Aarhus University, Denmark
World Scientific
NEW JERSEY • LONDON • SINGAPORE • BEIJING • SHANGHAI • HONG KONG • TAIPEI • CHENNAI
8527_9789814412513_tp.indd 2 24/1/13 4:19 PM
Published by
World Scientific Publishing Co. Pte. Ltd.
5 Toh Tuck Link, Singapore 596224
USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601
UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE
y.
nl British Library Cataloguing-in-Publication Data
o
e A catalogue record for this book is available from the British Library.
s
u
mal
on
co
entific.or pers Cover image: Johan Ludvig William Valdemar Jensen (1859–1925).
worldsci11/15. F MPhaotthoegmraaptihc ibayn Vanildh etelmle pRhioengee re (ncgoiunreteers.y of the Royal Library, Copenhagen).
w.1/
wwon 0
m O
oG
ed frDIE
oadAN UNDERGRADUATE CONVEXITY
wnl@ S From Fourier and Motzkin to Kuhn and Tucker
o
DA Copyright © 2013 by World Scientific Publishing Co. Pte. Ltd.
vexity ORNI All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means,
nF electronic or mechanical, including photocopying, recording or any information storage and retrieval
uate CoF CALI system now known or to be invented, without written permission from the Publisher.
dO
graY
UnderVERSIT
NI For photocopying of material in this volume, please pay a copying fee through the Copyright
U
y Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to
b photocopy is not required from the publisher.
ISBN 978-981-4412-51-3
ISBN 978-981-4452-76-2 (pbk)
Printed in Singapore.
LaiFun - Undergraduate Convexity.pmd 1 1/25/2013, 9:41 AM
January18,2013 13:22 WorldScientificBook-9inx6in master
Preface
y.
nl
o
e
s
u
mal
on
co
entific.or pers Convexity is a key concept in modern mathematics with rich applications
dsci5. F in economics and optimization.
worl11/1 This book is a basic introduction to convexity based on several years
m www.O on 01/ oKfotnevaeckhsiengFtuhnektoinone-eqrua(rctoenrvceoxurfsuenscKtioonnsv)ektsoe Munædnegrdgerard(ucaotnevesxtusdeetsn)tsanidn
oG
aded frN DIE mpraetrheqemuiasittiecss,aerceonmoimniimcsalancdoncsoismtipnugteornlsycioenfcfiersattyAeaarrhcuosuUrsnesiveinrsictayl.cuTlhues
nloSA and linear algebra.
ow@ I have attempted to strike a balance between different approaches to
DA
xity RNI convexity in applied and pure mathematics. Compared to the former the
veO mathematicstakesafrontseat.Comparedtosomeofthelatter,akeypoint
nF
CoLI is that the ability to carry out computations is considered paramount and
uate F CA a crucial stepping stone to the understanding of abstract concepts e.g., the
dO
graY definition of a face of a convex set does not make much sense before it is
UnderVERSIT viewCehdapintetrhse1c–o6nttreexattocfosnevveexraslusbismetpslefreoxmamthpelebsaasnicdscoofmlipnueatartiinoenqsu.alities
NI to Minkowski’s theorem on separation of disjoint convex subsets by hyper-
U
y planes.Thebasicideahasbeentoemphasizepartoftherich“finite” theory
b
ofpolyhedrabeforeenteringintothe“infinite” theoryofclosedconvexsub-
sets.
Fourier-MotzkineliminationistolinearinequalitieswhatGaussianelim-
inationistolinearequations.Itseemsappropriatetobeginacourseoncon-
vexity by introducing this simple, yet powerful method. The prerequisites
are barely present. Still the first chapter contains substantial results such
asasimplealgorithmforlinearoptimizationandthefundamentaltheorem
that projections of polyhedra are themselves polyhedra.
v
January18,2013 13:22 WorldScientificBook-9inx6in master
vi Undergraduate Convexity —From Fourier and Motzkin to Kuhn and Tucker
Before introducing closed convex subsets, several basic definitions and
highlightsfromthepolyhedralworldaregiven:aconcisetreatmentofaffine
subspaces,facesofconvexsubsets,Bland’srulefromthesimplexalgorithm
as a tool for computing with the convex hull, faces of polyhedra, Farkas’s
lemma, steady states for Markov chains, duality in linear programming,
doubly stochastic matrices and the Birkhoff polytope.
The chapter Computations with polyhedra contains a treatment of two
y. important polyhedral algorithms: the double description method and the
nl
e o simplex algorithm. The double description method is related to Fourier-
s
mal u Motzkinelimination.Itisveryeasilyexplainedinanundergraduatecontext
on
co especially as a vehicle for computing the bounding half spaces of a convex
entific.or pers hull.
dsci5. F Thesimplexalgorithmsolveslinearoptimizationproblemsandissome-
w.worl1/11/1 wrehaasotnmiytsstheroiuoludswfroormk waelml.aItnhefmacatt,ictahlepfaemrspouecstimvea.thTehmearteiciisannoJoohbnvivoouns
wwon 0 Neumann never really believed it would perform in practice. The inven-
m O tor George B. Dantzig also searched for alternate methods for years before
oG
ed frDIE confronting experimental data from some of the world’s first computers:
adN the simplex algorithm performed amazingly well in practice. Only recently
oA
wnl@ S has a mathematical explanation for this phenomenon been given by Spiel-
o
DA man and Teng. Our treatment of the simplex algorithm and the simplex
xity RNI tableaudeviatesfromthestandardformandworkswiththepolyhedronin
veO
nF its defining space.
CoLI
uate F CA setsTchoemtersanasftiteironthteofitrhset“ficvoentcihnaupotuesr”s.thHeeorreyiotfisnopnro-pvoedlyhtheadtracllocsoendvecxonsvuebx-
dO
graY subsets serve as generalizations of polyhedra, since they coincide with ar-
UnderVERSIT bpietrrpalrayneinatetrasebcotiuonndsaorfyapffioninethoaflfascpoancveesx.TsuhbeseetxiisstepnroceveodfaansdupMpionrktoinwgskhiy’s-
NI
U theoremsoncompactconvexsubsetsandseparationofdisjointconvexsub-
y
b sets are given.
Chapters 7–10 treat convex functions from the basic theory of convex
functions of one variable with Jensen’s inequality to the Karush-Kuhn-
Tuckerconditions,dualoptimizationproblemsandanoutlineofaninterior
point algorithm for solving convex optimization problems in several vari-
ables. The setting is almost always the simplest. Great generality is fine
whenyouhavelivedwithasubjectforyears,butinanintroductorycourse
it tends to become a burden. You accomplish less by including more.
The main emphasis is on differentiable convex functions. Since under-
graduate knowledge of differentiability may vary, we give an almost com-
January18,2013 13:22 WorldScientificBook-9inx6in master
Preface vii
plete review of the theory of differentiability in one and several variables.
Theonly“general” resultonconvexfunctionsnotassumingdifferentiability
is the existence of the subgradient at a point.
An understanding of convex functions of several variables is impossi-
ble without knowledge of the finer points of linear algebra over the real
numbers. Introducing convex functions of several variables, we also give a
thoroughreviewofpositivesemidefinitematricesandreductionofsymmet-
y. ricmatrices.Thisimportantpartoflinearalgebraisrarelyfullyunderstood
nl
e o at an undergraduate level.
s
mal u The final chapter treats Convex optimization. The key elements are the
on
co Karush-Kuhn-Tucker conditions, how saddle points of the Lagrangian lead
entific.or pers to a dual optimization problem and finally an outline of an interior point
dsci5. F algorithm using bisection and the modified Newton method. Monographs
w.worl1/11/1 hcoanvteaibneeedninwtrriotdteunctioonntwhietshestimhrpeleeteoxpaimcsp.leWs.e only give a brief but self-
wwon 0
m O
oG Suggestions for teaching a one-semester course
ed frDIE
adN The amount of material included in this book exceeds a realistic plan
oA
nlS
w@ for a one-semester undergraduate course on convexity. I consider Fourier-
o
DA Motzkin elimination (Chapter 1), affine subspaces (Chapter 2), basics of
xity RNI convex subsets (Chapter 3), the foundational material on polyhedra in
veO
nF Chapter4,atasteofoneofthetwoalgorithmsinChapter5andclosedcon-
CoLI
duate OF CA vinexCshuabpsteetrss(7C–h1a0p.ter 6) as minimum along with almost all of the material
graY Theprogressionoflearningdependsontheproficiencyinlinearalgebra
UnderVERSIT aAnpdpecnaldciuxluAs..TInheAnpepceenssdaixryBbatshiecrceoinsceaprtesvfireowmoafnlainlyesairs aarlgeeibnrtarofdruomcedthine
NI
U point of view of linear equations leading to the rank of a matrix.
y
b Inmyview,atoorigidfocusontheabstractmathematicaldetailsbefore
tellingaboutexamplesandcomputationsisamajorsetbackintheteaching
of mathematics at all levels. Certainly the material in this book benefits
from being presented in a computational context with lots of examples.
Aarhus, December 2012
January18,2013 13:22 WorldScientificBook-9inx6in master
TThhiiss ppaaggee iinntteennttiioonnaallllyy lleefftt bbllaannkk
y.
nl
o
e
s
u
mal
on
co
entific.or pers
dsci5. F
worl11/1
w.1/
wwon 0
m O
oG
ed frDIE
adN
oA
nlS
w@
o
DA
xity RNI
veO
nF
CoLI
uate F CA
dO
graY
UnderVERSIT
NI
U
y
b
January18,2013 13:22 WorldScientificBook-9inx6in master
Acknowledgments
y.
nl
o
e
s
u
mal
on
co
entific.or pers Ifoarmveerxytruemseefulyl garnadtedfueltatioleTdagcoemBmaienAtnsdoenrseansaencdonJdesdprearftFufonrchthTishobmosoekn.
dsci5. F CommentsfromKentAndersen,JensCarstenJantzen,AndersNedergaard
worl11/1 Jensen and Markus Kiderlen also led to several improvements.
m www.O on 01/ an Iinatemresatninalgceobmrapiusttabtiyontaralinalignegbarand(aenndcocuonmtepruetdercso!)n.veAxsitysubchec,aIusheavoef
ed froDIEG bfoelnloewfiitnegdmimomreeknnseolwyleodvgereatbhlee pyeeaorpslef:roTmagienBsiagihtAfunldeexrspelna,nKateionntsAfnrodmerstehne,
adN
nloSA Kristoffer Arnsfelt Hansen, Peter Bro Miltersen, Marcel Bökstedt, Komei
ow@ Fukuda, Anders Nedergaard Jensen, Herbert Scarf, Jacob Schach Møller,
DA
xity RNI Andrew du Plessis, Henrik Stetkær, Bernd Sturmfels, Rekha Thomas, Jør-
veO gen Tornehave, Jørgen Vesterstrøm and Bent Ørsted.
nF
CoLI IamgratefultoJensCarstenJantzen,JesperLützenandTageGutmann
uate F CA Madsen for help in tracking down the venerable Jensen inequality postage
gradY O stampusedforseveralyearsbytheDepartmentofMathematicalSciencesat
UnderVERSIT Uhenriveexrpseitrytisoef Conoptehnehfaagsecnin.aAtilnsgo,htihsatonrkysotfocToinnvneexiHtyoffanKdjeolpdtsiemnizfoartisohna.ring
NI A very special thanks to the teaching assistants on Konvekse Mængder
U
y and Konvekse Funktioner: Lisbeth Laursen, Jonas Andersen Seebach,
b
MortenLeanderPetersen,RolfWognsen,LinneaJørgensenandDanZhang.
They pointed out several inaccuracies in my lecture notes along the way.
I am grateful to Kwong Lai Fun and Lakshmi Narayanan of World
Scientific for their skilled help in the production of this book.
Lars ‘daleif’ Madsen has been crucial in the technical typesetting with
his vast knowledge of LATEX and his usual careful attention to detail.
Finally, Helle and William deserve an abundance of gratitude for their
patience and genuine love.
ix