Linear Programming: Foundations and Extensions (International Series in Operations Research & Management Science (196))

420 Pages·2013·5.232 MB·English

by  Robert J Vanderbei

International Series in
Operations Research & Management Science
Robert J. Vanderbei
Linear
Programming
Foundations and Extensions
Fourth Edition
International Series in Operations
Research & Management Science
Volume 196
SeriesEditor
FrederickS.Hillier
StanfordUniversity,CA,USA
SpecialEditorialConsultant
CamilleC.Price
StephenF.AustinStateUniversity,TX,USA
Forfurthervolumes:
http://www.springer.com/series/6161
Robert J. Vanderbei
Linear Programming
Foundations and Extensions
Fourth Edition
123
RobertJ.Vanderbei
DepartmentofOperationsResearch
andFinancialEngineering
PrincetonUniversity
Princeton,NewJersey,USA
ISSN0884-8289
ISBN978-1-4614-7629-0 ISBN978-1-4614-7630-6(eBook)
DOI10.1007/978-1-4614-7630-6
SpringerNewYorkHeidelbergDordrechtLondon
LibraryofCongressControlNumber:2013939593
©SpringerScience+BusinessMediaNewYork2014
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
andexecutedonacomputersystem, forexclusiveusebythepurchaserofthework. Duplicationof
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublica-
tiondoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromthe
relevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpub-
lication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforany
errorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespect
tothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
ToKrisadee,
MarisaandDiana
Preface
This book is about constrained optimization. It begins with a thorough treat-
mentoflinearprogrammingandproceedstoconvexanalysis,networkflows,integer
programming, quadratic programming, and convex optimization. Along the way,
dynamic programming and the linear complementarity problem are touched on as
well.
Thebookaimstobeafirstintroductiontothesubject. Specificexamplesand
concretealgorithmsprecedemoreabstracttopics. Nevertheless,topicscoveredare
developed in some depth, a large number of numerical examples are worked out
indetail,andmanyrecenttopicsareincluded,mostnotablyinterior-pointmethods.
Theexercisesattheendofeachchapterbothillustratethetheoryand,insomecases,
extendit.
Prerequisites. Thebookisdividedintofourparts. Thefirsttwopartsassume
a background only in linear algebra. For the last two parts, some knowledge of
multivariatecalculusisnecessary. Inparticular,thestudentshouldknowhowtouse
Lagrangemultiplierstosolvesimplecalculusproblemsin2and3dimensions.
Associated software. It is good to be able to solve small problems by hand,
buttheproblemsoneencountersinpracticearelarge,requiringacomputerfortheir
solution. Therefore,tofullyappreciatethesubject,oneneedstosolvelarge(prac-
tical) problems on a computer. An important feature of this book is that it comes
withsoftwareimplementingthemajoralgorithmsdescribedherein. Atthetimeof
writing,softwareforthefollowingfivealgorithmsisavailable:
Thetwo-phasesimplexmethodasshowninFigure6.1.
•
Theself-dualsimplexmethodasshowninFigure7.1.
•
Thepath-followingmethodasshowninFigure18.1.
•
Thehomogeneousself-dualmethodasshowninFigure22.1.
•
The long-step homogeneous self-dual method as described in Exercise
•
22.4.
The programs that implement these algorithms are written in C and can be
easily compiled on most hardware platforms. Students/instructors are encouraged
to install and compile these programs on their local hardware. Great pains have
beentakentomakethesourcecodefortheseprogramsreadable(seeAppendixA).
In particular, the names of the variables in the programs are consistent with the
notationofthisbook.
vii
viii PREFACE
Therearetwowaystoruntheseprograms. Thefirstistopreparetheinputin
astandardcomputer-fileformat, calledMPSformat, andtoruntheprogramusing
such a file as input. The advantage of this input format is that there is an archive
ofproblemsstoredinthisformat,calledtheNETLIBsuite,thatonecandownload
anduseimmediately(alinktotheNETLIBsuitecanbefoundatthewebsitemen-
tionedbelow). But,thisformatissomewhatarchaicand,inparticular,itisnoteasy
tocreatethesefilesbyhand. Therefore,theprogramscanalsoberunfromwithina
problemmodelingsystemcalledAMPL.AMPLallowsonetodescribemathemat-
ical programming problems using an easy to read, yet concise, algebraic notation.
ToruntheprogramswithinAMPL,onesimplytellsAMPLthenameofthesolver-
program before asking that a problem be solved. The text that describes AMPL,
Fourer et al. (1993) makes an excellent companion to this book. It includes a dis-
cussionofmanypracticallinearprogrammingproblems.Italsohaslotsofexercises
tohonethemodelingskillsofthestudent.
Severalinterestingcomputerprojectscanbesuggested. Hereareafewsugges-
tionsregardingthesimplexcodes:
Incorporate the partial pricing strategy (see Section 8.7) into the two-
•
phasesimplexmethodandcompareitwithfullpricing.
Incorporate the steepest-edge pivot rule (see Section 8.8) into the two-
•
phasesimplexmethodandcompareitwiththelargest-coefficientrule.
Modify the code for either variant of the simplex method so that it can
•
treatboundsandrangesimplicitly(seeChapter9),andcomparetheper-
formancewiththeexplicittreatmentofthesuppliedcodes.
Implement a “warm-start” capability so that the sensitivity analyses dis-
•
cussedinChapter7canbedone.
Extendthesimplexcodestobeabletohandleintegerprogrammingprob-
•
lemsusingthebranch-and-boundmethoddescribedinChapter23.
Asfortheinterior-pointcodes,onecouldtrysomeofthefollowingprojects:
Modify the code for the path-following algorithm so that it implements
•
the affine-scaling method (see Chapter 21), and then compare the two
methods.
Modifythecodeforthepath-followingmethodsothatitcantreatbounds
•
and ranges implicitly (see Section 20.3), and compare the performance
againsttheexplicittreatmentinthegivencode.
Modifythecodeforthepath-followingmethodtoimplementthehigher-
•
ordermethoddescribedinExercise18.5. Compare.
Extendthepath-followingcodetosolvequadraticprogrammingproblems
•
usingthealgorithmshowninFigure24.3.
Furtherextendthecodesothatitcansolveconvexoptimizationproblems
•
usingthealgorithmshowninFigure25.2.
And,perhapsthemostinterestingprojectofall:
Comparethesimplexcodesagainsttheinterior-pointcodeanddecidefor
•
yourselfwhichalgorithmisbetteronspecificfamiliesofproblems.
PREFACE ix
Thesoftwareimplementingthevariousalgorithmswasdevelopedusingconsistent
datastructuresandsomakingfaircomparisonsshouldbestraightforward. Thesoft-
warecanbedownloadedfromthefollowingwebsite:
http://www.princeton.edu/ rvdb/LPbook/
∼
If, in the future, further codes relating to this text are developed (for example, a
self-dualnetworksimplexcode),theywillbemadeavailablethroughthiswebsite.
Features. Herearesomeotherfeaturesthatdistinguishthisbookfromothers:
The development of the simplex method leads to Dantzig’s parametric
•
self-dual method. A randomized variant of this method is shown to be
immunetothetravailsofdegeneracy.
Thebookgivesabalancedtreatmenttoboththetraditionalsimplexmethod
•
and the newer interior-point methods. The notation and analysis is de-
veloped to be consistent across the methods. As a result, the self-dual
simplexmethodemergesasthevariantofthesimplexmethodwithmost
connectionstointerior-pointmethods.
Fromthebeginningandconsistentlythroughoutthebook,linearprogram-
•
mingproblemsareformulatedinsymmetricform. Byhighlighting sym-
metry throughout, it is hoped that the reader will more fully understand
andappreciatedualitytheory.
Byslightlychangingtheright-handsideintheKlee–Mintyproblem,we
•
areabletowritedownanexplicitdictionaryforeachvertexoftheKlee–
Minty problem and thereby uncover (as a homework problem) a simple,
elegantargumentwhytheKlee-Mintyproblemrequires2n 1pivotsto
−
solve.
The chapter on regression includes an analysis of the expected number
•
of pivots required by the self-dual variant of the simplex method. This
analysisissupportedbyanempiricalstudy.
Thereisanextensivetreatmentofmoderninterior-pointmethods,includ-
•
ingtheprimal–dualmethod, theaffine-scalingmethod, andtheself-dual
path-followingmethod.
Inadditiontothetraditionalapplications,whichcomemostlyfrombusi-
•
nessandeconomics, thebookfeaturesotherimportantapplicationssuch
astheoptimaldesignoftruss-likestructuresandL1-regression.
ExercisesontheWeb. Thereisalwaysaneedforfreshexercises. Hence,Ihave
createdandplantomaintainagrowingarchiveofexercisesspecificallycreatedfor
use in conjunction with this book. This archive is accessible from the book’s web
site:
http://www.princeton.edu/ rvdb/LPbook/
∼
Theproblemsinthearchivearearrangedaccordingtothechaptersofthisbookand
usenotationconsistentwiththatdevelopedherein.
Advice on solving the exercises. Some problems are routine while others are
fairlychallenging.Answerstosomeoftheproblemsaregivenatthebackofthebook.
x PREFACE
Ingeneral,theadvicegiventomebyLeonardGross(whenIwasastudent)should
helpevenonthehardproblems: followyournose.
Audience. Thisbookevolvedfromlecturenotesdevelopedformyintroductory
graduate course in linear programming as well as my upper-level undergraduate
course. A reasonable undergraduate syllabus would cover essentially all of Part 1
(Simplex Method and Duality), the first two chapters of Part 2 (Network Flows
and Applications), and the first chapter of Part 4 (Integer Programming). At the
graduatelevel,thesyllabusshoulddependonthepreparationofthestudents. Fora
well-preparedclass,onecouldcoverthematerialinParts1and2fairlyquicklyand
thenspendmoretimeonParts3(Interior-PointMethods)and4(Extensions).
Dependencies. In general, Parts 2 and 3 are completely independent of each
other. Bothdepend,however,onthematerialinPart1. ThefirstChapterinPart4
(IntegerProgramming)dependsonlyonmaterialfromPart1,whereastheremaining
chaptersbuildonPart3material.
Acknowledgments. MyinterestinlinearprogrammingwassparkedbyRobert
Garfinkel when we shared an office at Bell Labs. I would like to thank him for
hisconstantencouragement,advice,andsupport. Thisbookbenefitedgreatlyfrom
thethoughtfulcommentsandsuggestionsofDavidBernsteinandMichaelTodd. I
would also like to thank the following colleagues for their help: Ronny Ben-Tal,
LeslieHall,YoshiIkura,VictorKlee,IrvinLustig,AviMandelbaum,MarcMeke-
ton, Narcis Nabona, James Orlin, Andrzej Ruszczynski, and Henry Wolkowicz. I
would like to thank Gary Folven at Kluwer and Fred Hillier, the series editor, for
encouraging me to undertake this project. I would like to thank my students for
finding many typos and occasionally more serious errors: John Gilmartin, Jacinta
Warnie,StephenWoolbert,LuciaWu,andBingYang. MythankstoErhanC¸ınlar
for the many times he offered advice on questions of style. I hope this book re-
flectspositivelyonhisadvice. Finally,Iwouldliketoacknowledgethesupportof
the National Science Foundation and the Air Force Office of Scientific Research
for supporting me while writing this book. In a time of declining resources, I am
especiallygratefulfortheirsupport.
Princeton,NJ,USA RobertJ.Vanderbei
Preface to 2nd Edition
Forthe2ndedition,manynewexerciseshavebeenadded. AlsoIhaveworked
hardtodeveloponlinetoolstoaidinlearningthesimplexmethodanddualitytheory.
Theseonlinetoolscanbefoundonthebook’swebpage:
http://www.princeton.edu/ rvdb/LPbook/
∼
andarementionedatappropriateplacesinthetext.Besidesthelearningtools,Ihave
createdseveralonlineexercises. Theseexercisesuserandomlygeneratedproblems
and therefore represent a virtually unlimited collection of “routine” exercises that
can be used to test basic understanding. Pointers to these online exercises are in-
cludedintheexercisessectionsatappropriatepoints.
Someothernotablechangesinclude:
Thechapteronnetworkflowshasbeencompletelyrewritten. Hopefully,
•
thenewversionisanimprovementontheoriginal.
Two different fonts are now used to distinguish between the set of basic
•
indicesandthebasismatrix.
Thefirsteditionplacedgreatemphasisonthesymmetrybetweenthepri-
•
mal and the dual (the negative transpose property). The second edition
carriesthisfurtherwithadiscussionoftherelationshipbetweenthebasic
and nonbasic matrices B and N as they appear in the primal and in the
dual. We show that, even though these matrices differ (they even have
differentdimensions),B 1N inthedualisthenegativetransposeofthe
−
correspondingmatrixintheprimal.
Inthechaptersdevotedtothesimplexmethodinmatrixnotation,thecol-
•
lectionofvariablesz ,z ,...,z ,y ,y ,...,y wasreplaced,inthefirst
1 2 n 1 2 m
edition, withthesinglearrayofvariablesy ,y ,...,y . Thiscaused
1 2 n+m
great confusion as the variable y in the original notation was changed
i
to y in the new notation. For the second edition, I have changed the
n+i
notationforthesinglearraytoz ,z ,...,z .
1 2 n+m
Anumberoffigureshavebeenaddedtothechaptersonconvexanalysis
•
andonnetworkflowproblems.
The algorithm refered to as the primal–dual simplex method in the first
•
editionhasbeenrenamedtheparametricself-dualsimplexmethodinac-
cordancewithpriorstandardusage.
xi