Linear and Nonlinear Programming, 4th Edition: International Series in Operations Research & Management Science

547 Pages·2015·5.892 MB·English

by  David Luenberger, Yinyu Ye

International Series in
Operations Research & Management Science
David G. Luenberger
Yinyu Ye
Linear and
Nonlinear
Programming
Fourth Edition
www.it-ebooks.info
International Series in Operations
Research & Management Science
Volume228
SeriesEditor
CamilleC.Price
StephenF.AustinStateUniversity,TX,USA
AssociateSeriesEditor
JoeZhu
WorcesterPolytechnicInstitute,MA,USA
FoundingSeriesEditor
FrederickS.Hillier
StanfordUniversity,CA,USA
Moreinformationaboutthisseriesathttp://www.springer.com/series/6161
www.it-ebooks.info
www.it-ebooks.info
David G. Luenberger • Yinyu Ye
Linear and Nonlinear
Programming
Fourth Edition
123
www.it-ebooks.info
DavidG.Luenberger YinyuYe
DepartmentofManagementScience DepartmentofManagementScience
andEngineering andEngineering
StanfordUniversity StanfordUniversity
Stanford,CA,USA Stanford,CA,USA
ISSN0884-8289 ISSN2214-7934 (electronic)
InternationalSeriesinOperationsResearch&ManagementScience
ISBN978-3-319-18841-6 ISBN978-3-319-18842-3 (eBook)
DOI10.1007/978-3-319-18842-3
LibraryofCongressControlNumber:2015942692
SpringerChamHeidelbergNewYorkDordrechtLondon
©SpringerInternationalPublishingSwitzerland1973,1984(2003reprint),2008,2016
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
SpringerInternational PublishingAGSwitzerlandispartofSpringerScience+Business Media(www.
springer.com)
www.it-ebooks.info
ToSusan,Robert,Jill,andJenna;
Daisun,Fei,Tim,andKaylee
www.it-ebooks.info
www.it-ebooks.info
Preface
Thisbookisintendedasatextcoveringthecentralconceptsofpracticaloptimiza-
tion techniques. It is designed for either self-study by professionals or classroom
workattheundergraduateorgraduatelevelforstudentswhohaveatechnicalback-
groundinengineering,mathematics,orscience.Likethefieldofoptimizationitself,
whichinvolvesmanyclassicaldisciplines,thebookshouldbeusefultosystemana-
lysts,operationsresearchers,numericalanalysts,managementscientists,andother
specialists from the host of disciplines from which practical optimization appli-
cations are drawn. The prerequisites for convenientuse of the book are relatively
modest; the prime requirementbeing some familiarity with introductoryelements
of linear algebra. Certain sections and developments do assume some knowledge
ofmoreadvancedconceptsoflinearalgebra,suchaseigenvectoranalysis,orsome
backgroundinsetsofrealnumbers,butthetextisstructuredsothatthemainstream
ofthedevelopmentcanbefaithfullypursuedwithoutrelianceonthismoreadvanced
backgroundmaterial.
Althoughthebookcoversprimarilymaterialthatisnowfairlystandard,thisedi-
tion emphasizesmethodsthat are both state-of-the-artand popular.One major in-
sight is the connectionbetween the purely analytical character of an optimization
problem,expressedperhapsbypropertiesofthe necessaryconditions,and the be-
havior of algorithmsused to solve a problem.This was a major theme of the first
editionofthisbookandthefourtheditionexpandsandfurtherillustratesthisrela-
tionship.
Asintheearliereditions,thematerialinthisfourtheditionisorganizedintothree
separateparts.PartI isa self-containedintroductiontolinearprogramming,a key
component of optimization theory. The presentation in this part is fairly conven-
tional,coveringthemainelementsoftheunderlyingtheoryoflinearprogramming,
manyofthemosteffectivenumericalalgorithms,andmanyofitsimportantspecial
applications. Part II, which is independent of Part I, covers the theory of uncon-
strainedoptimization,includingbothderivationsoftheappropriateoptimalitycon-
ditionsandan introductionto basicalgorithms.Thispartofthe bookexploresthe
generalpropertiesofalgorithmsanddefinesvariousnotionsofconvergence.PartIII
vii
www.it-ebooks.info
viii Preface
extends the concepts developed in the second part to constrained optimization
problems.Exceptforafewisolatedsections,thispartisalsoindependentofPartI.
It is possible to go directly into Parts II and III omitting Part I, and, in fact, the
bookhasbeenusedinthiswayinmanyuniversities.Eachpartofthebookcontains
enoughmaterialtoformthe basisofa one-quartercourse.Ineitherclassroomuse
orforself-study,itisimportantnottooverlookthesuggestedexercisesattheendof
eachchapter.Theselectionsgenerallyincludeexercisesofacomputationalvariety
designedtotestone’sunderstandingofaparticularalgorithm,atheoreticalvariety
designed to test one’s understandingof a given theoreticaldevelopment,or of the
varietythatextendsthepresentationofthechaptertonewapplicationsortheoretical
areas.Oneshouldattemptatleastfourorfiveexercisesfromeachchapter.Inpro-
gressingthroughthebookitwouldbeunusualtoreadstraightthroughfromcover
tocover.Generally,onewillwishtoskiparound.Inordertofacilitatethismode,we
haveindicatedsectionsofaspecializedordigressivenaturewithanasterisk∗.
NewtothiseditionisaspecialChap.6devotedtoConicLinearProgramming,a
powerfulgeneralizationofLinearProgramming.Whiletheconstraintsetina nor-
mal linear program is defined by a finite number of linear inequalities of finite-
dimensionalvectorvariables,theconstraintsetinconiclinearprogrammingmaybe
defined, for example, as a linear combination of symmetric positive semi-definite
matricesofagivendimension.Indeed,manyconicstructuresarepossibleanduse-
ful in a variety of applications. It must be recognized, however, that conic linear
programmingisanadvancedtopic,requiringspecialstudy.
Anotherimportanttopicisan acceleratedsteepestdescentmethodthatexhibits
superiorconvergenceproperties,andforthisreason,hasbecomequitepopular.The
proofoftheconvergencepropertyforbothstandardandacceleratedsteepestdescent
methodsarepresentedinChap.8.
As the field of optimization advances, addressing greater complexity, treating
problemswithevermorevariables(asinBigDatasituations),rangingoverdiverse
applications. The field responds yo these challenges, developing new algorithms,
building effective software, and expanding overall theory. An example of a valu-
able new developmentis the work on big data problems. Surprisingly,coordinate
descent, with randomly selected coordinates at each step, is quite effective as ex-
plainedin Chap.8.As anotherexamplesomeproblemsareformulatedso thatthe
unknownscanbesplitintotwosubgroups,therearelinearconstraintsandtheobjec-
tivefunctionisseparablewithrespecttothetwogroupsofvariables.Theaugmented
Lagrangiancan be computedand it is naturalto use an alternatingseries method.
We discuss the alternating direction method with multipliers as a dual method in
Chap.14.Interestingly,thismethodisconvergentforwhenthenumberofpartition
groupsistwo,butnotforfinerpartitions.
We wish to thank the many students and researchers who over the years have
givenuscommentsconcerningthebookandthosewhoencouragedustocarryout
thisrevision.
Stanford,CA,USA D.G.Luenberger
Stanford,CA,USA Y.Ye
January2015
www.it-ebooks.info
Contents
1 Introduction................................................... 1
1.1 Optimization ............................................. 1
1.2 TypesofProblems ........................................ 2
1.3 SizeofProblems.......................................... 5
1.4 IterativeAlgorithmsandConvergence........................ 6
PartI LinearProgramming
2 BasicPropertiesofLinearPrograms ............................. 11
2.1 Introduction.............................................. 11
2.2 ExamplesofLinearProgrammingProblems................... 14
2.3 BasicSolutions ........................................... 19
2.4 TheFundamentalTheoremofLinearProgramming............. 20
2.5 RelationstoConvexity..................................... 23
2.6 Exercises ................................................ 27
3 TheSimplexMethod ........................................... 33
3.1 Pivots ................................................... 33
3.2 AdjacentExtremePoints ................................... 38
3.3 DeterminingaMinimumFeasibleSolution.................... 42
3.4 ComputationalProcedure:SimplexMethod ................... 45
3.5 FindingaBasicFeasibleSolution............................ 49
3.6 MatrixFormoftheSimplexMethod ......................... 54
3.7 SimplexMethodforTransportationProblems.................. 56
3.8 Decomposition ........................................... 68
3.9 Summary ................................................ 72
3.10 Exercises ................................................ 73
4 DualityandComplementarity ................................... 83
4.1 DualLinearPrograms ..................................... 83
4.2 TheDualityTheorem...................................... 86
ix
www.it-ebooks.info