Linear Optimization: The Simplex Workbook

272 Pages·2010·0.32 MB·English

by  Glenn Hurlbert (auth.)

Undergraduate Texts in Mathematics
Editorial Board
S. Axler
K.A. Ribet
For other titles Published in this series, go to
http://www.springer.com/series/666
Glenn H. Hurlbert
Linear Optimization
The Simplex Workbook
123
Glenn H. Hurlbert
School of Mathematical and Statistical Sciences
Arizona State University
Tempe, AZ 85287-1804
USA
[email protected]
Editorial Board:
S. Axler K. A. Ribet
Mathematics Department Mathematics Department
San Francisco State University University of California at Berkeley
San Francisco, CA 94132 Berkeley, CA 94720
USA USA
[email protected] [email protected]
ISBN 978-0-387-79147-0 e-ISBN 978-0-387-79148-7
DOI 10.1007/978-0-387-79148-7
Springer New York Dordrecht Heidelberg London
Library of Congress Control Number: 2009936080
Mathematics Subject Classification (2000): Primary: 90-01, Secondary: 05-01, 15-01, 52-01, 91-01
©SpringerScience+BusinessMedia,LLC2010
Allrightsreserved. Thisworkmaynotbetranslatedorcopiedinwholeorinpartwithoutthewritten
permissionofthepublisher(SpringerScience+BusinessMedia,LLC,233SpringStreet,NewYork,NY
10013,USA),exceptforbriefexcerptsinconnectionwithreviewsorscholarlyanalysis.Useinconnection
withanyformofinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdevelopedisforbidden.
Theuseinthispublicationoftradenames,trademarks,servicemarks,andsimilarterms,eveniftheyare
notidentifiedassuch,isnottobetakenasanexpressionofopinionastowhetherornottheyaresubject
toproprietaryrights.
Printedonacid-freepaper
Springer is part of Springer Science+Business Media (www.springer.com)
Dedication
To Karen, for her constant love and support.
To Calvin and Kate, for patiently waiting for Daddy to play.
To Uncle Frank, who would have loved to read it.
To Mom, my teaching model.
Preface
The Subject
A little explanation is in order for our choice of the title Linear Opti-
mization1 (andcorrespondingterminology)forwhathastraditionallybeen
called Linear Programming. The wordprogramming in this context can be
confusing and/or misleading to students. Linear programming problems
are referred to as optimization problems but the general term linear pro-
gramming remains. This can cause people unfamiliar with the subject to
think that it is about programming in the sense of writing computer code.
It isn’t. This workbook is about the beautiful mathematics underlying the
ideas of optimizing linear functions subject to linear constraints and the
algorithms to solve such problems. In particular, much of what we dis-
cuss is the mathematics of Simplex Algorithm for solving such problems,
developed by George Dantzig in the late 1940s.
The word program in linear programmingis a historicalartifact. When
DantzigfirstdevelopedtheSimplexAlgorithmtosolvewhatarenowcalled
linear programming problems, his initial model was a class of resource al-
location problems to be solvedfor the U.S. Air Force. The decisions about
theallocationswerecalled‘Programs’bytheAirForce,andhencetheterm.
Dantzig’s article2 is a fascinating description of the origins of this subject
written by the person who originated many of the ideas. Included is a de-
scription of how Tjalling Koopmans (who won a Nobel Prize in economics
for his workindecisionscience)suggestedshorteningDantzig’sdescription
‘programming in a linear structure’ to ‘linear programming’during a walk
onthebeachwithDantzig. Alsoincludedisanotethat,atthetime,‘code’
was the word used for computer instructions and not ‘program’.
To be clear to potential and current students that this is a mathemat-
ics course requiring a background in writing proofs and not a computer
coding class, we prefer the terminology linear optimization. We do look at
computer algorithms but focus on the underlying mathematics. A small
amount of computer coding (for example, simple MAPLE) programs will
be very useful, but writing code is not the central purpose of the course.
The shorthand LP has been used to refer to both the general subject of
1I’mnotthatoriginal—thereareatleastadozenbooks thatusethisterminology.
2G.Dantzig,LinearProgramming,Operations Research50(2002), 42–47.
viii Preface
linear programming as well as specific instances of linear programs. To
distinguish, in our notation, LO (linear optimization) refers to the general
class ofoptimizing linear functions subjectto linear constraintswhile LOP
(linear optimization problem) refers to specific instances of such problems.
Furthermore, using the optimization term brings the subject in line with
other, closelyrelatedfields thatareincreasinglycalledOptimization(Non-
linear, Quadratic, Convex, Integer, Combinatorial).
The Simplex Algorithm is the focus of study in this book. In par-
ticular, we do not discuss Karmarkar’s Algorithm or Khachian’s Ellipsoid
Algorithmormoregeneralinterior-pointapproachesto solvingLOPs. The
main reason for this is that, as I hope you’ll experience, the Simplex Al-
gorithm leads to richer connections with linear algebra, geometry, combi-
natorics, game theory, probability, and graph theory. Furthermore, in the
post-optimality analysis that occurs in economic modeling and in Integer
Optimization, the Simplex Algorithm plays a central role.
Terminology
Besides the usage of LOP and ILOP, we introduce other quirks into
the language, mostly for handiness and consistency and occasionally for
fun. For example, we discuss four kinds of linear combinations, based on
whetherornottheextraaffineandconicconditionshold,soitmakessense
to use the similar notations lspan, aspan, nspan, and vspan for linear,
affine,conic,andconvex(bothaffineandconic)combinations,respectively.
In particular, lspan makes more sense in this scheme than does span, the
more common term found in linear algebra texts. Geometric hulls get the
same treatment, with lhull, ahull, nhull, and vhull, respectively. For fun
we use the term FLOP (Fractional LOP) when we need to distinguish a
LOP from being an ILOP. Indeed, the first step in solving an ILOP is to
relax the integer constraints to allow for rationals and find the resulting
floptimal solution, which is used as a first approximation to the iloptimal
solution. The term BLOP refers to an ILOP whose variables are binary
(either 0 or 1).3 Also, when we discuss game theory, we talk about the
GLOP (Game LOP) derived from a game. We don’t go too much farther
down the self-parody road — hopefully there is no SLOP in the book.
Ithinkwe’realsothefirstLObooktousethetermparameterinplaceof
nonbasicvariable. Thatmustbeworthsomekindofaward,right? Actually,
I stole it from virtually every linear algebra text ever written.
Chapter Flow
Outlandish as it may seem, someone studying from this text will need
to startwithChapter 1,followedby Chapter2. After that, therearemany
directions of travel.
If and when you wish to study geometry, you’ll want to learn Chapter
3 before Chapter 8, but you can pretty much learn them whenever you
3Thusitisquitenaturalformanyacademics tobeinterestedinBO.
Preface ix
want. No other chapter uses Chapter 3 explicitly, although it does offer
very beneficial intuition that permeates almost every other chapter. This
is why it is placed so early. Chapter 8, however, isn’t necessary for much
(unless one continues on to study graduate level optimization), but does
use material from Chapter 7 (and Section 8.4 needs Chapter 6) — and is
kind of fun.
Chapter 4 is really the heart of the course. Everything feeds off of
duality. This is why we derivethe dualimmediately inSection1.1. Spend-
ing extra time here pays dividends later, as everything thereafter depends
critically upon it.
The material of Chapter 5 is useful for learning the tip of the how-
this-is-done-in-the-real-world iceberg. The ability to state and work with
everythinginmatrixformis averyusefulskillingeneral,andinparticular
comes in handy in Chapter 12. Thus it does not need to be studied before
any other chapter (if at all, as it is not essential material otherwise — in
fact, Chapter 5 is only crucial to proving Theorem 12.1.4).
Chapter 6 puts Chapter 4 in general context, and is required for all
subsequent chapters.
Chapter 7 contains materialthat is necessaryfor Chapter 8. Chapter 9
isnotneededbyanyonewhodoesn’twanttohaveagoodtime. Chapter10
isrequiredbyChapter11,andSection12.4iskeyforChapter13. Chapters
7–13 offer the greatest flexibility for studying your favorite topics within a
semester’s time. Of course, you could slow down, spend extra time on the
exercises, and complete the whole book in two semesters. Have at it!
A visual description of the above dependencies is given below.
Book Format and Usage
It is notdifficult to notice that this text is different frommost, and not
just because I can’t resist even the lamest of jokes,4 so it may be worth
somediscussiononwhythis isso,whatbenefits thismayhaveforyou,and
how best to take advantage of the new format.
First and foremost, a great deal of information is missing, information
that typically is included inmathematics texts. Havingspentmost ofyour
4Mostoftheserequiremyagetounderstandanyway.
x Preface
life reading from such texts, you may be used to being fed facts and algo-
rithms, and well used to memorizing them in order to reproduce them on
exams. ButIbelievethatyouarecapableofmuchmore: ofderivingresults,
in fact, discovering them through experimentation, of making conjectures,
of proving theorems, of solving problems and checking them yourself, and
of asking creative questions. That’s what this format is all about, giving
yourprofessortheopportunitytoleadyouthroughthekindsofexperiences
that will develop your skills in each of these areas, helping you become a
highly critical thinking machine,5 able to wrestle with complex problems
in all areas of society, rather than just someone who went to college and
remembers a few math facts.
You may find that your classroom environment may also differ from
the norm. Many professors who use this will ask their students to par-
ticipate more in discussion, rather than simply listen to lectures. Some
may even ask their students to make daily presentations of the theorems
they’ve proven, the exercises and workouts they’ve solved, and the algo-
rithms they’ve written, in order to create an environment in which the
students become responsible for their ownlearning,questioning eachother
for understanding, while the professor acts as a facilitator. Such an envi-
ronment may be upsettingly abnormal to you initially, but I promise you
will warm to it (indeed, embrace it) in time.
This form of discourse centers more on the learning than the teaching,
and those who engage in it deeply are affected (infected?) for life. My
hope is that you are enticed enough, not only by the material, but by the
interesting problems,challenging questions,and yourprofessor’sinvitation
to question, challenge, wonder, experiment, guess, and argue, to throw
your energies in this new direction so as to be stirred to the point that
it transforms the way you think about everything, from mathematics and
science, to politics and religion, to sports and fine arts. Be inquisitive, be
skeptical, be critical, be creative, and keep thinking.
Keep in mind, this isn’t some crazy, new, experimental pedagogy. This
approachis asoldasit gets,predatingallforms offormalclassroomteach-
ing. It has come to be known by many names through the years: the So-
craticMethod,Discovery-BasedTeaching,theMooreMethod,andInquiry-
Based Learning, among others. There are maybe two central tenets that
identify the philosophy.
• A thing isn’t true because someone says so or because it’s written in
a book, but because it is reasoned to be true.
• One doesn’t master something by hearing or seeing it, but by doing
it.
Sowhatelsecanyoudooutsideoftheclassroominordertomasterthis
material? Some students find that keeping a journal, separate from their
class notes, that holds all their proofs and solutions, can be quite useful.
5Youcanevenwearacape: leaptallbuildings,etc.
Preface xi
This is especiallytrueifoneuses awordprocessor(LATEXiscertainlybest,
whether using AMSTeX on Linux, MiKTeX through WinEdt or Scientific
Word on Windows, or even with TeXShop on MacOS, although Microsoft
Word,withitsEquationEditorshouldsuffice),sincetheworkcancontinue
to be editedandorganizedto followthe text. Also,dothe workoutsasyou
read along, such as the very first one:
Workout 0 What pattern is there regarding Section 3 of each chapter?
Thepointofdoingthemasyougoispartlytohelpyoulearnthematerial
by making sure you read the stuff (I can remember undergraduate books
that I wouldn’t read at all, instead just solving the exercises required for
homeworkbymimickingtheexamples—whatkindofcomprehensiondoes
that foster?), and partly to help you learn the process of experimenting,
conjecturing, clarifying,strategizing,proofwriting, andgeneralizing. That
willserveyoulongafteryouforgettheComplementarySlacknessTheorem.
On the other hand, you can simply distract yourself with the trivia
contest that floats throughout the book, signified by the ◦ in the margins.
Of course, you have to swear not to Google anything if you want to win.
Index
Just a small note about the Index. I believe that index entries should
signifywhat’smemorable,notimportant—importanceshouldbeindicated
by the page references instead. So in this era of inclusivity, I’ve thrown in
the kitchen sink. The main reason for this is that I don’t know how your
memory works. It doesn’t help you if you can’t find tractor pull because
it’s only listed under theological tractor pull and you couldn’t remember
the obvious connection to theology. So the index is fattened by including
variouspermutationsofwords. Also,eventhoughtractorpullisnotcentral
to the theory of linear optimization, in 25 years when you’re trying to
show your kids a nice example of how the theory works and all you can
remember is that awesome example with the tractor pull, you’ll thank me.
Plus,thisaddsanextrapagetothebook,whichincreasesitscostby.037c/.
With about 2.7 million readers annually, my 1% commission generates an
extra deluxe cheeseburger per year, which in turn helps me satisfy the
requirements of Problem 1.1.1 for that day. While the entries are many,
I did make an effort to restrict the page references of the most common
terms to their most important instances.
Withregardtothefontsyou’llencounter,theorems(intheirfullnames)
areinitalics,andpagenumbersthatrefertodefinitionsareinbold. (You’ll
notice that theorems and definitions find their way into the margins for
easy location.) Italicized page numbers denote appearances in theorems,
while sans serif numbers signal inclusion in workouts and exercises, and
romanfonts cite regularoccurrencesof the term. With regardto the order
of terms, mathematical symbols come first (numbers, then capital letters,
then lower case letters), followed by As,6 and then standard words.
6ShortforAcronyms.