Mathematics of optimization: how to do things faster

353 Pages·2017·2.599 MB·English

by  Miller, Steven J

Pure and Applied
S ally 30
The UNDERGRADUATE TEXTS
SERIES
Mathematics
of Optimization:
How to do
Things Faster
Steven J. Miller
American Mathematical Society
Mathematics
of Optimization:
How to do
Things Faster

T h e UNDERPGRuAreD aUnAdT EA p pTlEiXedTS • 30
SERIES
Sally
Mathematics
of Optimization:
How to do
Things Faster
Steven J. Miller
American Mathematical Society
Providence, Rhode Island
EDITORIAL COMMITTEE
Gerald B. Folland (Chair) Steven J. Miller
Jamie Pommersheim Serge Tabachnikov
2010 Mathematics Subject Classification. Primary 46N10, 65K10, 90C05, 97M40, 58C30,
11Y16, 68Q25.
For additional informationand updates on this book, visit
www.ams.org/bookpages/amstext-30
Library of Congress Cataloging-in-Publication Data
Names: Miller,StevenJ.,1974-author.
Title: Mathematicsofoptimization: howtodothingsfaster/StevenJ.Miller.
Description: Providence, Rhode Island: AmericanMathematicalSociety, [2017]—Series: Pure
andappliedundergraduatetexts;volume30|Includesbibliographicalreferencesandindex.
Identifiers: LCCN2017029521|ISBN9781470441142(alk. paper)
Subjects: LCSH: Mathematical optimization–Problems, exercises, etc. | Operations research–
Problems, exercises, etc. | Management science–Problems, exercises, etc. | AMS: Functional
analysis–Miscellaneousapplicationsoffunctionalanalysis–Applicationsinoptimization,con-
vexanalysis,mathematicalprogramming,economics. msc|Numericalanalysis–Mathematical
programming, optimization and variational techniques – Optimization and variational tech-
niques. msc|Operationsresearch,mathematicalprogramming–Mathematicalprogramming
– Linear programming. msc|Mathematicseducation– Mathematicalmodeling,applications
ofmathematics–Operationsresearch,economics. msc|Globalanalysis,analysisonmanifolds
– Calculus on manifolds; nonlinear operators – Fixed point theorems on manifolds. msc —
Number theory – Computational number theory – Algorithms; complexity. msc | Computer
science–Theoryofcomputing–Analysisofalgorithmsandproblemcomplexity. msc
Classification: LCC QA402.5 .M5534 2017 | DDC 519.6–dc23 LC record available at https://
lccn.loc.gov/2017029521
Copying and reprinting. Individual readersofthispublication,andnonprofit librariesacting
for them, are permitted to make fair use of the material, such as to copy select pages for use
in teaching or research. Permission is granted to quote brief passages from this publication in
reviews,providedthecustomaryacknowledgmentofthesourceisgiven.
Republication,systematiccopying,ormultiplereproductionofanymaterialinthispublication
is permitted only under license from the American Mathematical Society. Permissions to reuse
portions of AMS publication content are handled by Copyright Clearance Center’s RightsLink(cid:2)
service. Formoreinformation,pleasevisit: http://www.ams.org/rightslink.
Sendreq[e-mail].
Excludedfromtheseprovisionsismaterialforwhichtheauthorholdscopyright. Insuchcases,
requestsforpermissiontoreuseorreprintmaterialshouldbeaddresseddirectlytotheauthor(s).
Copyrightownershipisindicatedonthecopyrightpage,oronthelowerright-handcornerofthe
firstpageofeacharticlewithinproceedingsvolumes.
(cid:2)c 2017bytheAmericanMathematicalSociety. Allrightsreserved.
TheAmericanMathematicalSocietyretainsallrights
exceptthosegrantedtotheUnitedStatesGovernment.
PrintedintheUnitedStatesofAmerica.
(cid:2)∞ Thepaperusedinthisbookisacid-freeandfallswithintheguidelines
establishedtoensurepermanenceanddurability.
VisittheAMShomepageathttp://www.ams.org/
10987654321 222120191817
To my three J’s
(my brother Jeff, his wife Jackie, and their daughter Justine),
especially to my brother the engineer
for many conversations over the years on efficiency,
as well as a willingness to never grow up
and still play with Lego bricks with me!

Contents
Acknowledgements xiii
Preface xv
Course Outlines xix
Part 1. Classical Algorithms
Chapter 1. Efficient Multiplication, I 3
1.1. Introduction 3
1.2. Babylonian Multiplication 4
1.3. Horner’s Algorithm 5
1.4. Fast Multiplication 6
1.5. Strassen’s Algorithm 8
1.6. Eigenvalues, Eigenvectors and the Fibonacci Numbers 9
1.7. Exercises 11
Chapter 2. Efficient Multiplication, II 21
2.1. Binomial Coefficients 21
2.2. Pascal’s Triangle 22
2.3. Dimension 24
2.4. From the Pascal to the Sierpinski Triangle 26
2.5. The Euclidean Algorithm 28
2.6. Exercises 35
Part 2. Introduction to Linear Programming
Chapter 3. Introduction to Linear Programming 47
vii
viii Contents
3.1. Linear Algebra 48
3.2. Finding Solutions 50
3.3. Calculus Review: Local versus Global 51
3.4. An Introduction to the Diet Problem 54
3.5. Solving the Diet Problem 55
3.6. Applications of the Diet Problem 59
3.7. Exercises 60
Chapter 4. The Canonical Linear Programming Problem 67
4.1. Real Analysis Review 68
4.2. Canonical Forms and Quadratic Equations 70
4.3. Canonical Forms in Linear Programming: Statement 71
4.4. Canonical Forms in Linear Programming: Conversion 73
4.5. The Diet Problem: Round 2 75
4.6. A Short Theoretical Aside: Strict Inequalities 76
4.7. Canonical is Not Always Best 77
4.8. The Oil Problem 78
4.9. Exercises 79
Chapter 5. Symmetries and Dualities 83
5.1. Tic-Tac-Toe and a Chess Problem 83
5.2. Duality and Linear Programming 87
5.3. Appendix: Fun Versions of Tic-Tac-Toe 88
5.4. Exercises 90
Chapter 6. Basic Feasible and Basic Optimal Solutions 95
6.1. Review of Linear Independence 95
6.2. Basic Feasible and Basic Optimal Solutions 96
6.3. Properties of Basic Feasible Solutions 97
6.4. Optimal and Basic Optimal Solutions 99
6.5. Efficiency and Euclid’s Prime Theorem 100
6.6. Exercises 102
Chapter 7. The Simplex Method 107
7.1. The Simplex Method: Preliminary Assumptions 107
7.2. The Simplex Method: Statement 108
7.3. Phase II implies Phase I 109
7.4. Phase II of the Simplex Method 110
7.5. Run-time of the Simplex Method 113
7.6. Efficient Sorting 113
7.7. Exercises 115
Contents ix
Part 3. Advanced Linear Programming
Chapter 8. Integer Programming 121
8.1. The Movie Theater Problem 122
8.2. Binary Indicator Variables 125
8.3. Logical Statements 126
8.4. Truncation, Extrema and Absolute Values 128
8.5. Linearizing Quadratic Expressions 130
8.6. The Law of the Hammer and Sudoku 131
8.7. Bus Route Example 134
8.8. Exercises 135
Chapter 9. Integer Optimization 143
9.1. Maximizing a Product 143
9.2. The Knapsack Problem 146
9.3. Solving Integer Programs: Branch and Bound 147
9.4. Exercises 150
Chapter 10. Multi-Objective and Quadratic Programming 153
10.1. Multi-Objective Linear Programming 153
10.2. Quadratic Programming 154
10.3. Example: Quadratic Objective Function 155
10.4. Removing Quadratic (and Higher Order) Terms in Constraints 156
10.5. Summary 157
10.6. Exercises 157
Chapter 11. The Traveling Salesman Problem 161
11.1. Integer Linear Programming Version of the TSP 161
11.2. Greedy Algorithm to the TSP 164
11.3. The Insertion Algorithm 165
11.4. Sub-problems Method 166
11.5. Exercises 167
Chapter 12. Introduction to Stochastic Linear Programming 169
12.1. Deterministic and Stochastic Oil Problems 170
12.2. Expected Value approach 171
12.3. Recourse Approach 172
12.4. Probabilistic Constraints 174
12.5. Exercises 175
Part 4. Fixed Point Theorems
Chapter 13. Introduction to Fixed Point Theorems 179