COMBINATORIAL OPTIMIZATION LECTURE NOTES FOR CS(cid:2)(cid:3)(cid:2)(cid:4)OR(cid:2)(cid:5)(cid:6) Andrew V(cid:2) Goldberg Supported in part by NSF Presidential Young Investigator Grant CCR(cid:2)(cid:3)(cid:3)(cid:4)(cid:3)(cid:5)(cid:6)(cid:7) with matching funds from AT(cid:8)T(cid:9) DEC(cid:9) and (cid:10)M(cid:9) ONR Young Investigator Award N(cid:5)(cid:5)(cid:5)(cid:11)(cid:12)(cid:2)(cid:6)(cid:11)(cid:2)J(cid:2) (cid:11)(cid:3)(cid:4)(cid:4) (cid:0) Contents Preface (cid:2) Lecture I(cid:3) The Stable Marriage and Stable Roommates Problems (cid:4) (cid:5)(cid:3) The Stable Marriage Problem (cid:4) (cid:11)(cid:13)(cid:11)(cid:13) De(cid:14)nition of the Stable Marriage Problem (cid:3) (cid:11)(cid:13)(cid:15)(cid:13) The Gale(cid:2)Shapley Algorithm (cid:6) (cid:11)(cid:13)(cid:10)(cid:13) Correctness (cid:11)(cid:5) (cid:11)(cid:13)(cid:12)(cid:13) Male Optimality (cid:11)(cid:5) (cid:6)(cid:3) Introduction to Network Stability (cid:5)(cid:5) (cid:15)(cid:13)(cid:11)(cid:13) The Stable Roommates Problem (cid:11)(cid:11) (cid:15)(cid:13)(cid:15)(cid:13) Network De(cid:14)nitions (cid:11)(cid:11) (cid:15)(cid:13)(cid:10)(cid:13) The Stable Roommate Problem and Network Stability (cid:11)(cid:10) Lecture II(cid:3) The Stable Roommates Problem and Network Stability (cid:5)(cid:7) (cid:5)(cid:3) The Stable Roommates Problem and X(cid:8)Network Stability (cid:5)(cid:7) (cid:6)(cid:3) Adjacency(cid:8)Preserving Circuits and X(cid:8)Networks (cid:5)(cid:9) (cid:10)(cid:3) Network Stability and Simpli(cid:11)cation (cid:5)(cid:9) Lecture III(cid:3) The Maximum Flow Problem (cid:6)(cid:12) (cid:5)(cid:3) Network Flows (cid:6)(cid:12) (cid:11)(cid:13)(cid:11)(cid:13) Flows and Pseudo(cid:2)(cid:16)ows (cid:15)(cid:5) (cid:11)(cid:13)(cid:15)(cid:13) Cuts (cid:15)(cid:5) (cid:11)(cid:13)(cid:10)(cid:13) Residual Networks (cid:15)(cid:11) (cid:11)(cid:13)(cid:12)(cid:13) Augmenting Paths (cid:15)(cid:15) (cid:6)(cid:3) The Augmenting Path Algorithm (cid:6)(cid:6) (cid:2) CONTENTS (cid:3) (cid:10)(cid:3) The Decomposition Theorem (cid:6)(cid:13) (cid:13)(cid:3) The Capacity Scaling Algorithm (cid:6)(cid:9) Lecture IV(cid:3) The Push(cid:8)Relabel Algorithm (cid:6)(cid:4) (cid:5)(cid:3) Preliminaries (cid:6)(cid:4) (cid:6)(cid:3) The Algorithm (cid:6)(cid:14) (cid:15)(cid:13)(cid:11)(cid:13) A Rough Description of the Algorithm (cid:15)(cid:6) (cid:15)(cid:13)(cid:15)(cid:13) Initializing the Algorithm (cid:15)(cid:6) (cid:15)(cid:13)(cid:10)(cid:13) The Push Operation (cid:10)(cid:5) (cid:15)(cid:13)(cid:12)(cid:13) The Relabel Operation (cid:10)(cid:5) (cid:15)(cid:13)(cid:4)(cid:13) The Algorithm (cid:10)(cid:15) (cid:10)(cid:3) Analysis of the Algorithm (cid:10)(cid:6) Lecture V(cid:3) Implementations of Push(cid:8)Relabel Algorithms (cid:10)(cid:7) (cid:5)(cid:3) Simple Implementation (cid:10)(cid:7) (cid:11)(cid:13)(cid:11)(cid:13) Data Structures (cid:10)(cid:4) (cid:11)(cid:13)(cid:15)(cid:13) Discharge Subroutine (cid:10)(cid:17) (cid:6)(cid:3) Maximum Distance Discharge Algorithm (cid:10)(cid:2) (cid:15)(cid:13)(cid:11)(cid:13) Implementation of Maximum Distance Discharge Algorithm (cid:10)(cid:3) (cid:10)(cid:3) Implementation with Dynamic Trees (cid:10)(cid:14) (cid:10)(cid:13)(cid:11)(cid:13) Dynamic Trees Data Structure (cid:10)(cid:6) (cid:10)(cid:13)(cid:15)(cid:13) Send Operation (cid:10)(cid:6) Lecture VI(cid:3) Minimum Cycle Mean in a Digraph (cid:13)(cid:6) (cid:5)(cid:3) Problem Motivation (cid:13)(cid:6) (cid:6)(cid:3) Problem De(cid:11)nition (cid:13)(cid:6) (cid:10)(cid:3) Algorithm (cid:13)(cid:7) (cid:13)(cid:3) Correctness (cid:13)(cid:9) Lecture VII(cid:3) The Blocking Flow Algorithm and Bipartite Matching (cid:13)(cid:2) (cid:5)(cid:3) Dinitz(cid:15) Blocking Flow Algorithm (cid:13)(cid:2) (cid:6)(cid:3) Bipartite Matching (cid:13)(cid:14) Lecture VIII(cid:3) Two Phase Push(cid:8)Relabel Algorithm (cid:7)(cid:6) (cid:4) CONTENTS (cid:5)(cid:3) Two Phase Push(cid:8)Relabel Algorithm (cid:7)(cid:6) (cid:11)(cid:13)(cid:11)(cid:13) Phase I (cid:4)(cid:15) (cid:11)(cid:13)(cid:15)(cid:13) Phase II (cid:4)(cid:15) (cid:6)(cid:3) Correctness of the Two Phase Push(cid:8)Relabel Algorithm (cid:7)(cid:10) (cid:10)(cid:3) Some Remarks on the Two Phase Push(cid:8)Relabel Algorithm (cid:7)(cid:13) (cid:10)(cid:13)(cid:11)(cid:13) Exact Relabeling (cid:4)(cid:12) (cid:10)(cid:13)(cid:15)(cid:13) E(cid:18)cient practical implementation (cid:4)(cid:12) (cid:10)(cid:13)(cid:10)(cid:13) What is the best algorithm for the Maximum Flow Problem(cid:19) (cid:4)(cid:4) (cid:13)(cid:3) Bipartite Matching Problem (cid:7)(cid:7) Lecture IX(cid:3) The Minimum Cut Problem (cid:7)(cid:4) Minimum Cut Algorithms (cid:7)(cid:4) The Hao(cid:8)Orlin Algorithm (cid:7)(cid:14) Selecting the next sink (cid:9)(cid:5) Analysis of the algorithm (cid:9)(cid:6) Running Time (cid:17)(cid:10) Lecture X(cid:3) Minimum Cost Flow and Assignment Problems (cid:9)(cid:13) (cid:5)(cid:3) Minimum Cost Flows (cid:9)(cid:13) (cid:6)(cid:3) Prices and Reduced Costs (cid:9)(cid:9) (cid:10)(cid:3) Capacity Scaling (cid:9)(cid:2) (cid:13)(cid:3) Assignment Problem (cid:9)(cid:2) Lecture XI(cid:3) Cost Scaling for Min(cid:8)Cost Circulation Problem (cid:9)(cid:14) (cid:5)(cid:3) Cost Scaling (cid:9)(cid:14) Approximate Optimality (cid:17)(cid:6) (cid:6)(cid:3) The Algorithm (cid:2)(cid:12) (cid:10)(cid:3) Re(cid:11)nement Based on Push(cid:8)Relabel Operations (cid:2)(cid:12) Lecture XII(cid:3) Cost Scaling and Strongly Polynomial Algorithms (cid:2)(cid:13) (cid:5)(cid:3) Overview (cid:2)(cid:13) (cid:6)(cid:3) Cost(cid:8)Scaling (cid:2)(cid:13) (cid:15)(cid:13)(cid:11)(cid:13) Maximum Node Discharge and Wave Methods (cid:7)(cid:4) (cid:15)(cid:13)(cid:15)(cid:13) Running Times (cid:7)(cid:17) CONTENTS (cid:5) (cid:10)(cid:3) Strongly Polynomial Algorithms (cid:2)(cid:9) (cid:10)(cid:13)(cid:11)(cid:13) Fitting Price Functions and Tight Error Parameters (cid:7)(cid:17) (cid:10)(cid:13)(cid:15)(cid:13) Tight Error Parameters (cid:7)(cid:7) (cid:10)(cid:13)(cid:15)(cid:13)(cid:11)(cid:13) (cid:10)(cid:13)(cid:10)(cid:13) Strongly Polynomial Cost Scaling (cid:7)(cid:6) (cid:13)(cid:3) Minimum(cid:8)Mean Cycle(cid:8)Canceling (cid:4)(cid:12) Lecture XIII(cid:3) The Multicommodity Flow (cid:16) Concurrent Flow Problems (cid:4)(cid:10) (cid:5)(cid:3) The Multicommodity Flow Problem (cid:4)(cid:10) (cid:6)(cid:3) Concurrent Flow Problem (cid:4)(cid:13) (cid:15)(cid:13)(cid:11)(cid:13) De(cid:14)nitions and Notation (cid:3)(cid:12) (cid:15)(cid:13)(cid:15)(cid:13) Relaxed Optimality (cid:3)(cid:17) Lecture XIV(cid:3) The Concurrent Flow Algorithm (cid:4)(cid:4) (cid:5)(cid:3) Preliminaries (cid:4)(cid:4) (cid:11)(cid:13)(cid:11)(cid:13) Exponential Length Function (cid:3)(cid:3) (cid:6)(cid:3) The Algorithm (cid:4)(cid:14) (cid:15)(cid:13)(cid:11)(cid:13) Potential Function (cid:6)(cid:5) (cid:15)(cid:13)(cid:15)(cid:13) Initial Solution (cid:6)(cid:5) (cid:15)(cid:13)(cid:10)(cid:13) Improving the Current Solution (cid:6)(cid:5) (cid:15)(cid:13)(cid:12)(cid:13) Algorithm Analysis (cid:6)(cid:5) Lecture XV(cid:3) The General Matching Problem (cid:14)(cid:10) (cid:5)(cid:3) The Matching Problem (cid:14)(cid:10) (cid:6)(cid:3) Finding Augmenting Paths (cid:14)(cid:13) (cid:10)(cid:3) Blossom Shrinking Algorithm (cid:14)(cid:2) (cid:13)(cid:3) Complexity (cid:14)(cid:4) Lecture XVI(cid:3) The Travelling Salesman Problem (cid:5)(cid:12)(cid:12) (cid:5)(cid:3) NP(cid:8)Complete Problems (cid:5)(cid:12)(cid:12) (cid:6)(cid:3) Traveling Salesman Problem (cid:17) Preliminaries (cid:5)(cid:12)(cid:5) (cid:10)(cid:3) Approximation Algorithms for the TSP (cid:5)(cid:12)(cid:6) (cid:10)(cid:13)(cid:11)(cid:13) Preliminaries (cid:11)(cid:5)(cid:15) (cid:10)(cid:13)(cid:15)(cid:13) (cid:15)(cid:2)Approximation Tree Algorithm (cid:11)(cid:5)(cid:12) (cid:10)(cid:13)(cid:10)(cid:13) Christo(cid:14)des(cid:20) Algorithm (cid:11)(cid:5)(cid:4) (cid:6) CONTENTS (cid:13)(cid:3) Heuristic Algorithms (cid:5)(cid:12)(cid:9) Lecture XVII(cid:3) Branch and Bound Algorithms for TSP (cid:5)(cid:12)(cid:4) (cid:5)(cid:3) Heuristic Algorithms (cid:5)(cid:12)(cid:4) (cid:11)(cid:13)(cid:11)(cid:13) Tour Construction Heuristics (cid:11)(cid:5)(cid:3) (cid:11)(cid:13)(cid:15)(cid:13) Tour Improvement Heuristics (cid:11)(cid:5)(cid:3) (cid:6)(cid:3) Branch And Bound Method (cid:5)(cid:12)(cid:14) (cid:15)(cid:13)(cid:11)(cid:13) Integer Programming Formulation of TSP (cid:11)(cid:5)(cid:6) (cid:15)(cid:13)(cid:15)(cid:13) Branching (cid:11)(cid:5)(cid:6) (cid:15)(cid:13)(cid:10)(cid:13) Assignment Problem Relaxation (cid:11)(cid:11)(cid:5) (cid:15)(cid:13)(cid:12)(cid:13) Branching Rules (cid:11)(cid:11)(cid:11) Lecture XVIII(cid:3) Held(cid:8)Karp Lower Bound for TSP (cid:5)(cid:5)(cid:6) (cid:5)(cid:3) Problem Motivation (cid:5)(cid:5)(cid:6) (cid:6)(cid:3) Problem De(cid:11)nition (cid:5)(cid:5)(cid:10) (cid:10)(cid:3) Equivalence of TSP to an Integer Linear Program (cid:5)(cid:5)(cid:10) (cid:13)(cid:3) Branch and Bound (cid:5)(cid:5)(cid:10) (cid:7)(cid:3) Relaxation of TSP to Lagrangian dual (cid:5)(cid:5)(cid:13) (cid:4)(cid:13)(cid:11)(cid:13) (cid:11)(cid:2)Trees (cid:11)(cid:11)(cid:12) (cid:4)(cid:13)(cid:15)(cid:13) Penalty Function to Introduce Bias Towards Tours (cid:11)(cid:11)(cid:12) (cid:9)(cid:3) Revision of Relaxed Problem on (cid:5)(cid:8)Trees to Use Penalties (cid:5)(cid:5)(cid:7) (cid:17)(cid:13)(cid:11)(cid:13) Subgradient Optimization (cid:11)(cid:11)(cid:4) (cid:17)(cid:13)(cid:15)(cid:13) Held(cid:2)Karp lower bound is not strict (cid:11)(cid:11)(cid:17) (cid:17)(cid:13)(cid:10)(cid:13) A Branching Rule (cid:11)(cid:11)(cid:17) (cid:2)(cid:3) Misc (cid:5)(cid:5)(cid:2) Lecture XIX(cid:3) Interior Point Method for Linear Programming (cid:5)(cid:5)(cid:4) (cid:5)(cid:3) Overview (cid:5)(cid:5)(cid:4) (cid:6)(cid:3) Gonzaga(cid:15)s Algorithm (cid:5)(cid:5)(cid:14) Lecture XX(cid:3) Analysis of Gonzaga(cid:15)s Algorithm (cid:5)(cid:6)(cid:10) Bibliography (cid:5)(cid:6)(cid:2) PREFACE (cid:7) Preface This is a set of lecture notes for CS(cid:2)(cid:3)(cid:2)(cid:4)OR(cid:2)(cid:5)(cid:6)(cid:7) Combinatorial Optimization that I taught in Winter (cid:11)(cid:6)(cid:6)(cid:10)(cid:13) The notes were scribed by students in the class and reviewed me(cid:13) These notes are not ment to be in the polished form(cid:9) and probably contain some errors and ommisions(cid:13) In particular(cid:9) some references to the literature are missing(cid:13) The reader is directed to the relevant books and research papers for a formal treatment of the material(cid:13) These notes are intended as an aid in teaching and for studying advanced work in the area(cid:13) Much of the material covered in these notes is very recent and is available in recearch papers only(cid:13) I plan toupdate the notes each time the course is tought(cid:13) I would like toknow of any errors and ommisions so that they can be corrected in the future(cid:13) I would like to thank the students who took the course and scribed(cid:13) I would also like to thank my teaching assistants(cid:9) Sherry Listgarten and Robert Kennedy(cid:9) for their help with the class in general and with these notes in particular(cid:13) LECTURE I The Stable Marriage and Stable Roommates Problems Scribe(cid:2) Je(cid:3)ery D(cid:4) Oldham The stable marriage and network stability problems introduce combinatorial optimiza(cid:2) tion(cid:9) which involves discrete(cid:9) as opposed to continuous(cid:9) optimization(cid:13) (cid:5)(cid:3) The Stable Marriage Problem (cid:5)(cid:3)(cid:5)(cid:3) De(cid:11)nition of the Stable Marriage Problem(cid:3) The stable marriage problem consists of matching members of two di(cid:21)erent sets according to the members(cid:20) preferences for the other set(cid:20)s members(cid:13) For example(cid:9) medical school students serve as residents at hospitals after graduation(cid:13) Under the National Intern Matching Program which matches residents (cid:22)interns(cid:23) and hospitals(cid:9) the students submit rankings of preferred hospitals(cid:9) while hospitalssubmitrankingsofpreferredstudents(cid:13) Usingtherankings(cid:9)eachstudentisassigned to one hospital (cid:24)(cid:15)(cid:15)(cid:25)(cid:13) The input to the stable marriage problem consists of(cid:26) a set M of n males(cid:9) (cid:0) a set F of n females(cid:9) (cid:0) for each male and female(cid:9) a list of all the members of the opposite gender in order (cid:0) of preference(cid:13) (cid:0) Note that the size of the input is (cid:27)(cid:22)n (cid:23)(cid:9) since each of the (cid:15)n preference lists has length n(cid:13) Amarriage is aone(cid:2)to(cid:2)one mapping between males and females(cid:13) We wantto(cid:14)nd a stable marriage(cid:9) i(cid:8)e(cid:8)(cid:9) onein which thereis no pairmf such thatf prefersm toher currentpartner and m prefers f over his current partner(cid:13) Two partners are stable if they are matched in some stable marriage(cid:13) Example (cid:11)(cid:2) Consider the following preference lists for the sets M (cid:28) a(cid:2)b(cid:2)c and F (cid:28) f g A(cid:2)B(cid:2)C (cid:8) f g (cid:8) (cid:0)(cid:9) THE STABLE MARRIAGE PROBLEM (cid:10) a(cid:26) ABC A(cid:26) bac b(cid:26) ABC B(cid:26) acb c(cid:26) BCA C(cid:26) abc Male a prefers female A over females B and C(cid:9) and female A prefers male b(cid:8) The marriage aC(cid:2)bB(cid:2)cA is not stable because a prefers B over his current partner C and B prefers a f g over her current partner b(cid:8) The marriage aB(cid:2)bA(cid:2)cC is stable(cid:8) f g (cid:5)(cid:3)(cid:6)(cid:3) The Gale(cid:8)Shapley Algorithm(cid:3) The Gale(cid:2)Shapley algorithm (cid:24)(cid:11)(cid:10)(cid:25)(cid:9) given prefer(cid:2) ence lists for both males and females(cid:9) returns a stable marriage by having any unmatched male propose to females in the order of his preference lists until a female accepts(cid:13) (cid:22)See Figure I(cid:13)(cid:11)(cid:13)(cid:23) A female receiving a proposal accepts if she is not matched or she prefers the proposer over her current partner(cid:13) In the latter case(cid:9) the partner becomes single(cid:13) while there exists an unmarried male m do m proposes to the (cid:14)rst female f on his preference list to which he has not proposed if f is unmarried then match m and f else if f prefers m to her current match m(cid:0) then make m(cid:0) single match m and f else m remains single Figure I(cid:2)(cid:3)(cid:2) Gale(cid:2)Shapley Algorithm(cid:13) Example (cid:15)(cid:2) Suppose we desire a stable marriage for the preference lists a(cid:26) BAC A(cid:26) acb b(cid:26) BAC B(cid:26) bac c(cid:26) ACB C(cid:26) cab The algorithm yields the stable marriage aA(cid:2)bB(cid:2)cC by iterating these steps(cid:8) f g (cid:22)(cid:11)(cid:23) a proposes to B(cid:9) the (cid:10)rst female on his preference list(cid:9) and she accepts because she is single(cid:8) The set of matches is aB (cid:8) f g (cid:22)(cid:15)(cid:23) b proposes to B(cid:9) the (cid:10)rst female on his preference list(cid:9) and she accepts(cid:9) breaking her match with a(cid:9) because she prefers b over a(cid:8) The set of matches is bB (cid:8) f g (cid:22)(cid:10)(cid:23) Since a is now unmatched(cid:9) a proposes to A(cid:9) the next female on his list(cid:8) With A accepting because she is not matched(cid:9) the set of matches is aA(cid:2)bB (cid:8) f g (cid:22)(cid:12)(cid:23) Sincecisthe onlyunmatched male(cid:9)cproposestoA(cid:9) the(cid:10)rstfemaleonhispreference list(cid:8) She rejects the proposed match because she prefers her current partner a(cid:8) The set of matches is aA(cid:2)bB (cid:8) f g (cid:22)(cid:4)(cid:23) c proposes to C(cid:9) who accepts because she is not matched(cid:8) Since all males are now matched(cid:9) the algorithm terminates with matches aA(cid:2)bB(cid:2)cC (cid:8) f g The order of choosing unmatched males does not a(cid:11)ect the algorithm(cid:12)s result(cid:8)