Lecture Notes on Approximation Algorithms Volume I - The Stanford

134 Pages·1998·0.64 MB·English

Lecture Notes on Approximation
Algorithms { Volume I
(cid:3)
Rajeev Motwani
Department of Computer Science
Stanford University
Stanford, CA 94305-2140.
(cid:3)PartofthisworkwassupportedbyNSFGrantCCR-9010517,andgrants
from Mitsubishi and OTL.
Abstract
These lecturenotes are based on the course CS351 (Dept. of Computer
Science,Stanford University)o(cid:11)ered during the academicyear1991-92.
The notes below correspond to the (cid:12)rst half of the course. The second
half consists of topics such as MAX SNP, cliques, and colorings, as well
as morespecializedmaterialcoveringtopicssuchasgeometric problems,
Steiner trees and multicommodity (cid:13)ows. The second half is being re-
vised to incorporate the implicationsof recent results in approximation
algorithms and the complexity of approximation problems. Please let
me know if you would like to be on the mailing list for the second half.
Comments, criticisms and corrections are welcome, please send them
by electronic mail to [email protected].
Contents
1 Introduction 5
1.1 Preliminaries and Basic De(cid:12)nitions : : : : : : : : : : : : 7
1.2 Absolute Performance Guarantees : : : : : : : : : : : : : 10
1.2.1 Absolute Approximation Algorithms : : : : : : : 11
1.2.2 Negative Results for Absolute Approximation : : 12
1.3 Relative Performance Guarantees : : : : : : : : : : : : : 15
1.3.1 Multiprocessor Scheduling : : : : : : : : : : : : : 16
1.3.2 Bin Packing : : : : : : : : : : : : : : : : : : : : : 19
1.3.3 The Traveling Salesman Problem : : : : : : : : : 22
1.3.4 Negative Results for Relative Approximation : : : 25
1.4 Discussion : : : : : : : : : : : : : : : : : : : : : : : : : : 30
Problems : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 31
2 Approximation Schemes 33
2.1 Approximation Scheme for Scheduling : : : : : : : : : : 35
2.2 Approximation Scheme for Knapsack : : : : : : : : : : : 37
2.3 Fully Polynomial Approximation Schemes : : : : : : : : 41
2.4 Pseudo-Polynomial Algorithms : : : : : : : : : : : : : : 43
2.5 Strong -completeness and FPAS : : : : : : : : : : : : 48
NP
1
CONTENTS Page 2
2.6 Discussion : : : : : : : : : : : : : : : : : : : : : : : : : : 52
Problems : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 52
3 Bin Packing 53
3.1 Asymptotic Approximation Scheme : : : : : : : : : : : : 55
3.1.1 Restricted Bin Packing : : : : : : : : : : : : : : : 56
3.1.2 Eliminating Small Items : : : : : : : : : : : : : : 59
3.1.3 Linear Grouping : : : : : : : : : : : : : : : : : : 60
3.1.4 APAS for Bin Packing : : : : : : : : : : : : : : : 62
3.2 Asymtotic Fully Polynomial Scheme : : : : : : : : : : : : 64
3.2.1 Fractional Bin Packing and Rounding : : : : : : : 64
3.2.2 AFPAS for Bin Packing : : : : : : : : : : : : : : 69
3.3 Near-Absolute Approximation : : : : : : : : : : : : : : : 71
3.4 Discussion : : : : : : : : : : : : : : : : : : : : : : : : : : 76
Problems : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 76
4 Vertex Cover and Set Cover 79
4.1 Approximating Vertex Cover : : : : : : : : : : : : : : : : 82
4.2 Approximating Weighted Vertex Cover : : : : : : : : : : 89
4.2.1 A Randomized Approximation Algorithm: : : : : 91
4.2.2 The Nemhauser Trotter Algorithm : : : : : : : : 96
4.2.3 Clarkson’s Algorithm : : : : : : : : : : : : : : : : 100
4.3 Improved Vertex Cover Approximations: : : : : : : : : : 104
4.3.1 The Nemhauser-Trotter Algorithm Revisited : : : 105
4.3.2 A Local Ratio Theorem : : : : : : : : : : : : : : 108
4.3.3 An Algorithm for Graphs Without Small Odd
Cycles : : : : : : : : : : : : : : : : : : : : : : : : 114
4.3.4 The Overall Algorithm : : : : : : : : : : : : : : : 117
CONTENTS Page 3
4.4 Approximating Set Cover: : : : : : : : : : : : : : : : : : 119
4.5 Discussion : : : : : : : : : : : : : : : : : : : : : : : : : : 124
Problems : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 124
5 Bibliography 127
CONTENTS Page 4
Chapter 1
Introduction
Summary: The notion of approximation algorithm is introduced and
some motivation is provided for the issues to be considered later. Ba-
sic notation and some elementary concepts from complexity theory are
presented. Two measures of goodness for approximation algorithms are
contrasted: absolute and relative. Both positive and negative results are
described for the following problems: scheduling, bin packing, and the
traveling salesman problem.
Alarge numberof (ifnot, mostof) the optimizationproblemswhich
are required to be solved in practice are -hard. Complexity theory
NP
tellsusthatitisimpossibleto(cid:12)nde(cid:14)cientalgorithmsforsuchproblems
unless = , and this is very unlikely to be true. This does not
P NP
obviatetheneedforsolvingtheseproblems. Observethat -hardness
NP
only means that, if = , we cannot (cid:12)nd algorithms which will (cid:12)nd
P 6 NP
exactly the optimal solution to all instances of the problem in time
which is polynomial in the size of the input. If we relax this rather
stringent requirement, it may still be possible to solve the problem
reasonably well.
There are three possibilities for relaxing the requirements outlined
above to consider a problem well-solved in practice:
[Super-polynomial time heuristics.] We may no longer re-
(cid:15)
quire that the problem be solved in polynomial time. In some
5
CHAPTER 1. INTRODUCTION Page 6
cases there are algorithms whichare just barely super-polynomial
and run reasonably fast in practice. There are techniques(heuris-
tics) such as branch-and-bound or dynamic programming which
are useful from this point of view. For example, the Knapsack
problem is -complete but it is considered \easy" since there
NP
is a \pseudo-polynomial" time algorithm for it. (We shall say
more about this in Chapter 2.) A problem with this approach
is that very few problems are susceptible to such techniques and
for most -hard problems the best algorithm we know runs in
NP
truly exponential time.
[Probabilistic analysis of heuristics.] Another possibility is
(cid:15)
to drop the requirement that the solution to a problem cater
equally to all input instances. In some applications, it is possible
that the class of input instances is severely constrained and for
these instances there is an e(cid:14)cient algorithm which will always
do the trick. Consider for examplethe problem of (cid:12)nding Hamil-
tonian cycles in graphs. This is -hard. However, it can be
NP
shown that there is an algorithm which will (cid:12)nd a Hamiltonian
cycle in \almost every" graph which contains one. Such results
are usually derived using a probabilistic model of the constraints
on the input instances. It is then shown that certain heuristics
will solve the problem with very high probability. Unfortunately,
it is usually not very easy to justify the choice of a particular
input distribution. Moreover, in a lot of cases, the analysis of
algorithms under assumptions about distributions is in itself in-
tractable.
[Approximation algorithms.] Finally, we could relax the re-
(cid:15)
quirementthat wealways(cid:12)ndtheoptimal solution. Inpractice,it
is usually hard to tell the di(cid:11)erence between an optimal solution
and a near-optimal solution. It seems reasonable to devise algo-
rithms which are really e(cid:14)cient in solving -hard problems, at
NP
the cost of providing solutions which in all cases is guaranteed to
be only slightly sub-optimal
Insomesituations,thelastrelaxationoftherequirementsforsolving
a problemappears to be themostreasonable. Thisresults inthe notion
1.1. PRELIMINARIES AND BASIC DEFINITIONS Page 7
of the \approximate"solution of an optimizationproblem. In this book
we will attempt to classify as one of three types all hard optimization
problems, from the point of view of approximability. Some problems
seem to be extremely easy to approximate, e.g. Knapsack, Scheduling
and Bin Packing. Other problems are so hard that even (cid:12)nding very
poor approximations can be shown to be -hard, e.g. Graph Color-
NP
ing, TSP and Clique. Finally, there is a class of problems which seem
to be of intermediate complexity, e.g. Vertex Cover, Euclidean TSP or
Steiner Trees. In some cases we will be able to demonstrate that a
problem is provably hard to approximate within some error.
1.1. Preliminaries and Basic De(cid:12)nitions
We (cid:12)rst de(cid:12)ne an -hard optimization problem and explore two no-
NP
tions of approximation. The following is a formal de(cid:12)nition of a maxi-
mization problem; a minimization problem can be de(cid:12)ned analogously.
De(cid:12)nition 1.1: An optimization problem (cid:5) is characterized by
three components:
[Instances] D: a set of input instances.
(cid:15)
[Solutions] S(I): the set of all feasible solutions for an instance
(cid:15)
I D.
2
[Value] f: a function which assigns a value to each solution,
(cid:15)
i.e. f : S(I) .
! <
I
A maximization problem (cid:5) is: given I D, (cid:12)nd a solution (cid:27)opt
2 2
S(I) such that
I
(cid:27) S(I); f((cid:27)opt) f((cid:27))
8 2 (cid:21)
We will also refer to the value of the optimal solution as OPT(I),
(cid:1) I
i.e. OPT(I) = f((cid:27)opt).
CHAPTER 1. INTRODUCTION Page 8
We will abuse our notation a bit by sometimes referring to the op-
timal solution also as OPT(I). The meaning should be clear from the
context. The following example should help to (cid:13)esh out these de(cid:12)ni-
tions.
BIN PACKING (BP): Informally, we are given a collection of
items of sizes between 0 and 1. We are required to pack them into bins
of unit size so as to minimize the number of bins used. Thus, we have
the following minimization problem.
[Instances] I = s1;s2;...sn , such that i;si [0;1].
(cid:15) f g 8 2
[Solutions] A collection of subsets (cid:27) = B1;B2;...Bk which is
(cid:15) f g
a disjoint partition of I, such that i;Bi I and j Bisj 1.
8 (cid:26) 2 (cid:20)
P
[Value] The value of a solution is the number of bins used, or
(cid:15)
f((cid:27)) = (cid:27) = k,
j j
Wewouldliketospecifyattheoutsetthatanunderlyingassumption
throughout thisbook willbe that the optimizationproblemssatisfy the
following two technical conditions. This will be particularly important
when we present complexity-theoretic results.
1. The range of f and all the numbersin I have to be integers. Note
that we can easily extend this to allow rational numbers since
those can be represented as pairs of integers. For example,in the
Bin Packing problem we will assume all item sizes are rationals.
2. For any (cid:27) S(I), f((cid:27)) is polynomially bounded in the size of
2
any number which appears in I.
It is not very hard to see that the (cid:12)rst condition is reasonable since
no computer can deal with in(cid:12)nite precision real numbers. As for the
secondcondition,wedeferthejusti(cid:12)cationandthemotivationtoChap-
ter 2.
We are only going to be concerned with -complete optimiza-
NP
tion problems such as Bin Packing. Some people may (cid:12)nd this concept