Introduction to complexity theory, lecture notes

375 Pages·1999·2.333 MB·English

by  Goldreich O.

Introduction to Complexity Theory { Lecture Notes
Oded Goldreich
Department of Computer Science and Applied Mathematics
Weizmann Institute of Science, Israel.
Email: [email protected]
July 31, 1999
I
(cid:13)cCopyright 1999 by Oded Goldreich.
Permission to make copies of part or all of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for pro(cid:12)t or commercial advantage and that new copies
bear this notice and the full citation on the (cid:12)rst page. Abstracting with credit is permitted.
II
Preface
Complexity Theory is a central (cid:12)eld of Theoretical Computer Science, with a remarkable list of
celebrated achievements as well as a very vibrant present research activity. The (cid:12)eld is concerned
with the study of the intrinsic complexity of computational tasks, and this study tend to aim at
generality: It focuses on natural computational resources, and the e(cid:11)ect of limiting those on the
class of problems that can be solved.
These lecture notes were taken by students attending my year-long introductory course on
Complexity Theory, given in 1998{99 at the Weizmann Institute of Science. The course was aimed
at exposing the students to the basic results and research directions in the (cid:12)eld. The focus was on
concepts and ideas, and complex technical proofs were avoided. Speci(cid:12)c topics included:
Revisiting NP and NPC (with emphasis on search vs decision);
(cid:15)
Complexity classes de(cid:12)ned by one resource-bound { hierarchies, gaps, etc;
(cid:15)
Non-deterministic Space complexity (with emphasis on NL);
(cid:15)
Randomized Computations (e.g., ZPP, RP and BPP);
(cid:15)
Non-uniform complexity (e.g., P/poly, and lower bounds on restricted circuit classes);
(cid:15)
The Polynomial-time Hierarchy;
(cid:15)
The counting class #P, approximate-#P and uniqueSAT;
(cid:15)
Probabilistic proof systems (i.e., IP, PCP and ZK);
(cid:15)
Pseudorandomness (generators and derandomization);
(cid:15)
Time versus Space (in Turing Machines);
(cid:15)
Circuit-depth versus TM-space (e.g., AC, NC, SC);
(cid:15)
Average-case complexity;
(cid:15)
It was assumed that students have taken a course in computability, and hence are familiar with
Turing Machines.
Most of the presented material is quite independent of the speci(cid:12)c (reasonable) model of com-
putation, butsome (e.g., Lectures 5, 16, and19{20) dependsheavily on the locality of computation
of Turing machines.
III
IV
State of these notes
Thesenotesareneithercompletenorfullyproofread,letalonebeingfarfromuniformlywell-written
(although the notes of some lectures are quite good). Still, I do believe that these notes suggest a
good outline for an introduction to complexity theory course.
Using these notes
A total of 26 lectures were given, 13 in each semester. In general, the pace was rather slow, as
most students were (cid:12)rst year graduates and their background was quite mixed. In case the student
body is uniformly more advanced one should be able to cover much more in one semester. Some
concrete comments for the teacher follow
Lectures 1 and 2 revisit the P vs NP question and NP-completeness. The emphasis is on
(cid:15)
presentingNPintermsofsearchproblems,onthefactthatthemereexistenceofNP-complete
setsisinteresting(andeasilydemonstratable),andonreductionsapplicablealsointhedomain
of search problems (i.e., Levin reductions). A good undergraduate computability course
should cover this material, but unfortunately this is often not the case. Thus, I suggest to
give Lectures 1 and 2 if and only if the previous courses taken by the students failed to cover
this material.
There is something anal in much of Lectures 3 and 5. One may prefer to shortly discuss
(cid:15)
the material of these lectures (without providing proofs) rather than spend 4 hours on them.
(Note that many statements in the course are given without proof, so this will not be an
exception.)
Oneshouldbeableto mergeLectures 13 and14 into a single lecture (or at mosta lecture and
(cid:15)
a half). I failed to do so due to inessential reasons. Alternatively, may merge Lectures 13{15
into two lectures.
Lectures 21{23 were devoted to communication complexity, and circuit depth lower bounds
(cid:15)
derived via communication complexity. Unfortunately, this sample fails to touch upon other
important directions in circuit complexity (e.g., size lower bound for AC0 circuits). I would
recommend to try to correct this de(cid:12)ciency.
Lecture 25 was devoted to Computational Learning Theory. This area, traditionally associ-
(cid:15)
ated with \algorithms", does have a clear \complexity" (cid:13)avour.
Lecture 26 was spent discussing the (limited in our opinion) meaningfulness of relativization
(cid:15)
results. The dilemma of whether to discuss something negative or just ignore it is never easy.
Many interesting results were not covered. In many cases this is due to the trade-o(cid:11) between
(cid:15)
their conceptual importance as weighted against their technical di(cid:14)culty.
V
Bibliographic Notes
There are several books which cover small parts of the material. These include:
1. Garey, M.R., and D.S. Johnson. Computers and Intractability: A Guide to the Theory of
NP-Completeness, W.H. Freeman and Company, New York, 1979.
2. O.Goldreich. Modern Cryptography, Probabilistic Proofs and Pseudorandomness. Algorithms
and Combinatorics series (Vol. 17), Springer, 1998. Copies have been placed in the faculty’s
library.
3. J.E. Hopcroft and J.D. Ullman, Introduction to Automata Theory, Languages and Computa-
tion, Addison-Wesley, 1979.
4. M. Sipser. Introduction to the Theory of Computation, PWS Publishing Company, 1997.
However,thepresentationofmaterialintheselecturenotesdoesnotnecessarilyfollowthesesources.
Each lectureisplannedto includebibliographicnotes, butthis intensionhas beenonlypartially
ful(cid:12)lled so far.
VI
Acknowledgments
I am most grateful to the students who have attended the course and partipiated in the project
of preparing the lecture notes. So thanks to Sergey Benditkis, Reshef Eilon, Michael Elkin, Amiel
Ferman, Dana Fisman, Danny Harnik, Tzvika Hartman, Tal Hassner, HillelKugler, Oded Lachish,
Moshe Lewenstein, Yehuda Lindell, Yoad Lustig, Ronen Mizrahi, Leia Passoni, Guy Peer, Nir
Piterman, Ely Porate, Yoav Rodeh, Alon Rosen, Vered Rosen, Noam Sadot, Il’ya Safro, Tamar
Seeman, Ekaterina Sedletsky, Reuben Sumner, Yael Tauman, Boris Temkin, Erez Waisbard, and
Gera Weiss.
I am grateful to Ran Raz and Dana Ron who gave guess lectures during the course: Ran gave
Lectures 21{23 (on communication complexity and circuit complexity), and Dana gave Lecture 25
(on computational learning theory).
Thanks also to Paul Beame, Ruediger Reischuk and Avi Wigderson who have answered some
questions I’ve had while preparing this course.
VII
VIII
Lecture Summaries
Lecture 1: The P vs NP Question. We review the fundamentalquestion ofcomputerscience,
known as the versus question: Given a problem whose solution can be veri(cid:12)ed e(cid:14)ciently
P NP
(i.e., in polynomial time), is there necessarily an e(cid:14)cient method to actually (cid:12)nd such a solution?
Loosely speaking, the (cid:12)rst condition (i.e., e(cid:14)cient veri(cid:12)cation) is captured in the de(cid:12)nition of ,
NP
and the second in that of . The actual correspondence relies on the notion of self-reducibility,
P
which relates the complexity of determining whether a solution exists to the complexity of actually
(cid:12)nding one.
Notes taken by Eilon Reshef.
Lecture 2: NP-completeness and Self Reducibility. We prove that any relation de(cid:12)ningan
NP-complete language is self-reducible. This will be done using the SAT self-reducibility (proved
in Lecture 1), and the fact that SAT is NP-Hard under Levin Reductions. The latter are Karp
Reductionsaugmentedbye(cid:14)cienttransformationsofNP-witnessesfromtheoriginalinstancetothe
reducedone, andvice versa. Alongtheway, we give asimpleproofof theexistence ofNP-Complete
languages (by proving that Bounded Halting is NP-Complete).
Notes taken by Nir Piterman and Dana Fisman.
Lecture 3: More on NP and some on DTIME. Inthe(cid:12)rstpartofthislecturewediscusstwo
properties of the complexity classes P, NP and NPC: The property is that NP contains problems
whichareneitherNP-completenorinP(providedNP =P),andthesecondoneisthatNP-relations
6
haveoptimalsearch algorithms. Inthe secondpartwede(cid:12)nenewcomplexityclasses basedonexact
timebounds,andconsidersomerelationsbetweenthem. Wepointoutthesensitivityoftheseclasses
to the speci(cid:12)c model of computation (e.g., one-tape versus two-tape Turing machines).
Notes taken by Michael Elkin and Ekaterina Sedletsky.
Lecture 4: Space Complexity. We de(cid:12)ne \nice" complexity bounds; these are bounds which
can be computed within the resources they supposedly bound (e.g., we focus on time-constructible
and space-constructible bounds). We de(cid:12)ne space complexity using an adequate model of compu-
tation in which one is not allowed to use the area occupied by the input for computation. Before
dismissing sub-logarithmic space, we present two results regarding it (contrasting sub-loglog space
with loglog space). We show that for \nice" complexity bounds, there is a hierarchy of complexity
classes { the more resources one has the more tasks one can perform. One the other hand, we
mention that this increase in power may not happen if the complexity bounds are not \nice".
Notes taken by Leia Passoni and Reuben Sumner.
IX