Proofs, Computability, Undecidability, Complexity, And the Lambda Calculus: An Introduction

446 Pages·2020·4.041 MB·English

by  Gallier J., Quaintance J.

Proofs, Computability, Undecidability,
Complexity, And the Lambda Calculus
An Introduction
Jean Gallier and Jocelyn Quaintance
Department of Computer and Information Science
University of Pennsylvania
Philadelphia, PA 19104, USA
e-mail: [email protected]
c Jean Gallier
(cid:13)
Please, do not reproduce without permission of the author
March 22, 2020
2
Preface
The main goal of this book is to present a mix of material dealing with
1. Proof systems.
2. Computability and undecidability.
3. The Lambda Calculus.
4. Some aspects of complexity theory.
Historically, the theory of computability and undecidability arose from Hilbert’s efforts
to completely formalize mathematics and from Go¨del’s first incompleteness theorem that
showed that such a program was doomed to fail. People realized that to carry out both
Hilbert’s program and Go¨del’s work it was necessary to define precisely what is the notion of
a computable function and the notion of a mechanically checkable proof. The first definition
given around 1934 was that of the class of computable function in the sense of Herbrand–
Go¨del–Kleene. The second definition given by Church in 1935-1936 was the notion of a
function definable in the λ-calculus. The equivalence of these two definitions was shown by
Kleene in 1936. Shortly after in 1936, Turing introduced a third definition, that of a Turing-
computable function. Turing proved the equivalence of his definition with the Herbrand–
Go¨del–Kleene definition in 1937 (his proofs are rather sketchy compared to Kleene’s proofs).
All these historical papers can be found in a fascinating book edited by Martin Davis [25].
Negativeresultspointing outlimitationsof thenotionof computability started toappear:
Go¨del’s first (and second) incompleteness result, but also Church’s theorem on the undecid-
ability of validity in first-order logic, and Turing’s result on the undecidability of the halting
problem for Turing machines. Although originally the main focus was on the notion of func-
tion, these undecidability results triggered the study of computable and noncomputable sets
of natural numbers.
Other definitions of the computable functions were given later. From our point of view,
the most important ones are
1. RAM programs and RAM-computable functions by Shepherdson and Sturgis (1963),
and anticipated by Post (1944); see Machtey and Young [36].
2. Diophantine-definable sets (Davis–Putnam–Robinson–Matiyasevich); see Davis [8, 9].
3
4
We find the RAM-progam model quite attractive because it is a very simplified realistic
model of the true architecture of a modern computer. Technically, we also find it more
convenient to assign G¨odel numbers to RAM programs than assigning Go¨del numbers to
Turing machines. Every RAM program can be converted to a Turing machine and vice-
versa in polynomial time (going from a Turing machine to a RAM is quite horrific), so the
two models are equivalent in a strong sense. So from our perspective Turing machines could
be dispensed with, but there is a problem. The problem is that the Turing machine model
seems more convenient to cope with time or space restrictions, that is, to define complexity
classes.
There is actually no difficulty in defining nondeterministic RAM programs and to impose
a time restriction on the program counter or a space restriction on the size of registers,
but nobody seems to follows this path. This seems unfortunate to us because it appears
that it would be easier to justify the fact that certain reductions can be carried out in
polynomial time (or space) by writing a RAM program rather than by constructing a Turing
machine. Regarding this issue, we are not aware than anyone actually provides Turing
machines computing these reductions, even for SAT.
In any case, we will stick to the tradition of using Turing machines when discussing
complexity classes.
In addition to presenting the RAM-program model, the Turing machine model, the
Herbrand–Go¨del–Kleene definition of the computable functions, and showing their equiv-
alence, we provide an introduction to recursion theory (see Chapter 7). In particular, we
discuss creative and productive sets (see Rogers [44]). This allows us to cover most of the
main undecidability results. These include
1. The undecidability of the halting problem for RAM programs (and Turing machines).
2. Rice’s theorem for the computable functions.
3. Rice’s extended theorem for the listable sets.
4. A strong form of G¨odel’ first incompleteness theorem (in terms of creative sets) follow-
ing Rogers [44].
5. The fact that the true first-order sentences of arithmetic are not even listable (a pro-
ductive set) following Rogers [44].
6. The undecidability of the Post correspondence problem (PCP) using a proof due to
Dana Scott.
7. The undecidability of the validity in first-order logic (Church’s theorem), using a proof
due to Robert Floyd.
8. The undecidability of Hilbert’s tenth problem (the DPRM theorem) following Davis
[8].
5
9. Another strong form of G¨odel’ first incompleteness theorem, as a consequence of dio-
phantine definability following Davis [8].
The following two topics are rarely covered in books on the theory of computation and
undecidability.
In Chapter 6 we introduce Church’s λ-calculus and show how the computable functions
and the partial computable functions are definable in the λ-calculus, using a method due to
Barendregt [4]. We also give a glimpse of the second-order polymorphic λ-calculus of Girard.
In Chapter 8 we discuss the definability of the listable sets in terms of Diophantine
equations (zeros of polynomials with integer coefficients) and state the famous result about
the undecidability of Hilbert’s tenth problem (the DPRM theorem). We follow the masterly
exposition of Davis [8, 9].
A possibly unsusual aspect of our book is that we begin with two chapters on mathemat-
ical reasoning and logic. Given the origins of the theory of computation and undecidability,
we feel that this is very appropriate. We present proof systems in natural deduction style
(a la Prawitz), which makes it easy to discuss the special role of the proof–by–contradiction
principle, and to introduce intuitionistic logic, which is the result of removing this rule from
the set of inference rules. It is also quite natural to explain how proofs in intuitionistic
propositional logic are represented by simply-typed λ-terms. Then it is easy to introduce
the “Curry–Howard isomorphism.” This is a prelude to the introduction of the “pure”
(untyped) λ-calculus.
Our treatment of complexity theory is limited to , , co- , , ,
P NP NP EXP NEXP PS
(PSPACE) and (NPSPACE) and is fairly standard. However, we prove that SAT
NPS
is -complete by first proving (following Lewis and Papadimitriou [35]) that a bounded
NP
tiling problem is -complete.
NP
In Chapter 12 we treat the result that primality testing is in in more details than
NP
most other sources, relying on an improved version of a theorem of Lucas as discussed in
Crandall and Pomerance [6]. The only result that we omit is the existence of primitive roots
in (Z/pZ)∗ when p is prime.
In Chapter 13 we prove Savitch’s theorem ( = ). We state the fact that the
PS NPS
validity of quantified boolean formulae is -complete and provide parts of the proof. We
PS
conclude with the beautiful proof of Statman [46] that provability in intuitionistic logic is
-complete. We not give all the details but we prove the correctness of Statman’s amazing
PS
translation of a valid QBF into an intuitionistically provable proposition.
We feel strongly that one does not learn mathematics without reading (and struggling
through) proofs, so we tried to provide as many proofs as possible. Among some of the
omissions, we do not prove the DPRM nor do we show how to construct a Go¨del sentence in
the proof of the first incompleteness theorem; Rogers [44] leaves this as an exercise! We also
do not give a complete proof of Statman’s result. However, whenever a proof is omitted, we
provide a pointer to a source that contains such a proof.
6
Acknowledgement: We would like to thank Jo˜ao Sedoc and Marcelo Siqueira, for reporting
typos and for helpful comments. I was initiated to the theory of computation and undecid-
ability by my advisor Sheila Greibach who taught me how to do research. My most sincere
thanks to Sheila for her teachings and the inspiration she provided. In writing this book we
were inspired and sometimes borrowed heavily from the beautiful papers and books written
by the following people who have my deepest gratitude: Henk Barendregt, Richard Crandall,
Martin Davis, Herbert Enderton, Harvey Friedman, Jean-Yves Girard, John Hopcroft, Bill
Howard, Harry Lewis, Zohar Manna, Christos Papadimitriou, Carl Pomerance, Dag Prawitz,
Helmut Schwichtenberg, Dana Scott, Rick Statman, Jeff Ullman, Hartley Rogers, and Paul
Young. Of course, we must acknowledge Alonzo Church, Gerhard Gentzen, Kurl G¨odel,
Stephen Kleene and Alan Turing for their extraordinary seminal work.
Contents
Contents 7
1 Mathematical Reasoning And Basic Logic 11
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2 Logical Connectives, Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3 Meaning of Implication and Proof Templates for Implication . . . . . . . . . 16
1.4 Proof Trees and Deduction Trees . . . . . . . . . . . . . . . . . . . . . . . . 20
1.5 Proof Templates for . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
¬
1.6 Proof Templates for , , . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
∧ ∨ ≡
1.7 De Morgan Laws and Other Useful Rules of Logic . . . . . . . . . . . . . . . 34
1.8 Formal Versus Informal Proofs; Some Examples . . . . . . . . . . . . . . . . 35
1.9 Truth Tables and Truth Value Semantics . . . . . . . . . . . . . . . . . . . . 39
1.10 Proof Templates for the Quantifiers . . . . . . . . . . . . . . . . . . . . . . . 42
1.11 Sets and Set Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
1.12 Induction and Well–Ordering Principle . . . . . . . . . . . . . . . . . . . . . 55
1.13 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2 Mathematical Reasoning And Logic, A Deeper View 63
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.2 Inference Rules, Deductions, Proof Systems ⇒ and ⇒ . . . . . . . . . . 64
Nm NGm
2.3 Proof Rules, Deduction and Proof Trees for Implication . . . . . . . . . . . . 66
2.4 Examples of Proof Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
2.5 A Gentzen-Style System for Natural Deduction . . . . . . . . . . . . . . . . 78
2.6 Adding , , ; The Proof Systems ⇒,∧,∨,⊥ and ⇒,∧,∨,⊥ . . . . . . . . . 80
∧ ∨ ⊥ Nc NGc
2.7 Clearing Up Differences Among Rules Involving . . . . . . . . . . . . . . . 90
⊥
2.8 De Morgan Laws and Other Rules of Classical Logic . . . . . . . . . . . . . . 93
2.9 Formal Versus Informal Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . 96
2.10 Truth Value Semantics for Classical Logic . . . . . . . . . . . . . . . . . . . 97
2.11 Kripke Models for Intuitionistic Logic . . . . . . . . . . . . . . . . . . . . . . 100
2.12 Decision Procedures, Proof Normalization . . . . . . . . . . . . . . . . . . . 103
2.13 The Simply-Typed λ-Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . 106
7
8 CONTENTS
2.14 Completeness and Counter-Examples . . . . . . . . . . . . . . . . . . . . . . 113
2.15 Adding Quantifiers; Proof Systems ⇒,∧,∨,∀,∃,⊥, ⇒,∧,∨,∀,∃,⊥ . . . . . . . . 115
Nc NGc
2.16 First-Order Theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
2.17 Basics Concepts of Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 134
2.18 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
3 RAM Programs, Turing Machines 165
3.1 Partial Functions and RAM Programs . . . . . . . . . . . . . . . . . . . . . 168
3.2 Definition of a Turing Machine . . . . . . . . . . . . . . . . . . . . . . . . . . 173
3.3 Computations of Turing Machines . . . . . . . . . . . . . . . . . . . . . . . . 175
3.4 Equivalence of RAM programs And Turing Machines . . . . . . . . . . . . . 178
3.5 Computably Enumerable and Computable Languages . . . . . . . . . . . . . 179
3.6 A Simple Function Not Known to be Computable . . . . . . . . . . . . . . . 180
3.7 The Primitive Recursive Functions . . . . . . . . . . . . . . . . . . . . . . . 183
3.8 Primitive Recursive Predicates . . . . . . . . . . . . . . . . . . . . . . . . . . 191
3.9 The Partial Computable Functions . . . . . . . . . . . . . . . . . . . . . . . 193
4 Universal RAM Programs and the Halting Problem 199
4.1 Pairing Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
4.2 Equivalence of Alphabets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
4.3 Coding of RAM Programs; The Halting Problem . . . . . . . . . . . . . . . 208
4.4 Universal RAM Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
4.5 Indexing of RAM Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
4.6 Kleene’s T-Predicate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
4.7 A Non-Computable Function; Busy Beavers . . . . . . . . . . . . . . . . . . 220
5 Elementary Recursive Function Theory 225
5.1 Acceptable Indexings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
5.2 Undecidable Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
5.3 Reducibility and Rice’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . 231
5.4 Listable (Recursively Enumerable) Sets . . . . . . . . . . . . . . . . . . . . . 234
5.5 Reducibility and Complete Sets . . . . . . . . . . . . . . . . . . . . . . . . . 240
6 The Lambda-Calculus 245
6.1 Syntax of the Lambda-Calculus . . . . . . . . . . . . . . . . . . . . . . . . . 247
6.2 β-Reduction and β-Conversion; the Church–Rosser Theorem . . . . . . . . . 252
6.3 Some Useful Combinators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
6.4 Representing the Natural Numbers . . . . . . . . . . . . . . . . . . . . . . . 258
6.5 Fixed-Point Combinators and Recursively Defined Functions . . . . . . . . . 264
6.6 λ-Definability of the Computable Functions . . . . . . . . . . . . . . . . . . 266
6.7 Definability of Functions in Typed Lambda-Calculi . . . . . . . . . . . . . . 272
6.8 Head Normal-Forms and the Partial Computable Functions . . . . . . . . . . 279
CONTENTS 9
7 Recursion Theory; More Advanced Topics 285
7.1 The Recursion Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
7.2 Extended Rice Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
7.3 Creative and Productive Sets; Incompleteness . . . . . . . . . . . . . . . . . 294
8 Listable and Diophantine Sets; Hilbert’s Tenth 301
8.1 Diophantine Equations; Hilbert’s Tenth Problem . . . . . . . . . . . . . . . . 301
8.2 Diophantine Sets and Listable Sets . . . . . . . . . . . . . . . . . . . . . . . 304
8.3 Some Applications of the DPRM Theorem . . . . . . . . . . . . . . . . . . . 308
8.4 Go¨del’s Incompleteness Theorem . . . . . . . . . . . . . . . . . . . . . . . . 311
9 The Post Correspondence Problem; Applications 317
9.1 The Post Correspondence Problem . . . . . . . . . . . . . . . . . . . . . . . 317
9.2 Some Undecidability Results for CFG’s . . . . . . . . . . . . . . . . . . . . . 323
9.3 More Undecidable Properties of Languages . . . . . . . . . . . . . . . . . . . 326
9.4 Undecidability of Validity in First-Order Logic . . . . . . . . . . . . . . . . . 327
10 Computational Complexity; and 331
P NP
10.1 The Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
P
10.2 Directed Graphs, Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
10.3 Eulerian Cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
10.4 Hamiltonian Cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
10.5 Propositional Logic and Satisfiability . . . . . . . . . . . . . . . . . . . . . . 336
10.6 The Class , -Completeness . . . . . . . . . . . . . . . . . . . . . . . 341
NP NP
10.7 The Bounded Tiling Problem is -Complete . . . . . . . . . . . . . . . . . 347
NP
10.8 The Cook-Levin Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
10.9 Satisfiability of Arbitrary Propositions and CNF . . . . . . . . . . . . . . . . 354
11 Some -Complete Problems 359
NP
11.1 Statements of the Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
11.2 Proofs of -Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
NP
11.3 Succinct Certificates, co , and . . . . . . . . . . . . . . . . . . . . . 387
NP EXP
12 Primality Testing is in 393
NP
12.1 Prime Numbers and Composite Numbers . . . . . . . . . . . . . . . . . . . . 393
12.2 Methods for Primality Testing . . . . . . . . . . . . . . . . . . . . . . . . . . 394
12.3 Modular Arithmetic, the Groups Z/nZ, (Z/nZ)∗ . . . . . . . . . . . . . . . . 397
12.4 The Lucas Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
12.5 Lucas Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
12.6 Algorithms for Computing Powers Modulo m . . . . . . . . . . . . . . . . . 411
12.7 PRIMES is in . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
NP
10 CONTENTS
13 Polynomial-Space Complexity; and 417
PS NPS
13.1 The Classes (or PSPACE) and (NPSPACE) . . . . . . . . . . . . 417
PS NPS
13.2 Savitch’s Theorem: = . . . . . . . . . . . . . . . . . . . . . . . . . 419
PS NPS
13.3 A Complete Problem for : QBF . . . . . . . . . . . . . . . . . . . . . . . 420
PS
13.4 Provability in Intuitionistic Propositional Logic . . . . . . . . . . . . . . . . 428
Bibliography 435
Symbol Index 439
Index 441