ebook img

Introduction to Theory of Computing Lecture Notes (Cornell CS4810) PDF

46 Pages·2017·1.15 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Introduction to Theory of Computing Lecture Notes (Cornell CS4810)

CS 4810 — Fall'13 Syllabus| Lectures| Homework| Resources Lecture 1: Boolean Functions Boolean values There are two Boolean values, TRUE and FALSE. For convenience, we will 1 0 denote these values as and . Boolean values are often also called bits. We will use Boolean values to encode information. Basic Boolean operations Basic Boolean operations are functions that take one or two Boolean values as input and output a Boolean value. We will focus the following operations: 1  if x = 1, (x) := { NOT 0  otherwise. 1  if x = 1 and y = 1, (x,y) := { AND 0  otherwise. 1  if x = 1 or y = 1, (x,y) := { OR 0  otherwise. The following notation is common: ¬x for  (x), NOT x∧y for  (x,y), AND x∨y for  (x,y). OR Truth table We can represent the Boolean operations NOT, AND, OR by the following truth table. x y (x) (x,y) (x,y) NOT AND OR 0 0 1 0 0 0 1 1 0 1 1 0 0 0 1 1 1 0 1 1 De Morgan’s Laws There are redundancies among the operations NOT, AND, OR. In particular, we can write AND in terms of NOT, OR. Similarly, OR can be expressed in terms of NOT, AND. ¬(x∧y) = ¬x∨¬y, ¬(x∨y) = ¬x∧¬y. We can check that these identities hold by looking at the above truth table. Boolean functions n f n An -bit Boolean function is a function that take Boolean values as input and outputs a Boolean value, i.e., f:{0,1}n → {0,1}. Boolean function can express interesting algorithmic questions: 1  if x is (the binary encoding of) a prime number, (x) := { PRIMESn 0  otherwise. (x) RIEMANNn := 1  if x is (the binary encoding of) a proof of the Riemann hypothesis, { 0  otherwise. (The Riemann hypothesis is an important open question in mathematics. If you resolve the hypothesis, you get a Millennium Prize.) We will try to understand whether these kind of functions can be expressed by a small number of basic Boolean operations. (It turns out that both functions above can be expressed by a number of basic Boolean operations that is n polynomial in .) CS 4810 — Fall'13 Syllabus| Lectures| Homework| Resources Lecture 2: Boolean Circuits Boolean circuit n 1 A Boolean circuit with inputs and output is a directed acyclic graph that consists of the following kind of nodes: 1,…,n » Input nodes, labeled , without ingoing edges (sources). » Any number of gates, labeled with NOT, AND, OR. » All AND, OR gates have two incoming edges. » All NOT gates have one incoming edge. » One output gate, without outgoing edges (sink). (All other gates have at least one outgoing edge.) Circuit computation x ,…,x Suppose we assign Boolean values 1 n to the input nodes of a circuit. Then, we can propagate these values through the gates of the circuit by applying the corresponding Boolean operations. More formally, we would process the gates of the circuit in topological u order. To process a gate labeled with a Boolean operation T ∈ { , , } T NOT AND OR , we apply the operation to the values assigned to the u u ancestors of and we assign the output of this operation to . (Since we u process nodes in topological order, the ancestors of have already been u processed at the time we process .) C n 1 n We say a Boolean circuit with inputs and output computes an -bit f:{0,1}n → {0,1} Boolean function if for every input x = (x ,…,x ) ∈ {0,1}n f(x) 1 n , the function value is equal to the value C x ,…,x propagated to the output gate of when we assign 1 n to the input C nodes of . One can show (e.g., by induction) that a circuit computes only one function. In particular, the function does not depend on the order we process the gates. f C We write C to denote the function computed by a circuit . Example: parity n Consider the following -bit Boolean function 1  if x +…+x  is an odd number, (x) := { 1 n PARITYn 0  otherwise. n = 2 , , For , we can express PARITY in terms of NOT AND OR as follows (x ,x ) = (x ∧¬x )∨(¬x ∧x ) PARITY2 1 2 1 2 1 2 2 The parity of bits is called exclusive-or. A common notation for (x ,x ) x ⊕x PARITY2 1 2 is 1 2. n 2 If is a power of , we can construct a circuit for PARITY recursively based on the following identity (x) = (x ,…,x )⊕ (x ,…,x ). PARITYn PARITYn/2 1 n/2 PARITYn/2 n/2+1 n Circuit complexity For a given Boolean functions, there could be many ways of computing it by a circuit. But some circuits are better than others. C We measure the efficiency of a circuit by its size size(C) := the number of gates of C. f For a Boolean function , we define its size as the minimum size of circuit f that computes . Another measure for the efficiency of a circuit is its depth depth(C) := length of longest path from an input node to the output gate Generic upper bound n f Theorem: For every -bit Boolean function there exists a circuit of 10⋅2n −10 size at most to compute it. n f Proof: We will use induction on . If we fix one of input bit of the function , n−1 then we are left with an -bit Boolean function. n = 1 1 For , the theorem is true because for every -bit function, we can find 10 n > 1 f f n−1 a circuit of size at most . Suppose . Let 0 and 1 be the two bit f functions obtained by fixing the first bit of , i.e., f (x ,...,x ) = f(0,x ,...,x ), 0 2 n 2 n f (x ,...,x ) = f(1,x ,...,x ). 1 2 n 2 n f f n−1 Both 0 and 1 are bit functions. The following identity allows us to f f f express the original function in terms of 0 and 1 as well as a few basic operations f = (¬x ∧f )∨(x ∧f ). 1 0 1 1 f f Therefore, we can construct a circuit for from a circuit for 0 and a f 4 circuit for 1 as well as additional gates (one NOT gate, one OR gate, and two size(f) ≤ size(f )+size(f )+4 AND gates). It follows that 0 1 . By induction size(f ) size(f ) 10⋅2n−1 −10 hypothesis, both 0 and 1 are at most . By combining these bound, we get size(f) ≤ 10⋅2n −20+4 = 10⋅2n −16 ≤ 10⋅2n −10. CS 4810 — Fall'13 Syllabus| Lectures| Homework| Resources Lecture 3: Hard functions Generic lower bound n ∈ N n ≥ 100 n Theorem: For every with , there exists an -bit Boolean 2n/100n function that requires circuits of size at least . n Proof: The idea is to compare the number of -bit Boolean functions f:{0,1}n → {0,1} s = 2n/100n to the number of circuits of size at most . If the number of functions is larger than the number of such circuits, it means that there exists a functions that is not computed by such a circuit. n Let us first count the number of -bit Boolean functions. In general, the X Y |Y ||X| number of functions from a finite set to a finite set is equal to . (You n should convince yourself of this fact.) Hence, the number of -bit Boolean 22n functions is . s s Now let’s count the number of size- circuits. A circuit of size consists of a s′ = s+n ≤ 2s n s directed acyclic graph (DAG) on nodes ( input nodes and 2 gates). Each node has in-degree at most . (The in-degree of a node is the 0 number of ingoing edges.) Furthermore, each input node (in-degree ) is i ∈ [n] labeled with an input variable . Each gate is labeled with a Boolean operation, either AND, OR, or NOT. We claim that we can describe such a circuit completely by a bit string of 10slogs length at most . First, we give every node a unique ID. Since there are s′ logs′ only nodes in the circuit, we only need bits for the ID. (We can use the 1 s′ s′ numbers from to as IDs.) Our description of the circuit will consist of 10logs′ parts, each of length at most (one part per vertex). To describe a single u u logs′ vertex , we store the ID of ( bits), the IDs of the vertices that have u 2logs′ u incoming arcs to ( bits) and the label of , which is either a Boolean logn ≤ logs′ function or the index of an input/output variable (at most bits). We don’t need to store the outgoing arcs of a vertex, because those will be 4logs′ stored as incoming arcs by other vertices. In total, we store at most bits per vertex. Hence, the description of the whole circuit is indeed shorter than 10slogs bits. 10 log s 210slogs It follows that the number of circuits of size is at most (the 10slogs s = 2n/100n number of bit strings of length ). For , this number is 22n much less than . (Check this calculation.) It follows that an overwhelming n 2n/100n majority of -bit Boolean functions does not have circuits of size . \ Remark: While the theorem shows that some functions require large circuits, the proof of the theorem does not give an explicit construction of such a function. (The proof is non-constructive.) It’s a famous open problem to show strong lower bounds on the circuit size of concrete functions. For example, the following funtion is conjectured to require circuits of exponential size ⎧0  if x is the binary encoding of a circuit (x) := ⎨  that computes the constant 0 function, CIRCUIT−SATn ⎩ 1  otherwise. Straight-line Programs In this part, we will discuss an exact correspondence between circuits and a simple kind of programs. Here is an example of a straight-line program: x ,x Input: 1 2 Instructions: y := ¬x » 1 1 y := ¬x » 2 2 y := x ∧y » 3 1 2 y := y ∧x » 4 1 2 y := y ∨y » 5 3 4 y Output: 5 This program outputs the parity of the two input bits. We can construct a circuit that corresponds to this program in the following way: For every instruction, we have a gate in the circuit that applies the same Boolean operation and has incoming edges from nodes corresponding to the right-hand-side variables of the instruction. The size of the resulting circuit is exactly equal to the number of instructions of the straight-line program. Definition: A straight-line program is a sequence of instructions of the following form: Each instruction applies a basic Boolean operation ( , , NOT AND OR) to input variables or previously assigned variables and assigns the outcome of this operation to a new variable. The output of the program is the outcome of last instruction. The length of a straight- line program is the number of instructions. ℓ Exercise: A Boolean function has a straight-line program of length at most if ℓ and only if it has a Boolean circuit of size at most . CS 4810 — Fall'13 Syllabus| Lectures| Homework| Resources Lecture 4: Deterministic Finite Automata The following figure shows the state diagram of a deterministic finite automaton (DFA), 0 1 0 1 q₀ q₁ q₂ 0,1 q q q q The automaton has three states 0, 1, and 2. The state 0 is designated as the q start state and 1 is designated as an accept state. The directed edges between states specify the transitions of the automaton when reading an input string. 1 This automaton accepts the set of strings that contain and have an even 0 1 number of ’s after the last . Languages Σ Σ = {0,1} Σ = a,b,c An alphabet is a finite set of symbols, e.g., or . (We w Σ will usually content ourselves with the binary alphabet.) A string over is a w = x ⋯x x ,…,x ∈ Σ finite sequence of symboles, written as 1 n for 1 n . The ε empty string (string of length zero) is denoted . The set of all strings is denoted Σ∗ L L ⊆ Σ∗ . A language is any subset of strings, so that . L We will use languages to encode computational problems. (A language f:Σ∗ → {0,1} f(w) = 1 can be thought to encode the function with if and w ∈ L only if ). A,B ⊆ Σ∗ Operations on languages: Let be two languages. The following operations on languages are useful: A∪B A∩B » Basic set-theoretic operations, union , intersection , and ¯A¯¯¯ := Σ∗ ∖A complementation . (These operations correspond to , , OR AND NOT.) AB := {uv ∣ u ∈ A,v ∈ B} » Concatentation . Ai i » Kleene star $A^*:=A0A1A^2 $, where denotes the -fold concatenation of A i = 0,1 with itself. (The corner cases are defined so that the rule AiAj = Ai+j A0 = {ε} A1 = A holds. Concretely, and ).

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.