Fall 2013 CMU CS 15-855 Computational Complexity
-
Upload
amethyst-whitley -
Category
Documents
-
view
29 -
download
0
description
Transcript of Fall 2013 CMU CS 15-855 Computational Complexity
Fall 2013 CMU CS 15-855Computational Complexity
Lecture 1
Introduction
April 2, 2013
April 2, 2013 2
Complexity Theory
Classify problems according to the computational resources required – running time– storage space– parallelism– randomness– rounds of interaction, communication, others…
Attempt to answer: what is computationally feasible with limited resources?
April 2, 2013 3
The central questions• Is finding a solution as easy as recognizing one?
P = NP? • Is every efficient sequential algorithm parallelizable?
P = NC?• Can every efficient algorithm be converted into one that
uses a tiny amount of memory? P = L?
• Are there small Boolean circuits for all problems that require exponential running time?
EXP P/poly?• Can every efficient randomized algorithm be converted
into a deterministic algorithm one?P = BPP?
April 2, 2013 4
Languages
• Let ∑ be any finite alphabet. • L µ ∑*
• L is a language.• The decision problem corresponding to L is
the function: f:∑* ! {yes, no} deciding membership in L, i.e., {x | f(x)=‘yes}
• Examples:– set of strings encoding satisfiable formulas– set of strings that encode pairs (n,k) for which n
has factor < k
April 2, 2013 5
Historical:One Tape Turing Machine
. . .
finite control
a b a b
infinite tape
read/write head
April 2, 2013 6
Turing Machines• Q finite set of states
• ∑ alphabet including blank: “_”
• qstart, qaccept, qreject in Q
• δ : Q x ∑ ! Q x ∑ x {L, R, -} transition fn.
• input written on tape, head on 1st square, state qstart
• sequence of steps specified by δ
• if reach qaccept or qreject then halt
April 2, 2013 7
Turing Machines
• three notions of computation with Turing machines. In all, input x written on tape…– function computation: output f(x) is left on
the tape when TM halts
– language decision: TM halts in state qaccept if x L; TM halts in state qreject if x L.
– language recognition: TM halts in state qaccept if x L; may loop forever otherwise.
April 2, 2013 8
q σ δ(q,σ)
start 0 (start, 0, R)
start 1 (start, 1, R)
start _ (t, _, L)
start # (start, #, R)
# 0 1
# 0 1
# 0 1
# 0 1
# 0 1
# 0 0
start
start
start
start
t
t
# 1 0 accept
Example:
q σ δ(q,σ)
t 0 (accept, 1, -)
t 1 (t, 0, L)
t # (accept, #, R)
April 2, 2013 9
Model for this class;Multi-tape Turing Machines
• Read-only Input tape; write only output
. . . finite control
a b a b
a a
b b c d
. . .
. . .
k tapes
δ:Q x ∑k ! Q x ∑k x {L,R,-}k
(input tape)
read-only “input tape”• write-only “output tape”• k-2 read/write “work tapes”
April 2, 2013 10
Turing Machines• Q finite set of states
• ∑ alphabet including blank: “_”
• qstart, qaccept, qreject in Q
• δ:Q x ∑k ! Q x ∑k x {L,R,-}k transition fn.
• input written on tape, head on 1st square, state qstart
• sequence of steps specified by δ
• if reach qaccept or qreject then halt
April 4, 2013 11
TIME and SPACETIME(f(n)) = languages decidable by a multi-tape TM in at most f(n) steps, where n is the input length, and f :N ! N
SPACE(f(n)) = languages decidable by a multi-tape TM that touches at most f(n) squares of its work tapes, where n is the input length, and f :N ! N
L = k SPACE(k log(n)) = SPACE(log(n))
April 4, 2013 12
Robust Time and Space Classes
L = SPACE(log n)
PSPACE = k SPACE(nk)
P = k TIME(nk)
EXP = k TIME(2nk)
April 4, 2013 13
Conjecture: P /neq NPP = k TIME(nk)
= the set of languages decidable in polynomial time
NP = the set of languages L where
L = { x : 9 y, |y| ≤ |x|k, (x, y) R }
and R is a language in P
April 2, 2013 14
The world we expect:• Is finding a solution as easy as recognizing one?
P = NP? • Is every sequential algorithm parallelizable?
P = NC?• Can every efficient algorithm be converted into one that
uses a tiny amount of memory? P = L?
• Are there small Boolean circuits for all problems that require exponential running time?
EXP P/poly?• Can every randomized algorithm be converted into a
deterministic algorithm one?P = BPP?
probably FALSE
probably FALSE
probably FALSE
probably FALSE
probably TRUE
April 2, 2013 15
A question
• Given: polynomial f(x1, x2, …, xn) as arithmetic formula (fan-out 1):
• Question: is f identically zero?
-
*
x1 x2
*
+ -
x3 … xn
*
• multiplication (fan-in 2)
• addition (fan-in 2)
• negation (fan-in 1)
April 2, 2013 16
A question
• Given: multivariate polynomial
f(x1, x2, …, xn)
as an arithmetic formula.
• Question: is f identically zero?
• Challenge: devise a deterministic poly-time algorithm for this problem.
April 2, 2013 17
A randomized algorithm
• Given: multivariate degree r poly. f(x1, x2, …, xd)note: r = deg(f) · size of formula
• Algorithm:– pick small number of random points– if f is zero on all of these points, answer “yes”– otherwise answer “no”
(low-degree non-zero polynomial evaluates to zero on only a small fraction of its domain)
• No efficient deterministic algorithm known
April 23, 2013 18
Polynomial identity testing
Lemma (Schwartz-Zippel): Let
p(x1, x2, …, xn)
be a total degree d polynomial over a field F and let S be any subset of F. Then if p is not identically 0,
Prr1,r2,…,rnS[ p(r1, r2, …, rn) = 0] ≤ d/|S|.
April 23, 2013 19
Polynomial identity testing
• Proof:– induction on number of variables n– base case: n = 1, p is univariate polynomial of
degree at most d– at most d roots, so
Pr[ p(r1) = 0] ≤ d/|S|
April 23, 2013 20
Polynomial identity testing
– write p(x1, x2, …, xn) as
p(x1, x2, …, xn) = Σi (x1)i pi(x2, …, xn)
– k = max. i for which pi(x2, …, xn) not id. zero
– by induction hypothesis:
Pr[ pk(r2, …, rn) = 0] ≤ (d-k)/|S|
– whenever pk(r2, …, rn) ≠ 0, p(x1, r2, …, rn) is a univariate polynomial of degree k
Pr[p(r1,r2,…,rn)=0 | pk(r2,…,rn) ≠ 0] ≤ k/|S|
April 23, 2013 21
Polynomial identity testing
Pr[ pk(r2, …, rn) = 0] ≤ (d-k)/|S|
Pr[p(r1,r2,…,rn)=0 | pk(r2,…,rn) ≠ 0] ≤ k/|S|
– conclude:
Pr[ p(r1, …, rn) = 0] ≤ (d-k)/|S| + k/|S| = d/|S|
– Note: can add these probabilities because
Pr[E1] = Pr[E1|E2]Pr[E2] + Pr[E1|E2]Pr[ E2]
≤ Pr[E2] + Pr[E1|E2]
April 23, 2013 22
Polynomial identity testing
• Given: polynomial p(x1, x2, …, xn)
• Is p identically zero?
• Note: degree d is at most the size of input
-
*
x1 x2
*
+ -
x3 … xn
*
April 23, 2013 23
Polynomial identity testing
• randomized algorithm: field F, pick a subset S F of size 2d– pick r1, r2, …, rn from S uniformly at random– if p(r1, r2, …, rn) = 0, answer “yes”– if p(r1, r2, …, rn) ≠ 0, answer “no”
• if p identically zero, never wrong• if not, Schwartz-Zippel ensures probability
of error at most ½
April 2, 2013 24
Derandomization
• Here is a deterministic algorithm that works under the assumption that there exist hard problems, say SAT.
• solve SAT on all instances of length log n
• encode using error-correcting code (variant of a Reed-Muller code)
1 1 0 0 1 1 1 0 0 1
1 1 0 0 1 1 1 0 0 1 1 1 0 0 1 1 1 0 0 1
April 2, 2013 25
Derandomization
• run randomized alg. using these strings in place of random evaluation points– if f is zero on all of these points, answer “yes”– otherwise answer “no”
• This works. (proof in this course)
1 1 0 0 1 1 1 0 0 1 1 1 0 0 1 1 1 0 0 1
1 1 0 0 1 1 0 0 1 1 0 0 1 1 1
April 2, 2013 26
Derandomization
This technique works on any randomized algorithm.
Gives generic “derandomization” of randomized procedures.
April 2, 2013 27
The world we expect:• Is finding a solution as easy as recognizing one?
P = NP? • Is every sequential algorithm parallelizable?
P = NC?• Can every efficient algorithm be converted into one that
uses a tiny amount of memory? P = L?
• Are there small Boolean circuits for all problems that require exponential running time?
EXP P/poly?• Can every randomized algorithm be converted into a
deterministic algorithm one?P = BPP?
probably FALSE
probably FALSE
probably FALSE
probably FALSE
probably TRUE
April 2, 2013 28
Complexity Classes
• complexity classes you know:– P = the set of languages decidable in
polynomial time – NP = the set of languages L where
L = { x : 9 y, |y| ≤ |x|k, (x, y) R }
and R is a language in P
• easy to define complexity classes…
April 2, 2013 29
TMs formalize our intuitive notion of an efficient algorithm is
•quantum computers challenge this belief
The “extended” Church-Turing Thesis
everything we can compute in time t(n), space s(n) on a physical computer can be computed on a Turing Machine in time tO(1)(n) (polynomial slowdown) in
the same space s(n)
April 2, 2013 30
Turing Machines
• Amazing fact: there exist (natural) undecidable problems
HALT = { (M, x) : M halts on input x }
• Theorem: HALT is undecidable.
April 2, 2013 31
Turing Machines
• Proof:– Suppose TM H decides HALT– Define new TM H’: on input <M>
• if H accepts (M, <M>) then loop• if H rejects (M, <M>) then halt
– Consider H’ on input <H’>:• if it halts, then H rejects (H’, <H’>), which implies it
cannot halt• if it loops, then H accepts (H’, <H’>) which implies
it must halt
– contradiction.
April 2, 2013 32
Diagonalization
Turing Machines
inputs
Y
n
Y
n
n
Y
n
Y n Y Y nn YH’ :
box (M, x): does M halt on x?
The existence of H which tells us yes/no for each box allows us to construct a TM H’ that cannot be in the table.