Average-case Complexity
description
Transcript of Average-case Complexity
![Page 1: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/1.jpg)
Average-case Complexity
Luca TrevisanUC Berkeley
![Page 2: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/2.jpg)
Distributional Problem
<P,D>
P computational problem– e.g. SAT
D distribution over inputs– e.g. n vars 10n clauses
![Page 3: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/3.jpg)
Positive Results:• Algorithm that solves P efficiently on most
inputs– Interesting when P useful problem, D
distribution arising “in practice”
Negative Results:• If <assumption>, then no such algorithm– P useful, D natural• guide algorithm design
– Manufactured P,D, • still interesting for crypto, derandomization
![Page 4: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/4.jpg)
Positive Results:• Algorithm that solves P efficiently on most
inputs– Interesting when P useful problem, D
distribution arising “in practice”
Negative Results:• If <assumption>, then no such algorithm– P useful, D natural• guide algorithm design
– Manufactured P,D, • still interesting for crypto, derandomization
![Page 5: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/5.jpg)
Holy Grail
If there is algorithm A that solves P efficiently on most inputs from D
Then there is an efficient worst-case algorithm for [the complexity class] P [belongs to]
![Page 6: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/6.jpg)
Part (1)
In which the Holy Grail proves elusive
![Page 7: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/7.jpg)
The Permanent
Perm (M) := Ss Pi M(i,s(i))
Perm() is #P-complete
Lipton (1990): If there is algorithm that solves Perm()
efficiently on most random matrices, Then there is an algorithm that solves it
efficiently on all matrices (and BPP=#P)
![Page 8: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/8.jpg)
Lipton’s Reduction
Suppose operations are over finite field of size >n
A is good-on-average algorithm (wrong on < 1/(10(n+1)) fraction of matrices)
Given M, pick random X, compute A(M+X), A(M+2X),…,A(M+(n+1)X)
Whp the same as Perm(M+X),Perm(M+2X),…,Perm(M+(n+1)X)
![Page 9: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/9.jpg)
Lipton’s Reduction
Given Perm(M+X),Perm(M+2X),…,Perm(M+
(n+1)X)
Find univariate degree-n polynomial p such thatp(t) = Perm(M+tX) for all t
Output p(0)
![Page 10: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/10.jpg)
Improvements / Generalizations
• Can handle constant fraction of errors[Gemmel-Sudan]
• Works for PSPACE-complete, EXP-complete,…[Feigenbaum-Fortnow, Babai-Fortnow-Nisan-Wigderson]Encode the problem as a polynomial
![Page 11: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/11.jpg)
Strong Average-Case Hardness
• [Impagliazzo, Impagliazzo-Wigderson] Manufacture problems in E, EXP, such that– Size-t circuit correct on ½ + 1/t inputs implies– Size poly(t) circuit correct on all inputs
Motivation:[Nisan-Wigderson]
P=BPP if there is problem in E of exponential average-case complexity
![Page 12: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/12.jpg)
Strong Average-Case Hardness
• [Impagliazzo, Impagliazzo-Wigderson] Manufacture problems in E, EXP, such that– Size-t circuit correct on ½ + 1/t inputs
implies– Size poly(t) circuit correct on all inputs
Motivation:[Impagliazzo-Wigderson]
P=BPP if there is problem in E of exponential average worst-case complexity
![Page 13: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/13.jpg)
Open Question 1
• Suppose there are worst-case intractable problems in NP
• Are there average-case intractable problems?
![Page 14: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/14.jpg)
Strong Average-Case Hardness
• [Impagliazzo, Impagliazzo-Wigderson] Manufacture problems in E, EXP, such that– Size-t circuit correct on ½ + 1/t inputs implies– Size poly(t) circuit correct on all inputs
• [Sudan-T-Vadhan]– IW result can be seen as coding-theoretic
– Simpler proof by explicitly coding-theoretic ideas
![Page 15: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/15.jpg)
Encoding Approach
• Viola proves that an error-correcting code cannot be computed in AC0
• The exponential-size error-correcting code computation not possible in PH
![Page 16: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/16.jpg)
Problem-specific Approaches?
[Ajtai]
• Proves that there is a lattice problem such that:
– If there is efficient average-case algorithm
– There is efficient worst-case approximation algorithm
![Page 17: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/17.jpg)
Ajtai’s Reduction
• Lattice Problem– If there is efficient average-case algorithm– There is efficient worst-case approximation
algorithm
The approximation problem is in NPIcoNPNot NP-hard
![Page 18: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/18.jpg)
Holy Grail
• Distributional Problem:– If there is efficient average-case algorithm– P=NP
(or NP in BPP, or NP has poly-size circuits,…)
Already seen: no “encoding” approach works
Can extensions of Ajtai’s approach work?
![Page 19: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/19.jpg)
A Class of Approaches
• L problem in NP, D distribution of inputs• R reduction of SAT to <L,D>:
• Given instance f of SAT,– R produces instances x1,…,xk of L, each distributed
according to D– Given L(x1),…,L(x1), R is able to decide f
If there is good-on-average algorithn for <L,D>, we solve SAT in polynomial time
[cf. Lipton’s work on Permanent]
![Page 20: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/20.jpg)
A Class of Approaches
• L,W problems in NP, D (samplable) distribution of inputs
• R reduction of W to <L,D>
• Given instance w of W,– R produces instances x1,…,xk of L, each distributed
according to D– Given L(x1),…,L(x1), R is able to decide w
If there is good-on-average algorithm for <L,D>, we solve W in polynomial time;
Can W be NP-complete?
![Page 21: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/21.jpg)
A Class of Approaches
• Given instance w of W,– R produces instances x1,…,xk of L, each
distributed according to D– Given L(x1),…,L(x1), R is able to decide w
Given good-on-average algorithm for <L,D>, we solve W in polynomial time;
If we have such reduction, and W is NP-complete, we have Holy Grail!
Feigenbaum-Fortnow: W is in “coNP”
![Page 22: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/22.jpg)
Feigenbaum-Fortnow
• Given instance w of W,– R produces instances x1,…,xk of L, each
distributed according to D– Given L(x1),…,L(x1), R is able to decide w
• Using R, Feigenbaum-Fortnow design a 2-round interactive proof with advice for coW
• Given w, Prover convinces Verifier that R rejects w after seeing L(x1),…,L(x1)
![Page 23: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/23.jpg)
Feigenbaum-Fortnow
• Given instance w of W,– R produces instances x of L distributed as in D– w in L iff x in L
Suppose we know PrD[ x in L]= ½
VP
w
R(w) = x1
R(w) = x2
. . .R(w) = xm
x1, x2,. . . , xm
(Yes,w1),No,. . . , (Yes, wm)
Accept iff all simulations of R rejectand m/2 +/- sqrt(m) answers are certified Yes
![Page 24: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/24.jpg)
Feigenbaum-Fortnow
• Given instance w of W, p:= Pr[ xi in L]
– R produces instances x1,…,xk of L, each distrib. according to D
– Given L(x1),…,L(xk), R is able to decide wV
wR(w) -> x1
1,…,xk
1 . . .
R(w) -> x1m,
…,xkm
P
x11,…,xk
m
(Yes,w11),…,NO
Accept iff -pkm +/- sqrt(pkm) YES with certificates-R rejects in each case
![Page 25: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/25.jpg)
Generalizations
• Bogdanov-Trevisan: arbitrary non-adaptive reductions
• Main Open Question:What happens with adaptive reductions?
![Page 26: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/26.jpg)
Open Question 1
Prove the following:
Suppose: W,L are in NP, D is samplable distribution,
R is poly-time reduction such that– If A solves <L,D> on 1-1/poly(n) frac of
inputs– Then R with oracle A solves W on all inputs
Then W is in “coNP”
![Page 27: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/27.jpg)
By the Way
• Probably impossible by current techniques:If NP not contained in BPPThere is a samplable distribution D and an NP
problem L Such that <L,D> is hard on average
![Page 28: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/28.jpg)
By the Way
• Probably impossible by current techniques:If NP not contained in BPPThere is a samplable distribution D and an NP
problem L Such that for every efficient A A makes many mistakes solving L on D
![Page 29: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/29.jpg)
By the Way
• Probably impossible by current techniques:If NP not contained in BPPThere is a samplable distribution D and an NP problem
L Such that for every efficient A A makes many mistakes solving L on D
• [Guttfreund-Shaltiel-TaShma] Prove:If NP not contained in BPPFor every efficient A There is a samplable distribution D Such that A makes many mistakes solving SAT on D
![Page 30: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/30.jpg)
Part (2)
In which we amplify average-case complexity and we discuss a short paper
![Page 31: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/31.jpg)
Revised Goal
• Proving“If NP contains worst-case intractable problems, then NP contains average-case intractable problems”
Might be impossible
• Average-case intractability comes in different quantitative degrees
• Equivalence?
![Page 32: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/32.jpg)
Average-Case Hardness
What does it mean for <L,D> to be hard-on-average?
Suppose A is efficient algorithm Sample x ~ DThen A(x) is noticeably likely to be wrong
How noticeably?
![Page 33: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/33.jpg)
Average-Case Hardness Amplification
Ideally:
• If there is <L,Uniform>, L in NP, such that every poly-time algorithm (poly-size circuit) makes > 1/poly(n) mistakes
• Then there is <L’,Uniform>, L’ in NP, such that every poly-time algorithm (poly-size circuit) makes > ½ - 1/poly(n) mistakes
![Page 34: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/34.jpg)
Amplification
“Classical” approach: Yao’s XOR Lemma
Suppose: for every efficient APrD [ A(x) = L(x) ] < 1- d
Then: for every efficient A’ PrD [ A’(x1,…,xk) = L(x1) xor … xor L(xk) ]
< ½ + (1 - 2d)k + negligible
![Page 35: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/35.jpg)
Yao’s XOR Lemma
Suppose: for every efficient APrD [ A(x) = L(x) ] < 1- d
Then: for every efficient A’ PrD [ A’(x1,…,xk) = L(x1) xor … xor L(xk) ]
< ½ + (1 - 2d)k + negligible
Note: computing L(x1) xor … xor L(xk) need not be in NP, even if L is in NP
![Page 36: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/36.jpg)
O’Donnell Approach
Suppose: for every efficient APrD [ A(x) = L(x) ] < 1- d
Then: for every efficient A’ PrD [ A’(x1,…,xk) = g(L(x1), …, L(xk)) ]
< ½ + small(k, d)
For carefully chosen monotone function g
Now computing g(L(x1),…, L(xk)) is in NP, if L is in NP
![Page 37: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/37.jpg)
Amplification (Circuits)
Ideally:• If there is <L,Uniform>, L in NP, such that every poly-time
algorithm (poly-size circuit) makes > 1/poly(n) mistakes• Then there is <L’,Uniform>, L’ in NP, such that every poly-
time algorithm (poly-size circuit) makes > ½ - 1/poly(n) mistakes
Achieved by [O’Donnell, Healy-Vadhan-Viola] for poly-size circuits
![Page 38: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/38.jpg)
Amplification (Algorithms)
• If there is <L,Uniform>, L in NP, such that every poly-time algorithm makes > 1/poly(n) mistakes
• Then there is <L’,Uniform>, L’ in NP, such that every poly-time algorithm makes > ½ - 1/polylog(n) mistakes
[T]
[Impagliazzo-Jaiswal-Kabanets-Wigderson] ½ - 1/poly(n) but for PNP||
![Page 39: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/39.jpg)
Open Question 2
Prove:
• If there is <L,Uniform>, L in NP, such that every poly-time algorithm makes > 1/poly(n) mistakes
• Then there is <L’,Uniform>, L’ in NP, such that every poly-time algorithm makes > ½ - 1/poly(n) mistakes
![Page 40: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/40.jpg)
Completeness
• Suppose we believe there is L in NP, D distribution, such that <L,D> is hard
• Can we point to a specific problem C such that <C,Uniform> is also hard?
![Page 41: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/41.jpg)
Completeness
• Suppose we believe there is L in NP, D distribution, such that <L,D> is hard
• Can we point to a specific problem C such that <C,Uniform> is also hard?
Must put restriction on D, otherwise assumption is the same as P != NP
![Page 42: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/42.jpg)
Side Note
Let K be distribution such that x has probability proportional to 2-K(x)
Suppose A solves <L,K> on 1-1/poly(n) fraction of inputs of length n
Then A solves L on all but finitely many inputs
Exercise: prove it
![Page 43: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/43.jpg)
Completeness
• Suppose we believe there is L in NP, D samplable distribution, such that <L,D> is hard
• Can we point to a specific problem C such that <C,Uniform> is also hard?
![Page 44: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/44.jpg)
Completeness
• Suppose we believe there is L in NP, D samplable distribution, such that <L,D> is hard
• Can we point to a specific problem C such that <C,Uniform> is also hard?
Yes we can! [Levin, Impagliazzo-Levin]
![Page 45: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/45.jpg)
Levin’s Completeness Result
• There is an NP problem C, such that
• If there is L in NP, D computable distribution, such that <L,D> is hard
• Then <C,Uniform> is also hard
![Page 46: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/46.jpg)
![Page 47: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/47.jpg)
Reduction
Need to define reduction that preserves efficiency on average
(Note: we haven’t yet defined efficiency on average)
R is a (Karp) average-case reduction from <A,DA> to <B,DB> if
1. x in A iff R(x) in B2. R(DA) is “dominated” by DB:
Pr[ R(DA)=y] < poly(n) * Pr [DB = y]
![Page 48: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/48.jpg)
Reduction
R is an average-case reduction from <A, DA> to <B, DB> if
• x in A iff R(x) in B• R(DA) is “dominated” by DB:
Pr[ R(DA)=y] < poly(n) * Pr [DB = y]
Suppose we have good algorithm for <B, DB>
Then algorithm also good for <B,R(DA)>
Solving <A, DA> reduces to solving <B,R(DA)>
![Page 49: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/49.jpg)
Reduction
If Pr[ Y=y] < poly(n) * Pr [DB = y]
and we have good algorithm for <B, DB >
Then algorithm also good for <B,Y>
Reduction works for any notion of average-case tractability for which above is true.
![Page 50: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/50.jpg)
Levin’s Completeness Result
Follow presentation of [Goldreich]
• If <BH,Uniform> is easy on average
• Then for every L in NP, every D computable distribution, <L,D> is easy on average
BH is non-deterministic Bounded Halting: given <M,x,1t>,does M(x) accept with t steps?
![Page 51: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/51.jpg)
Levin’s Completeness Result
BH, non-deterministic Bounded Halting: given <M,x,1t>,does M(x) accept with t steps?
Suppose we have good-on-average alg A
Want to solve <L,D>, where L solvable by NDTM M
First try: x -> <M,x, 1poly(n)>
![Page 52: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/52.jpg)
Levin’s Completeness Result
First try: x -> <M,x, 1poly(n)>
Doesn’t work: x may have arbitrary distribution, we need target string to be nearly uniform (high entropy)
Second try: x -> <M’,C(x), 1poly(n)>Where C() is near-optimal compression alg,
M’ recover x from C(x), then runs M
![Page 53: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/53.jpg)
Levin’s Completeness Result
Second try: x -> <M’,C(x), 1poly(n)>Where C() is near-optimal compression alg,
M’ recover x from C(x), then runs M
Works! Provided C(x) has length at mostO(log n) + log 1/PrD[x]
Possible if cumulative distribution function of D is computable.
![Page 54: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/54.jpg)
Impagliazzo-Levin
Do the same but for all samplable distribution
Samplable distribution not necessarily efficiently compressible in coding theory sense. (E.g. output of PRG)
Hashing provides “non-constructive” compression
![Page 55: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/55.jpg)
Complete Problems
BH with Uniform distribution
Tiling problem with Uniform distribution [Levin]
Generalized edge-coloring [Venkatesan-Levin] Matrix representability [Venkatesan-
Rajagopalan]Matrix transformation [Gurevich]. . .
![Page 56: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/56.jpg)
Open Question 3
L in NP, M NDTM for L is specified by k bits
Levin’s reduction incurs 2k bits in fraction of “problematic” inputs(comparable to having 2k slowdown)
Limited to problems having non-deterministic algorithm of 5 bytes
Inherent?
![Page 57: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/57.jpg)
More Reductions?
Still relatively few complete problems
Similar to study of inapproximability before Papadimitriou-Yannakakis and PCP
Would be good, as in Papadimitriou-Yannakakis, to find reductions between problems that are not known to be complete but are plausibly hard
![Page 58: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/58.jpg)
Open Question 4
(Heard from Russell Impagliazzo)
Prove that
If 3SAT is hard on instances with n variables and 10n clauses,
Then it is also hard on instances with 12n clauses
![Page 59: Average-case Complexity](https://reader035.fdocuments.in/reader035/viewer/2022070405/56813e86550346895da8c251/html5/thumbnails/59.jpg)
See
• http://www.cs.berkeley.edu/~luca/average[slides, references, addendum to Bogdanov-T, coming soon]
• http://www.cs.uml.edu/~wang/acc-forum/ [average-case complexity forum]
• Impagliazzo A personal view of average-case complexityStructures’95
• Goldreich Notes on Levin’s theory of average-case complexityECCC TR-97-56
• Bogdanov-T. Average case complexityF&TTCS 2(1): (2006)