CSC373 Week 11: Randomized Algorithmsnisarg/teaching/373f20/...Randomized Algorithms 373F20 - Nisarg...
Transcript of CSC373 Week 11: Randomized Algorithmsnisarg/teaching/373f20/...Randomized Algorithms 373F20 - Nisarg...
-
CSC373
Week 11: Randomized Algorithms
373F20 - Nisarg Shah 1
-
Randomized Algorithms
373F20 - Nisarg Shah 2
Deterministic AlgorithmInput Output
Randomized AlgorithmInput
OutputRandomness
-
Randomized Algorithms
373F20 - Nisarg Shah 3
โข Running time
โข Harder goal: the running time should always be small
o Regardless of both the input and the random coin flips
โข Easier goal: the running time should be small in expectation
o Expectation over random coin flips
o But it should still be small for every input (i.e. worst-case)
โข Approximation Ratio
โข The objective value of the solution returned should, in expectation, be close to the optimum objective value
o Once again, the expectation is over random coin flips
o The approximation ratio should be small for every input
-
Derandomization
373F20 - Nisarg Shah 4
โข After coming up with a randomized approximation algorithm, one might ask if it can be โderandomizedโโข Informally, the randomized algorithm is making random choices that,
in expectation, turn out to be good
โข Can we make these โgoodโ choices deterministically?
โข For some problemsโฆโข It may be easier to first design a simple randomized approximation
algorithm and then de-randomize itโฆ
โข Than to try to directly design a deterministic approximation algorithm
-
Recap: Probability Theory
373F20 - Nisarg Shah 5
โข Random variable ๐โข Discrete
o Takes value ๐ฃ1 with probability ๐1, ๐ฃ2 w.p. ๐2, โฆ
o Expected value ๐ธ ๐ = ๐1 โ ๐ฃ1 + ๐2 โ ๐ฃ2 + โฏ
o Examples: coin toss, the roll of a six-sided die, โฆ
โข Continuous
o Has a probability density function (pdf) ๐
o Its integral is the cumulative density function (cdf) ๐น
โข ๐น ๐ฅ = Pr ๐ โค ๐ฅ = โโ๐ฅ
๐ ๐ก ๐๐ก
o Expected value ๐ธ ๐ = โโโ
๐ฅ ๐ ๐ฅ ๐๐ฅ
o Examples: normal distribution, exponential distribution, uniform distribution over [0,1], โฆ
-
Recap: Probability Theory
373F20 - Nisarg Shah 6
โข Things you should be aware ofโฆโข Conditional probabilities
โข Conditional expectations
โข Independence among random variables
โข Moments of random variables
โข Standard discrete distributions: uniform over a finite set, Bernoulli, binomial, geometric, Poisson, โฆ
โข Standard continuous distributions: uniform over intervals, Gaussian/normal, exponential, โฆ
-
Three Pillars
373F20 - Nisarg Shah 7
โข Deceptively simple, but incredibly powerful!
โข Many many many many probabilistic results are just interesting applications of these three results
Linearity of Expectation Union Bound Chernoff Bound
-
Three Pillars
373F20 - Nisarg Shah 8
โข Linearity of expectationโข ๐ธ ๐ + ๐ = ๐ธ ๐ + ๐ธ[๐]
โข This does not require any independence assumptions about ๐ and ๐
โข E.g. if you want to find out how many people will attend your party on average, just ask each person the probability with which they will attend and sum up the probabilities
o It does not matter whether some of them are friends and either all will attend together or none will attend
-
Three Pillars
373F20 - Nisarg Shah 9
โข Union boundโข For any two events ๐ด and ๐ต, Pr ๐ด โช ๐ต โค Pr ๐ด + Pr[๐ต]
โข โProbability that at least one of the ๐ events ๐ด1, โฆ , ๐ด๐ will occur is at most ฯ๐ Pr ๐ด๐ โ
โข Typically, ๐ด1, โฆ , ๐ด๐ are โbad eventsโ
o You do not want any of them to occur
o If you can individually bound Pr ๐ด๐ โค ฮค1
2๐ for each ๐, then probability that at least one them occurs โค ฮค1 2
o Thus, with probability โฅ ฮค1 2, none of the bad events will occur
โข Chernoff bound & Hoeffdingโs inequalityโข Read up!
-
373F20 - Nisarg Shah 10
Exact Max-๐-SAT
-
Exact Max-๐-SAT
373F20 - Nisarg Shah 11
โข Problem (recall)โข Input: An exact ๐-SAT formula ๐ = ๐ถ1 โง ๐ถ2 โง โฏ โง ๐ถ๐,
where each clause ๐ถ๐ has exactly ๐ literals, and a weight ๐ค๐ โฅ 0 of each clause ๐ถ๐
โข Output: A truth assignment ๐ maximizing the number (or total weight) of clauses satisfied under ๐
โข Let us denote by ๐(๐) the total weight of clauses satisfied under ๐
-
Exact Max-๐-SAT
373F20 - Nisarg Shah 12
โข Recall our local searchโข ๐๐(๐) = set of all truth assignments which can be obtained by
changing the value of at most ๐ variables in ๐
โข Result 1: Neighborhood ๐1(๐) โ ฮค2
3-apx for Exact Max-2-SAT.
โข Result 2: Neighborhood ๐1 ๐ โช ๐๐ โ ฮค3 4-apx for Exact Max-
2-SAT.
โข Result 3: Neighborhood ๐1 ๐ + oblivious local search โ ฮค3
4-apx for Exact Max-2-SAT.
-
Exact Max-๐-SAT
373F20 - Nisarg Shah 13
โข Recall our local searchโข ๐๐(๐) = set of all truth assignments which can be obtained by
changing the value of at most ๐ variables in ๐
โข We claimed that ยพ-apx for Exact Max-2-SAT can be
generalized to 2๐โ1
2๐-apx for Exact Max-๐-SAT
โข Algorithm becomes slightly more complicated
โข What can we do with randomized algorithms?
-
Exact Max-๐-SAT
373F20 - Nisarg Shah 14
โข Recall: โข We have a formula ๐ = ๐ถ1 โง ๐ถ2 โง โฏ โง ๐ถ๐โข Variables = ๐ฅ1, โฆ , ๐ฅ๐, literals = variables or their negations
โข Each clause contains exactly ๐ literals
โข The most naรฏve randomized algorithmโข Set each variable to TRUE with probability ยฝ and to FALSE with
probability ยฝ
โข How good is this?
-
Exact Max-๐-SAT
373F20 - Nisarg Shah 15
โข Recall: โข We have a formula ๐ = ๐ถ1 โง ๐ถ2 โง โฏ โง ๐ถ๐โข Variables = ๐ฅ1, โฆ , ๐ฅ๐, literals = variables or their negations
โข Each clause contains exactly ๐ literals
โข Let ๐ be a random assignment
โข For each clause ๐ถ๐: Pr[๐ถ๐ is not satisfied] = เต1
2๐(WHY?)
o Hence, Pr[๐ถ๐ is satisfied] = เต2๐โ1
2๐
โข ๐ธ ๐ ๐ = ฯ๐=1๐ ๐ค๐ โ Pr[๐ถ๐ is satisfied] (WHY?)
โข ๐ธ ๐ ๐ =2๐โ1
2๐โ ฯ๐=1
๐ ๐ค๐ โฅ2๐โ1
2๐โ ๐๐๐
-
Derandomization
373F20 - Nisarg Shah 16
โข Can we derandomize this algorithm?โข What are the choices made by the algorithm?
o Setting the values of ๐ฅ1, ๐ฅ2, โฆ , ๐ฅ๐โข How do we know which set of choices is good?
โข Idea:โข Do not think about all the choices at once.
โข Think about them one by one.
โข Goal: Gradually convert the random assignment ๐ to a deterministic assignment ฦธ๐ such that ๐ ฦธ๐ โฅ ๐ธ ๐ ๐
o Combining with ๐ธ ๐ ๐ โฅ2๐โ1
2๐โ ๐๐๐ will give the desired
deterministic approximation ratio
-
Derandomization
373F20 - Nisarg Shah 17
โข Start with the random assignment ๐ and writeโฆ
๐ธ ๐ ๐ = Pr ๐ฅ1 = ๐ โ ๐ธ ๐ ๐ ๐ฅ1 = ๐ + Pr ๐ฅ1 = ๐น โ ๐ธ ๐ ๐ ๐ฅ1 = ๐น= เต1 2 โ ๐ธ ๐ ๐ ๐ฅ1 = ๐ + เต
12 โ ๐ธ ๐ ๐ ๐ฅ1 = ๐น
โข Hence, max ๐ธ ๐ ๐ ๐ฅ1 = ๐ , ๐ธ ๐ ๐ ๐ฅ1 = ๐น โฅ ๐ธ ๐ ๐
o What is ๐ธ ๐ ๐ ๐ฅ1 = ๐ ?
โข It is the expected weight when setting ๐ฅ1 = ๐ deterministically but still keeping ๐ฅ2, โฆ , ๐ฅ๐ random
โข If we can compute both ๐ธ ๐ ๐ ๐ฅ1 = ๐ and ๐ธ ๐ ๐ ๐ฅ1 = ๐น , and pick the better oneโฆ
o Then we can set ๐ฅ1 deterministically without degrading the expected objective value
-
โข After deterministically making the right choice for ๐ฅ1 (say T), we can apply the same logic to ๐ฅ2
๐ธ ๐ ๐ |๐ฅ1 = ๐ = เต1
2 โ ๐ธ ๐ ๐ ๐ฅ1 = ๐, ๐ฅ2 = ๐
+ เต1 2 โ ๐ธ ๐ ๐ ๐ฅ1 = ๐, ๐ฅ2 = ๐น
โข Pick the better of the two conditional expectations
โข Derandomized Algorithm:โข For ๐ = 1, โฆ , ๐
o Let ๐ง๐ = ๐ if ๐ธ ๐ ๐ ๐ฅ1 = ๐ง1, โฆ , ๐ฅ๐โ1 = ๐ง๐โ1, ๐ฅ๐ = ๐ โฅ๐ธ ๐ ๐ ๐ฅ1 = ๐ง1, โฆ , ๐ฅ๐โ1 = ๐ง๐โ1, ๐ฅ๐ = ๐น , and ๐ง๐ = ๐น otherwise
o Set ๐ฅ๐ = ๐ง๐
Derandomization
373F20 - Nisarg Shah 18
-
โข This is called the method of conditional expectationsโข If weโre happy when making a choice at random, we should be at least
as happy conditioned on at least one of the possible values of that choice
โข Remaining question:โข How do we compute & compare the two conditional expectations:
๐ธ ๐ ๐ ๐ฅ1 = ๐ง1, โฆ , ๐ฅ๐โ1 = ๐ง๐โ1, ๐ฅ๐ = ๐ and ๐ธ ๐ ๐ ๐ฅ1 = ๐ง1, โฆ , ๐ฅ๐โ1 = ๐ง๐โ1, ๐ฅ๐ = ๐น ?
Derandomization
373F20 - Nisarg Shah 19
-
Derandomization
373F20 - Nisarg Shah 20
โข ๐ธ ๐ ๐ ๐ฅ1 = ๐ง1, โฆ , ๐ฅ๐โ1 = ๐ง๐โ1, ๐ฅ๐ = ๐โข ฯ๐ ๐ค๐ โ Pr[๐ถ๐ is satisfied ๐ฅ1 = ๐ง1, โฆ , ๐ฅ๐โ1 = ๐ง๐โ1, ๐ฅ๐ = ๐โข Set the values of ๐ฅ1, โฆ , ๐ฅ๐โ1, ๐ฅ๐โข If ๐ถ๐ resolves to TRUE already, the corresponding probability is 1
โข If ๐ถ๐ resolves to FALSE already, the corresponding probability is 0
โข Otherwise, if there are โ literals left in ๐ถ๐ after setting ๐ฅ1, โฆ , ๐ฅ๐โ1, ๐ฅ๐,
the corresponding probability is 2โโ1
2โ
โข Compute ๐ธ ๐ ๐ ๐ฅ1 = ๐ง1, โฆ , ๐ฅ๐โ1 = ๐ง๐โ1, ๐ฅ๐ = ๐น similarly
-
Max-SAT
373F20 - Nisarg Shah 21
โข Simple randomized algorithm
โข
2๐โ1
2๐โapproximation for Max-๐-SAT
โข Max-3-SAT โ ฮค7 8o [Hรฅstad]: This is the best possible assuming P โ NP
โข Max-2-SAT โ ฮค3 4 = 0.75
o The best known approximation is 0.9401 using semi-definite programming and randomized rounding
โข Max-SAT โ ฮค1 2o Max-SAT = no restriction on the number of literals in each clause
o The best known approximation is 0.7968, also using semi-definite programming and randomized rounding
-
Max-SAT
373F20 - Nisarg Shah 22
โข Better approximations for Max-SATโข Semi-definite programming is out of the scope
โข But we will see the simpler โLP relaxation + randomized roundingโ approach that gives 1 โ ฮค1 ๐ โ 0.6321 approximation
โข Max-SAT:โข Input: ๐ = ๐ถ1 โง ๐ถ2 โง โฏ โง ๐ถ๐, where each clause ๐ถ๐ has weight ๐ค๐ โฅ
0 (and can have any number of literals)
โข Output: Truth assignment that approximately maximizes the weight of clauses satisfied
-
LP Formulation of Max-SAT
373F20 - Nisarg Shah 23
โข First, IP formulation:โข Variables:
o ๐ฆ1, โฆ , ๐ฆ๐ โ {0,1}
โข ๐ฆ๐ = 1 iff variable ๐ฅ๐ = TRUE in Max-SAT
o ๐ง1, โฆ , ๐ง๐ โ {0,1}
โข ๐ง๐ = 1 iff clause ๐ถ๐ is satisfied in Max-SAT
o Program:
Maximize ฮฃ๐ ๐ค๐ โ ๐ง๐s.t.ฮฃ๐ฅ๐โ๐ถ๐ ๐ฆ๐ + ฮฃ าง๐ฅ๐โ๐ถ๐ 1 โ ๐ฆ๐ โฅ ๐ง๐ โ๐ โ 1, โฆ , ๐
๐ฆ๐ , ๐ง๐ โ 0,1 โ๐ โ 1, โฆ , ๐ , ๐ โ 1, โฆ , ๐
-
LP Formulation of Max-SAT
373F20 - Nisarg Shah 24
โข LP relaxation:โข Variables:
o ๐ฆ1, โฆ , ๐ฆ๐ โ [0,1]
โข ๐ฆ๐ = 1 iff variable ๐ฅ๐ = TRUE in Max-SAT
o ๐ง1, โฆ , ๐ง๐ โ [0,1]
โข ๐ง๐ = 1 iff clause ๐ถ๐ is satisfied in Max-SAT
o Program:
Maximize ฮฃ๐ ๐ค๐ โ ๐ง๐s.t.ฮฃ๐ฅ๐โ๐ถ๐ ๐ฆ๐ + ฮฃ าง๐ฅ๐โ๐ถ๐ 1 โ ๐ฆ๐ โฅ ๐ง๐ โ๐ โ 1, โฆ , ๐
๐ฆ๐ , ๐ง๐ โ [0,1] โ๐ โ 1, โฆ , ๐ , ๐ โ 1, โฆ , ๐
-
Randomized Rounding
373F20 - Nisarg Shah 25
โข Randomized roundingโข Find the optimal solution (๐ฆโ, ๐งโ) of the LP
โข Compute a random IP solution เท๐ฆ such that
o Each เท๐ฆ๐ = 1 with probability ๐ฆ๐โ and 0 with probability 1 โ ๐ฆ๐
โ
o Independently of other เท๐ฆ๐โs
o The output of the algorithm is the corresponding truth assignment
โข What is Pr[๐ถ๐ is satisfied] if ๐ถ๐ has ๐ literals?
1 โ ฮ ๐ฅ๐โ๐ถ๐ 1 โ ๐ฆ๐โ โ ฮ าง๐ฅ๐โ๐ถ๐ ๐ฆ๐
โ
โฅ 1 โฮฃ๐ฅ๐โ๐ถ๐ 1 โ ๐ฆ๐
โ + ฮฃ าง๐ฅ๐โ๐ถ๐ ๐ฆ๐โ
๐
๐
โฅ 1 โ๐ โ ๐ง๐
โ
๐
๐
AM-GM inequality LP constraint
-
Randomized Rounding
373F20 - Nisarg Shah 26
โข Claim
โข 1 โ 1 โ๐ง
๐
๐โฅ 1 โ 1 โ
1
๐
๐โ ๐ง for all ๐ง โ [0,1] and ๐ โ โ
โข Assuming the claim:
Pr ๐ถ๐ is satisfied โฅ 1 โ๐โ๐ง๐
โ
๐
๐
โฅ 1 โ 1 โ1
๐
๐โ ๐ง๐
โ โฅ 1 โ1
๐โ ๐ง๐
โ
โข Hence,
๐ผ[#weight of clauses satisfied] โฅ 1 โ1
eฯ๐ ๐ค๐ โ ๐ง๐
โ โฅ 1 โ1
๐โ ๐๐๐
Standard inequality
Optimal LP objective โฅ optimal ILP objective
-
Randomized Rounding
373F20 - Nisarg Shah 27
โข Claim
โข 1 โ 1 โ๐ง
๐
๐โฅ 1 โ 1 โ
1
๐
๐โ ๐ง for all ๐ง โ [0,1] and ๐ โ โ
โข Proof of claim:โข True at ๐ง = 0 and ๐ง = 1 (same quantity on both sides)
โข For 0 โค ๐ง โค 1:
o LHS is a convex function
o RHS is a linear function
o Hence, LHS โฅ RHS โ
-
Improving Max-SAT Apx
373F20 - Nisarg Shah 28
โข Best of both worlds:โข Run both โLP relaxation + randomized roundingโ and โnaรฏve
randomized algorithmโ
โข Return the best of the two solutions
โข Claim without proof: This achieves a ฮค3 4 = 0.75 approximation!
o This algorithm can be derandomized.
โข Recall:
o โnaรฏve randomizedโ = independently set each variable to TRUE/FALSE with probability 0.5 each, which only gives ฮค1 2 = 0.5approximation by itself
-
โข Max-2-SAT is NP-hard (we didnโt prove this!)
โข But 2-SAT can be efficiently solvedโข โGiven a 2-CNF formula, check whether all clauses can be satisfied
simultaneously.โ
โข Algorithm:โข Repeatedly eliminate a clause with one literal & set the literal to true
โข Create a graph with each remaining literal as a vertex
โข For every clause (๐ฅ โจ ๐ฆ), add two edges: าง๐ฅ โ ๐ฆ and เดค๐ฆ โ ๐ฅ
o ๐ข โ ๐ฃ means if ๐ข is true, ๐ฃ must be true
โข Formula is satisfiable iff no path from ๐ฅ to าง๐ฅ or าง๐ฅ to ๐ฅ for any ๐ฅ
o Can be checked in polynomial time
Back to 2-SAT
373F20 - Nisarg Shah 29
-
Random Walk + 2-SAT
373F20 - Nisarg Shah 30
โข Hereโs a cute randomized algorithm by Papadimitriou [1991]
โข Algorithm:โข Start with an arbitrary assignment.
โข While there is an unsatisfied clause ๐ถ = (๐ฅ โจ ๐ฆ)
o Pick one of the two literals with equal probability.
o Flip the variable value so that ๐ถ is satisfied.
โข But canโt this hurt the other clauses?โข In a given step, yes.
โข But in expectation, we will still make progress.
-
Random Walk + 2-SAT
373F20 - Nisarg Shah 31
โข Theorem:โข If there is a satisfying assignment ๐โ, then this algorithm reaches a
satisfying assignment in ๐ ๐2 expected time.
โข Proof: โข Fix a satisfying assignment ๐โ
โข Let ๐0 be the starting assignment
โข Let ๐๐ be the assignment after ๐ iterations
โข Consider the โhamming distanceโ ๐๐ between ๐๐ and ๐โ
o Number of coordinates in which the two differ
o ๐๐ โ {0,1, โฆ , ๐}
โข Claim: the algorithm hits ๐๐ = 0 in ๐ ๐2 iterations in expectation,
unless it stops before that
-
Random Walk + 2-SAT
373F20 - Nisarg Shah 32
โข Observation: ๐๐+1 = ๐๐ โ 1 or ๐๐+1 = ๐๐ + 1โข Because we change one variable in each iteration.
โข Claim: Pr ๐๐+1 = ๐๐ โ 1 โฅ 1/2
โข Proof:โข Iteration ๐ considers an unsatisfied clause ๐ถ = (๐ฅ โจ ๐ฆ)
โข ๐โ satisfies at least one of ๐ฅ or ๐ฆ, while ๐๐ satisfies neither
โข Because we pick a literal randomly, w.p. at least ยฝ we pick one where ๐๐ and ๐
โ differ and decrease the distance
โข Q: Why did we need an unsatisfied clause? What if we pick one of ๐variables randomly and flip it?
-
Random Walk 2-SAT
373F20 - Nisarg Shah 33
โข Answer:
โข We want the distance to decrease with probability at least 1
2no
matter how close or far we are from ๐โ
โข If we are already close, choosing a variable at random will likely choose one where ๐ and ๐โ already match
โข Flipping this variable will increase the distance with high probability
โข An unsatisfied clause narrows it down to two variables s.t. ๐ and ๐โ
differ on at least one of them
-
Random Walk + 2-SAT
373F20 - Nisarg Shah 34
โข Observation: ๐๐+1 = ๐๐ โ 1 or ๐๐+1 = ๐๐ + 1
โข Claim: Pr ๐๐+1 = ๐๐ โ 1 โฅ 1/2
โข How does this help?
0 1 2 3 4 ๐5
โฅ1
2โค
1
2โฅ
1
2โค
1
2
-
Random Walk + 2-SAT
373F20 - Nisarg Shah 35
โข How does this help?โข Can view this as a โMarkov chainโ and use known results
on โhitting timeโ
โข But letโs prove it using elementary methods
0 1 2 3 4 ๐5
โฅ1
2โค
1
2โฅ
1
2โค
1
2
-
Random Walk + 2-SAT
373F20 - Nisarg Shah 36
โข For ๐ > โ, define:โข ๐๐,โ = expected number of iterations it takes to hit distance โ
for the first time when you start at distance ๐
โข ๐๐+1,๐ โค1
2โ 1 +
1
2โ 1 + ๐๐+2,๐
=1
2โ (1) +
1
2โ 1 + ๐๐+2,๐+1 + ๐๐+1,๐
โข Simplifying: โข ๐๐+1,๐ โค 2 + ๐๐+2,๐+1 โค 4 + ๐๐+3,๐+2 โค โฏ โค ๐ ๐ + ๐๐,๐โ1 โค ๐ ๐
o Uses ๐๐,๐โ1 = 1 (Why?)
โข ๐๐,0 โค ๐๐,๐โ1 + โฏ + ๐1,0 = ๐ ๐2
Which pillar did we use?
-
Random Walk + 2-SAT
373F20 - Nisarg Shah 37
โข Can view this algorithm as a โdrunken local searchโโข We are searching the local neighborhood
โข But we donโt ensure that we necessarily improve
โข We just ensure that in expectation, we arenโt hurt
โข Hope to reach a feasible solution in polynomial time
โข Schรถning extended this technique to ๐-SATโข Schรถningโs algorithm no longer runs in polynomial time, but this is
okay because ๐-SAT is NP-hard
โข It still improves upon the naรฏve 2๐
โข Later derandomized by Moser and Scheder [2011]
-
Schรถningโs Algorithm for ๐-SAT
373F20 - Nisarg Shah 38
โข Algorithm:โข Choose a random assignment ๐
โข Repeat 3๐ times (๐ = #variables)
o If ๐ satisfies the CNF, stop
o Else, pick an arbitrary unsatisfied clause and flip a random literal in the clause
-
Schรถningโs Algorithm
373F20 - Nisarg Shah 39
โข Randomized algorithm with one-sided errorโข If the CNF is satisfiable, it finds an assignment with probability at
least 1
2โ
๐
๐โ1
๐
โข If the CNF is unsatisfiable, it never finds an assignment
โข Expected #times we need to repeat in order to find a
satisfying assignment when one exists: 2 1 โ 1๐
๐
โข For ๐ = 3, this gives ๐(1.3333๐)
โข For ๐ = 4, this gives ๐ 1.5๐
-
Best Known Results
373F20 - Nisarg Shah 40
โข 3-SAT
โข Deterministicโข Derandomized Schรถningโs algorithm: ๐(1.3333๐)
โข Best known: ๐(1.3303๐) [HSSW]
o If we are assured that there is a unique satisfying assignment: ๐(1.3071๐) [PPSZ]
โข Randomizedโข Nothing better known without one-sided error
โข With one-sided error, best known is ๐ 1.30704๐ [Modified PPSZ]
-
Random Walk + 2-SAT
373F20 - Nisarg Shah 41
โข Random walks are not only of theoretical interestโข WalkSAT is a practical SAT algorithm
โข At each iteration, pick an unsatisfied clause at random
โข Pick a variable in the unsatisfied clause to flip:
o With some probability, pick at random.
o With the remaining probability, pick one that will make the fewest previously satisfied clauses unsatisfied
โข Restart a few times (avoids being stuck in local minima)
โข Faster than โintelligent local searchโ (GSAT)โข Flip the variable that satisfies most clauses
-
Random Walks on Graphs
373F20 - Nisarg Shah 42
โข Aleliunas et al. [1979]โข Let ๐บ be a connected undirected graph. Then a random walk starting
from any vertex will cover the entire graph (visit each vertex at least once) in ๐(๐๐) steps.
โข Limiting probability distribution
โข In the limit, the random walk will visit a vertex with degree ๐๐ in ๐๐
2๐fraction of the steps
โข Markov chains โข Generalize to directed (possibly infinite) graphs with unequal edge
traversal probabilities
-
373F20 - Nisarg Shah 43
Randomization for Sublinear Running Time
-
Sublinear Running Time
373F20 - Nisarg Shah 44
โข Given an input of length ๐, we want an algorithm that runs in time ๐(๐)
โข ๐(๐) examples: log ๐ , ๐, ๐0.999,๐
log ๐, โฆ
โข The algorithm doesnโt even get to read the full input!
โข There are four possibilities:โข Exact vs inexact: whether the algorithm always returns the
correct/optimal solution or only does so with high probability (or gives some approximation)
โข Worst-case versus expected running time: whether the algorithm always takes ๐(๐) time or only does so in expectation (but still on every instance)
-
373F20 - Nisarg Shah 45
Exact algorithms, expected sublinear time
-
Searching in Sorted List
373F20 - Nisarg Shah 46
โข Input: A sorted doubly linked list with ๐ elements.โข Imagine you have an array ๐ด with ๐(1) access to ๐ด[๐]
โข ๐ด[๐] is a tuple (๐ฅ๐ , ๐๐ , ๐๐)
o Value, index of previous element, index of next element.
โข Sorted: ๐ฅ๐๐ โค ๐ฅ๐ โค ๐ฅ๐๐
โข Task: Given ๐ฅ, check if there exists ๐ s.t. ๐ฅ = ๐ฅ๐
โข Goal: We will give a randomized + exact algorithm with expected running time ๐ ๐ !
-
Searching in Sorted List
373F20 - Nisarg Shah 47
โข Motivation: โข Often we deal with large datasets that are stored in a large file on
disk, or possibly broken into multiple files
โข Creating a new, sorted version of the dataset is expensive
โข It is often preferred to โimplicitly sortโ the data by simply adding previous-next pointers along with each element
โข Would like algorithms that can operate on such implicitly sorted versions and yet achieve sublinear running time
o Just like binary search achieves for an explicitly sorted array
-
Searching in Sorted List
373F20 - Nisarg Shah 48
Algorithm:โข Select ๐ random indices ๐
โข Access ๐ฅ๐ for each ๐ โ ๐
โข Find โaccessed ๐ฅ๐ nearest to ๐ฅ in either directionโ
o either the largest among all ๐ฅ๐ โค ๐ฅโฆ
o or the smallest among all ๐ฅ๐ โฅ ๐ฅ
โข If you take the largest ๐ฅ๐ โค ๐ฅ, start from there and keep going โnextโ until you find ๐ฅ or go past its value
โข If you take the smallest ๐ฅ๐ โฅ ๐ฅ, start from there and keep going โpreviousโ until you find ๐ฅ or go past its value
-
Searching in Sorted List
373F20 - Nisarg Shah 49
โข Analysis sketch:โข Suppose you find the largest ๐ฅ๐ โค ๐ฅ and keep going โnextโ
โข Let ๐ฅ๐ be smallest value โฅ ๐ฅ
โข Algorithm stops when it hits ๐ฅ๐โข Algorithm throws ๐ random โdartsโ on the sorted list
โข Chernoff bound:
o Expected distance of ๐ฅ๐ to the closest dart to its left is ๐ ๐
o Weโll assume this without proof!
โข Hence, the algorithm only does โnextโ ๐ ๐ times in expectation
-
Searching in Sorted List
373F20 - Nisarg Shah 50
โข Note:โข We donโt really require the list to be doubly linked. Just โnextโ
pointer suffices if we have a pointer to the first element of the list (a.k.a. โanchored listโ).
โข This algorithm is optimal!
โข Theorem: No algorithm that always returns the correct answer can run in ๐ ๐ expected time.โข Can be proved using โYaoโs minimax principleโ
โข Beyond the scope of the course, but this is a fundamental result with wide-ranging applications
-
Sublinear Geometric Algorithms
373F20 - Nisarg Shah 51
โข Chazelle, Liu, and Magen [2003] proved the ฮ ๐ bound for searching in a sorted linked list
โข Their main focus was to generalize these ideas to come up with sublinear algorithms for geometric problems
โข Polygon intersection: Given two convex polyhedra, check if they intersect.
โข Point location: Given a Delaunay triangulation (or Voronoi diagram) and a point, find the cell in which the point lies.
โข They provided optimal ๐ ๐ algorithms for both these problems.
-
373F20 - Nisarg Shah 52
Inexact algorithms, expected sublinear time
-
Estimating Avg Degree in Graph
373F20 - Nisarg Shah 53
โข Input:โข Undirected graph ๐บ with ๐ vertices
โข ๐ 1 access to the degree of any queried vertex
โข Output: โข Estimate the average degree of all vertices
โข More precisely, we want to find a (2 + ๐)-approximation in expected time ๐ ๐โ๐ 1 ๐
โข Wait!โข Isnโt this equivalent to โgiven an array of ๐ numbers between 1 and
๐ โ 1, estimate their averageโ?
โข No! That requires ฮฉ(๐) time for any constant approximation!
o Consider an instance with constantly many ๐ โ 1โs, and all other 1โs: you may not discover any ๐ โ 1 until you query ฮฉ(๐) numbers
-
Estimating Avg Degree in Graph
373F20 - Nisarg Shah 54
โข Why are degree sequences more special?
โข ErdลsโGallai theorem:
โข ๐1 โฅ โฏ โฅ ๐๐ is a degree sequence iff their sum is even and ฯ๐=1
๐ ๐๐ โค ๐ ๐ โ 1 + ฯ๐=๐+1๐ ๐๐
โข Intuitively, we will sample ๐ ๐ verticesโข We may not discover the few high degree vertices but weโll find their
neighbors and thus account for their edges anyway!
-
Estimating Avg Degree in Graph
373F20 - Nisarg Shah 55
โข Algorithm:
โข Take ฮค8 ๐ random subsets ๐๐ โ ๐ with ๐๐ = ๐๐
๐
โข Compute the average degree ๐๐๐ in each ๐๐.
โข Return ๐ = min๐ ๐๐๐
โข Analysis beyond the scope of this course
โข This gets the approximation right with probability at least 5
6
โข By repeating the experiment ฮฉ log ๐ times and reporting the median answer, we can get the approximation right with probability at least 1 โ 1/๐ ๐ and a bad approximation with the other 1/๐ ๐probability cannot hurt much