Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an...
Transcript of Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an...
![Page 1: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/1.jpg)
The Probabilistic Method
Joshua BrodyCS49/Math59
Fall 2015
Week 11: Applications, Randomized Algorithms
![Page 2: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/2.jpg)
Reading Quiz
(A) memory space (hint: not the right answer)
(B) runtime
(C) power consumption
(D) approximation factor
(E) error guarantee
What resource is typically not an important factor in the analysis of streaming algorithms?
![Page 3: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/3.jpg)
Reading Quiz
(A) memory space (hint: not the right answer)
(B) runtime
(C) power consumption
(D) approximation factor
(E) error guarantee
What resource is typically not an important factor in the analysis of streaming algorithms?
![Page 4: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/4.jpg)
Definition: A streaming algorithm A is an (ɛ,𝜹)-approximation for f if for any stream S
Pr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-𝜹!
PAC Algorithms
F2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)
A(S)
Probably Approximately Correct (PAC) model:“most of the time we’re close enough”
![Page 5: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/5.jpg)
Definition: A streaming algorithm A is an (ɛ,𝜹)-approximation for f if for any stream S
Pr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-𝜹!
Theorem: For any ɛ > 0, there is a streaming algorithm that (ɛ,1/3)-approximates for F2 in O([log(n) + log(m)]/ɛ2) space.
PAC Algorithms
F2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)
A(S)
Probably Approximately Correct (PAC) model:“most of the time we’re close enough”
![Page 6: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/6.jpg)
Clicker Question
(A) no ε makes this work
(B) ε > 1/3
(C) ε > ∏/2
(D) ε > √6
(E) None of the above
For what values of ε does the AMS Basic Estimator have error < 1/3?
![Page 7: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/7.jpg)
Clicker Question
(A) no ε makes this work
(B) ε > 1/3
(C) ε > ∏/2
(D) ε > √6
(E) None of the above
For what values of ε does the AMS Basic Estimator have error < 1/3?
![Page 8: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/8.jpg)
Clicker Question
(A) no k makes this work
(B) k > 6/ε(C) k > 6/ε2
(D) k > √6
(E) None of the above
For arbitrary ε > 0, what values of k give error < 1/3?
![Page 9: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/9.jpg)
Clicker Question
(A) no k makes this work
(B) k > 6/ε(C) k > 6/ε2
(D) k > √6
(E) None of the above
For arbitrary ε > 0, what values of k give error < 1/3?
![Page 10: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/10.jpg)
Improved F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)
X12X22 X32 X42 X52 X62X72X82 X92
AMS Improved Estimator Y:= (∑ Xi2)/k
![Page 11: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/11.jpg)
Improved F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)
X12X22 X32 X42 X52 X62X72X82 X92 Y
AMS Improved Estimator Y:= (∑ Xi2)/k
• k = Ω(1/ε2) : error < 1/3
![Page 12: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/12.jpg)
Improved F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)
AMS Improved Estimator Y:= (∑ Xi2)/k
• k = Ω(1/ε2) : error < 1/3• k = Ω(1/𝜹ε2) : error < 𝜹
X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X52X72X72X722X192Y
![Page 13: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/13.jpg)
Improved F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)
AMS Improved Estimator Y:= (∑ Xi2)/k
• k = Ω(1/ε2) : error < 1/3• k = Ω(1/𝜹ε2) : error < 𝜹
X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X12X22 X32 X42 X52 X62X72X82 X92 X52X72X72X722X192Y
Better error dependence possible with Chernoff Bounds
![Page 14: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/14.jpg)
Final F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)
AMS Improved Estimator Y:= (∑ Xi2)/k
• k = Ω(1/ε2) : error < 1/3
Y
![Page 15: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/15.jpg)
Final F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)
AMS Improved Estimator Y:= (∑ Xi2)/k
• k = Ω(1/ε2) : error < 1/3
Y
AMS Final Estimator median of k’ improved estimators
• k’ = Ω(log(1/𝜹)) : error < 𝜹
YY Y YY Y YY
![Page 16: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/16.jpg)
Final F2 EstimatorF2(S) F2(S)(1+ɛ)F2(S)(1-ɛ)
Y
AMS Improved Estimator Y:= (∑ Xi2)/k
• k = Ω(1/ε2) : error < 1/3
Y
AMS Final Estimator median of k’ improved estimators
• k’ = Ω(log(1/𝜹)) : error < 𝜹
YY Y YY Y Y
![Page 17: Week 11: Applications, Randomized AlgorithmsDefinition: A streaming algorithm A is an (ɛ,!)-approximation for f if for any stream SPr[f(s)(1-ɛ) ≤ A(S) ≤ f(s)(1+ɛ)] ≥ 1-!!Theorem:](https://reader033.fdocuments.in/reader033/viewer/2022051902/5ff1992fb684a92818472f7f/html5/thumbnails/17.jpg)
The Probabilistic Method