Turnstile Streaming Algorithms Might as Well Be Linear Sketches
description
Transcript of Turnstile Streaming Algorithms Might as Well Be Linear Sketches
![Page 1: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/1.jpg)
Turnstile Streaming Algorithms Might as Well Be Linear Sketches
Yi Li Huy L. Nguyen David Woodruff Max-Planck Princeton IBM Almaden
![Page 2: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/2.jpg)
Turnstile Streaming Model
• Underlying n-dimensional vector x initialized to 0n
• Long stream of updates x à x + ei or x à x - ei for standard unit vector ei
• At end of the stream, x 2 {-m, -m+1, …, m-1, m}n for some bound m · poly(n)
• Output an approximation to f(x) whp
• Goal: use as little space (in bits) as possible
![Page 3: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/3.jpg)
Example: Euclidean Norm
• Want to output Z with (1-Ɛ) |x|2 · Z · (1+Ɛ) |x|2
• Let r = 1/Ɛ2
• Choose an r x n matrix A of i.i.d. sign random variables (+1 w.pr. ½, -1 w.pr. ½)
• Maintain Ax in the stream
• Output |Ax|2
• Proof: Johnson-Lindenstrauss Lemma
![Page 4: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/4.jpg)
Generic Features
Algorithm for 2-norm has the following form:
1. Choose a random matrix A independent of x
2. Maintain Ax in the stream
3. Output a function of Ax
Question (?!): does the optimal algorithm for approximating any function in the turnstile model have this form?
All known algorithms have this form
Some functions f(x) may be weird:
What is xx1?
Some functions f(x) may be weird:
What is xx1?
![Page 5: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/5.jpg)
Our Result
• Yes, up to a factor of log n
• Theorem: for computing a relation f for x in {-m, -m+1, …, m}n in the turnstile model, there is a correct (whp) algorithm which:
1. samples an integer matrix A uniformly from O(n log m) hardwired matrices with poly(n) bounded integer entries, independent of x,
2. outputs a function of Ax
Logarithm of the number of states of Ax, for x in {-m, -m+1, …, m}n, plus amount of randomness, is optimal up to a log n factor
![Page 6: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/6.jpg)
Consequences
a 2 {0,1}n
Create stream s(a)
b 2 {0,1}n
Create stream s(b)
Lower Bound Technique1. Run Alg on s(a), transmit state of Alg(s(a)) to Bob
2. Bob computes Alg(s(a), s(b))
3. If Bob solves g(a,b), space complexity of Alg at least the 1-way communication complexity of g
![Page 7: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/7.jpg)
Consequences
a 2 {0,1}n
Create stream s(a)b 2 {0,1}n
Create stream s(b)
Our main theorem implies:If players can solve g(a,b), then space of Alg at least the simultaneous communication complexity of g
Weaker public-coin model in which Alice and Bob simultaneously send a message to a referee
![Page 8: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/8.jpg)
Non-Uniformity Restriction
• Careful wording: “samples an integer matrix A uniformly from O(n log m) hardwired matrices, with poly(n) bounded entries, independent of x”
• Algorithm is non-uniform– Output of each state for each A also hardwired
• Alternatively, allow algorithm to use more space to process a stream update, provided it only retains Ax and its randomness– Regenerate A during each stream update
![Page 9: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/9.jpg)
Comment on the Model
• For each random seed, algorithm is a deterministic automaton with a finite number of states
• Main theorem only requires correctness for
x 2 {-m, -m+1, …, m}n
It counts the number of states as x varies in this range
• While processing the stream, may have |x|1 > m
• The algorithm can’t abort if this happens. It must still be correct at the end of the stream for x in {-m, -m+1, …, m}n
![Page 10: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/10.jpg)
Related Work
• Ganguly
– Specific to heavy hitters problem
– Holds only for deterministic algorithms
![Page 11: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/11.jpg)
Talk Outline
• Proof Overview
1. Reduction to path-independent automata
2. From path-independent automata to linear sketches
• Applications and Open Questions
![Page 12: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/12.jpg)
Start
+e1
-e1, +e2
…-en
-e1
+e1
+en
…
+e5
…
…
… …
Stream Automaton for Fixed Randomness
Streaming algorithm only depends on x, not how it got there
Streaming algorithm only depends on x, not how it got there
0n in two different states
0n in two different states
![Page 13: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/13.jpg)
Path-Independent Automaton
• Each x 2 Zn in a unique state
• Undirected connected graph
• Goal: for each randomness, can we modify the automaton to make it path-independent?
• Rule out algorithms that e.g., an algorithm that stores the last 5 stream updates
![Page 14: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/14.jpg)
Strategy
For stream σ, freq(σ) 2 Zn is “net update” to each coordinate
Idea: 1. if in a state s, and update by a stream σ, with freq(σ) = 0, answers ought to be similar
2. collapse all states s, s’ for which s+σ = s’ and freq(σ) = 0 for some stream σ
Issue: how to formally define states, transition and output function of new automaton?
Intuitively makes things
path-independent
Intuitively makes things
path-independent
![Page 15: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/15.jpg)
Zero-Frequency Graph• Directed multi-graph G = (V,E)
• V = states of old automaton Aold (for fixed randomness)
• (s,t) 2 E for each stream σ of finite length with s+σ=t and freq(σ) = 0
• Terminal equivalence class: strongly connected component with no outgoing edge– Walk in G eventually reaches a terminal equivalence class (Walk in G is a long sequence of zero-streams)
– States of new automaton Anew = terminal equivalence classes
![Page 16: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/16.jpg)
New Transition Function
• Suppose in terminal equivalence class C
• Given an update ei
• Let v 2 C be an arbitrary node
• Compute v+ei using transition function of Aold
• Walk from v+ei until reach terminal equivalence class C’
• C’ is unique• Does not depend on choice of v• Only one terminal equivalence class reachable in any walk
![Page 17: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/17.jpg)
Terminal equivalence
class
Terminal equivalence
class
u v
+ei +ei
Terminal equivalence
class
Terminal equivalence
class
freq(σ) = 0 freq(σ’) = 0
Terminal equivalence
class
Terminal equivalence
class
w
-ei
x
y
Contradiction: zero frequencypath w-x-v-y
![Page 18: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/18.jpg)
Output Function of Anew
• In each terminal equivalence class C, sample node u from stationary distribution from random walk in C (add self-loops)– Output of Anew on C = Output of Aold on u
• If v is starting vertex of Aold,
– take a random walk in G from v
– let starting vertex of Anew be terminal equivalence class C reached
• Why is it correct?
![Page 19: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/19.jpg)
Correctness
• Let ¦ be an arbitrary distribution on streams ¾
• Choose fixed randomness so Aold correct on ¦’: – Long sequence of zero frequency streams, – Followed by ¾ sampled from ¦,– Followed by long sequence of zero frequency streams
• Output of Anew on ¦ statistically close to output of Aold on ¦’
• => for every ¦ there is an Anew correct on ¦
• Can show Anew is path-independent (lying a little..)
![Page 20: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/20.jpg)
Path Independence to Linear Sketches
• M = {x 2 Zn such that x in same state as 0n}
• States of automaton are cosets of Zn/M
• Use lattice tools…
![Page 21: Turnstile Streaming Algorithms Might as Well Be Linear Sketches](https://reader034.fdocuments.in/reader034/viewer/2022051622/56815144550346895dbf64b9/html5/thumbnails/21.jpg)
Applications and Open Questions
• Simpler proof of existing lower bounds– No communication complexity
• Many dimension lower bounds known for sketching norms over the reals– Matrix norms, etc.– Do these give turnstile streaming lower
bounds with finite precision?