Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles
description
Transcript of Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles
![Page 1: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/1.jpg)
Lower Bounds on the Communication of Distributed Graph Algorithms:
Progress and Obstacles
Rotem OshmanADGA 2013
![Page 2: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/2.jpg)
Overview: Network ModelsCONGESTED CLIQUE
ASYNC MESSAGE-PASSING
LOCAL
CONGEST / general network
X
![Page 3: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/3.jpg)
Talk Overview
I. Lower bound techniquesa. CONGEST general networks: reductions from 2-
party communication complexityb. Asynchronous message passing: reductions from
multi-party communication complexityII. Obstacles on proving lower bounds for the
congested clique
![Page 4: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/4.jpg)
Communication Complexity
𝑋 𝑌
= ?
![Page 5: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/5.jpg)
Example: DISJOINTNESS
𝑋⊆ {1 ,…,𝑛} 𝑌⊆ {1 ,…,𝑛}
𝑋∩𝑌=∅ ?
bitsneeded
[Kalyanasundaram and Schnitger, Razborov ’92]
DISJ :
![Page 6: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/6.jpg)
Applying 2-Party Communication Complexity Lower Bounds
Textbook reduction:Given algorithm for solving task …
Solution for answer for DISJOINTNESS
bits
𝑌𝑋
Based on
Based on
Simulate
![Page 7: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/7.jpg)
Example: Spanning Trees
• Setting: directed, strongly-connected network• Communication by local broadcast with
bandwidth • UIDs • Diameter 2• Question: how many rounds to find a rooted
spanning tree?
![Page 8: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/8.jpg)
New Problem: PARTITION
• Inputs: , with the promise that
• Goal: Alice outputs ,Bob outputs such that partition .
𝑋 𝑌
![Page 9: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/9.jpg)
The PARTITION Problem
• Trivial algorithm:– Alice sends her input to Bob– Alice outputs all tasks in her input– Bob outputs all remaining tasks
• Communication complexity: bits• Lower bound?
![Page 10: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/10.jpg)
Reduction from DISJ to PARTITION
• Given input for DISJ :– Notice: iff – To test whether :• Try to solve PARTITION on • Ensure • Check if is a partition of : Alice sends Bob hash(), Bob
compares it to hash()
![Page 11: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/11.jpg)
4 65
From PARTITION to Spanning Tree
a b
1 2 3
𝑋={1,2,3 } 𝑌={2,4,5,6 }
Given a spanning tree algorithm …
![Page 12: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/12.jpg)
4 65
From PARTITION to Spanning Tree
a b
1 2 3
𝑋={1,2,3 } 𝑌={2,4,5,6 }
Simulating one round of :
Node a’s message
Node b’s message
![Page 13: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/13.jpg)
4 65
From PARTITION to Spanning Tree
a b
1 2 3
𝑋={1,2,3 } 𝑌={2,4,5,6 }
When outputs a spanning tree:
![Page 14: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/14.jpg)
From PARTITION to Spanning Tree
• If runs for rounds, we use bits
• One detail: randomness– Solution: Alice and Bob use public randomness
![Page 15: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/15.jpg)
When Two Players Just Aren’t Enough
• No bottlenecks in the network
![Page 16: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/16.jpg)
When Two Players Just Aren’t Enough
• Too much information revealed
![Page 17: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/17.jpg)
Multi-Player Communication Complexity
• Communication by shared blackboard• Number-on-forehead• Number-in-hand
??
![Page 18: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/18.jpg)
The Message-Passing Model
• players• Private channels• Private -bit inputs • Private randomness
• Goal: compute • Cost: total communication
![Page 19: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/19.jpg)
The Coordinator Model
• players, one coordinator• The coordinator has no input
![Page 20: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/20.jpg)
Message-Passing vs. Coordinator
≈
![Page 21: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/21.jpg)
Prior Work on Message-Passing
• For players with -bit inputs…• Phillips, Verbin, Zhang ’12:– for bitwise problems (AND/OR, MAJ, …)
• Woodruff, Zhang ‘12, ‘13:– for threshold and graph problems
• Braverman, Ellen, O., Pitassi, Vaikuntanathan ‘13: for
![Page 22: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/22.jpg)
Set Disjointness
Disj𝑛 ,𝑘 =¿ 𝑖=1¿𝑛¿ 𝑗=1¿𝑘 𝑋 𝑖𝑗¿
?
𝑋 1𝑋 2
𝑋 3
𝑋 4𝑋 5
![Page 23: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/23.jpg)
Notation
• : randomized protocol– Also, the protocol’s transcript– : player ’s view of the transcript
• worst-case communication of
in the worst case
![Page 24: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/24.jpg)
Entropy and Mutual Information
• Entropy:
• A lossless encoding of requires bits• Conditional entropy:
![Page 25: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/25.jpg)
Entropy and Mutual Information
• Mutual information:
• Conditional mutual information:
![Page 26: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/26.jpg)
Information Cost for Two Players[Chakrabarti, Shi, Wirth, Yao ’01], [Bar-Yossef, Jayram, Kumar, Sivakumar ‘04], [Braverman, Rao ‘10], …
Fix a distribution , • External information cost:
• Internal information cost:
Extension to the coordinator model:
![Page 27: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/27.jpg)
Why is Info Complexity Nice?
• Formalizes a natural notion– Analogous to causality/knowledge
• Admits direct sum theorem:
“The cost of solving independent copies of problem is times the cost of
solving ”
![Page 28: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/28.jpg)
Example
![Page 29: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/29.jpg)
Example (Work in Progress)
• Triangle detection in general congested graphs• “Is there a triangle” =
”is a triangle”
![Page 30: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/30.jpg)
Application of DISJ Lower Bound
• Open problem from Woodruff & Zhang ‘13:– Hardness of computing the diameter of a graph
• We can show: bits to distinguish diameter 3 from diameter
• Reduction from DISJ : given ,– Notice: disjoint iff
![Page 31: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/31.jpg)
Application of DISJ Lower Bound
• Diameter • Diameter
𝑝1 𝑝3
𝑝2
𝑝4
1
32 4
5
6
𝑋 3
![Page 32: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/32.jpg)
Part II: The Power of the Congested Clique
CONGESTED CLIQUE
![Page 33: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/33.jpg)
Conversion from Boolean Circuit
• Suppose we have a Boolean circuit – Any type of gate, inputs– Fan-in – Depth = , #gates and wires =
• Step 1: reduce the fan-out to – Convert large fan-out gates to “copying tree”– Blowup: depth, size
• Step 2: convert to a layered circuit
![Page 34: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/34.jpg)
Conversion from Boolean Circuit
• Now we have a layered circuit of depth and size = – With fan-in and fan-out
• Design a CONGEST protocol:– Fix partition of inputs of size each– Assign each gate to a random CONGEST node– Simulate the circuit layer-by-layer
![Page 35: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/35.jpg)
Simulating a Layer
• If node “owns” gate on layer , it sends ’s output to the nodes that need it on layer
• Size of layer size of layer • What is the load on edge ?– For each wire from layer to layer ,
– At most wires in total– By Chernoff, w.h.p. the load is
![Page 36: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/36.jpg)
Conversion from Boolean Circuit
• A union-bound finishes the proof• Corollary: explicit lower bounds in the
congested clique imply explicit lower bounds on Boolean circuits with polylogarithmic depth and nearly-linear size.
• Even worse:– Reasons to believe even bound hard
![Page 37: Lower Bounds on the Communication of Distributed Graph Algorithms: Progress and Obstacles](https://reader036.fdocuments.in/reader036/viewer/2022070422/568164d9550346895dd7211e/html5/thumbnails/37.jpg)
ConclusionCONGESTED CLIQUE
ASYNC MESSAGE-PASSING
LOCAL
CONGEST / general network
X