Your name. Bullet point Sub Bullet Example Bullet Point Slide.
Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh
description
Transcript of Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh
Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh
by
Dejan Kostic, Adolfo Rodriguez,Jeannie Albrecht and Amin Vahdat
presented by
Jon Turner
4-2 - Jon Turner - 04/21/23
Introduction Problem: large-scale data dissemination.
» focus on high bandwidth streaming media Current solutions
» IP multicast – not complete solution and not deployed» overlay multicast – too dependent on quality of multicast trees
Proposed approach.» partial distribution using multicast tree
– all data distributed, but each node gets only fraction from parent» peer-to-peer distribution of remainder
– periodic distribution of information about data present at random subsets of entire group
– nodes request missing data from peers» system combines number of elements developed earlier
– erasure encoding (redundant tornado codes)– random subset distribution– informed content-delivery– TCP friendly rate control
4-3 - Jon Turner - 04/21/23
Overview of Bullet Operation Distribute data over tree
»limited replication»uses bandwidth feedback
Nodes retrieve missing data from peers.
Periodic distribution of content availability info.»random subset of nodes»summary of their content
Limited set of peering relationships.»each node receives from
limited set of senders»each node limits
receivers»sets evolve over time to
improve utility of peers Data sent using TCP
friendly rate control.
4-4 - Jon Turner - 04/21/23
Data Distribution System operates in series of epochs (typ. 5 seconds).
»at start of epoch, nodes learn # of descendants children have»child i is assigned a sending factor sfi equal to its share of
descendants (child with 20% of descendants has sfi=.2) Nodes forward data received from parent to children.
»packets are “assigned” to children according to their sf values»additional copies forwarded to children with spare bandwidth
– use limiting factors, lfi to determine which children have spare bw– lf values are adjusted dynamically in (0,1] based on bw
Algorithm sketch – executed for each input packet pFind child t that has been assigned fewer than its share of packets.» attempt to send p to t (transport
protocol blocks send if sending rate too high)
» if attempt succeeds, assign p to t
For each child c» attempt to send to c if no successful
attempt yet or if c has spare bw» if attempt succeeds and this is first
successful attempt, assign p to c» if attempt succeeds, but not first
success, increase lfc
» if attempt fails, decrease lfc
4-5 - Jon Turner - 04/21/23
Finding Prospective Data Sources At start of each epoch, each node receives information
about data present at a random subset of peers.» summary ticket describing data present at each peer from
current “working set”» by comparing peer summary tickets to its own, node determines
similarity of peers’ data to its data» select new peers with dissimilar data sets» limit on number of concurrent senders
– discard senders that have been supplying too few new packets– creates space in sender list for new sender
Computing summary ticket» if node has packets i1,i2,...,in» let tj=min{fj(i1), fj(i2),...} for 1≤j≤k where fj(ik)=(aj ik+bj) mod U
» summary ticket is (t1, t2,....,tk)
» each tj depends on the entire sequence» similarity of two tickets is fraction of values in common
4-6 - Jon Turner - 04/21/23
Recovering Data From Peers Node periodically supplies its senders with a
representation of the set of packets it currently has.» limited to current “working set” (range of sequence numbers)» set is represented by a Bloom filter» each sender is also assigned a fraction of the working set
– packets with sequence numbers equal to i mod s where s is number of senders
Senders transmit missing packets using available bw.» packets checked against Bloom filter
– don’t send if packet’s sequence number is in Bloom filter» because senders selected for dissimilarity, most packets
should pass check Example of numerical parameters.
» 5 second epoch, 30 second working set, 500 Kb/s=50 p/s,10 senders
» so asking each sender for at most 150 packets– simpler and possibly more efficient to use bit vector
4-7 - Jon Turner - 04/21/23
Distributing Random Subsets Objective – give each node a random sample of other
nodes in the tree, excluding its descendants. Two phases
»collect phase – propagate random subsets up the tree»distribute phase – propagate random subsets back down tree
– mix subset received from parent with child subsets
u
S1,n1 S2,n2 Sk,nk
S=subset of {u}S1...Sk
n=1+n1+...+nk
u
D,N
D1=subset of {u}DS2...Sk
N1=1+N+n2+...+nk
4-8 - Jon Turner - 04/21/23
Other Features Formation of overlay tree.
»not a focus of this work»paper cites variety of previous tree construction algorithms»argues that use of mesh makes tree quality less important»most results use random tree
Data encoding»not a focus of this work»suggests use of erasure codes or multiple-description codes»reported results neglect encoding
Transport protocol»unreliable version of TFRC»adjusts sending rate to match fair-share bandwidth based on
detected loss probability»feedback from transport sender used by Bullet to adjust rates
at which it attempts to send to children
4-9 - Jon Turner - 04/21/23
Performance Evaluation Most results emulated in ModelNet (Duke network
emulation system, similar to Emulab)»uses 50 machines (2 GHz P4s running Linux) to emulate 1,000 node
overlay 20,000 node network topology generated using INET
»link delays determined by geographic separation»random subset of network “client” nodes selected for overlay»random node in overlay selected as root
Link bandwidths»four classes of links»three scenarios
client-stub stub-stub transit-stub transit-transit
low bandwidth
.3-.6 Mb/s .5-1 Mb/s 1-2 Mb/s 3-4 Mb/s
medium .8-2.8 Mb/s 1-4 Mb/s 1-4 Mb/s 5-10 Mb/s
high 1.6-5.6 Mb/s 2-8 Mb/s 2-8 Mb/s 10-20 Mb/s
4-10 - Jon Turner - 04/21/23
Tree Quality for Streaming Data
random tree
bottleneck bandwidth tree
bottleneck bandwidth tree constructed
heuristically to have no small capacity links
(off-line construction)
point is to show that bottleneck tree is much better than random tree
used by Bullet.
4-11 - Jon Turner - 04/21/23
Baseline Bullet Performance
from parent
raw total
useful total
±std. deviation
not too much excess traffic
most data from mesh
average better than bottlneck
tree for streaming case
3 point suggests a few percent get less than half average
4-12 - Jon Turner - 04/21/23
Cum. Dist. of Received Bandwidth
median about 525 Kb/sabout 100 nodes get
<80% of median
about 50 nodes get <70% of median
4-13 - Jon Turner - 04/21/23
Comparing Bullet to Streaming
high bandwidth
medium
low
BulletBottleneck
when plenty of bandwidth both
do wellBullet does better when bandwidth
limited
much better when bandwidth
scarce
4-14 - Jon Turner - 04/21/23
Effect of Oblivious Data Distribution
useful total
raw total
from parent
average throughput
drops by 25%
tree nodes attempt to forward all received
packets to all children
4-15 - Jon Turner - 04/21/23
Bullet vs “Epidemic Routing” Push gossiping
»nodes forward non-duplicate packets to randomly chosen set of peers in their local view
»packets forwarded as they arrive»no tree required
Streaming with anti-entropy»data streamed over multicast tree»gossip with random peers to retrieve missing data»anti-entropy (?) used to locate missing data
– apparently, this means periodically select a random peer and send it list of missing packets, peer supplies missing packets it has
Experiments done on 5000 node topology.»no physical losses»link bandwidths chosen from medium range
4-16 - Jon Turner - 04/21/23
Bullet vs “Epidemic Routing”
push gossipingstreaming w/AEbullet
raw
useful
bulletpush gossipingstreaming w/AE
gossiping has high overhead,bullet and streaming w/AE are
comparable
bullet outperforms epidemic routing
4-17 - Jon Turner - 04/21/23
Performance on Lossy Links
Bullet – lowbottleneck – low
bottleneckhigh, medium
Bullet – high bw, medium
non-transit links lose 0 to .3% of packetstransit links lose 0 to .1%5% of links lose 5 to 10%
Bullet dramatically better than streaming over
bottleneck tree topology
4-18 - Jon Turner - 04/21/23
Performance with Failing Node Child of root with 110 descendants fails at time 250. Assume no underlying tree recovery. Evaluate effect of failure detection and establishment
of new peer relationships
useful totalbandwidth received
from parent
new peers disabled
bandwidth receiveduseful total
from parent
new peers enabled
degraded throughput
using existing peers
adding new peers eliminates
degradation
4-19 - Jon Turner - 04/21/23
Bullet Performance on Planet Lab
Bullet
good tree
worst tree
European nodes all near
tree root
47 nodes with 10 in Europe, including root. Compare to streaming over hand-crafted trees.
select children of root to have poor
bandwidth from root,recurse
4-20 - Jon Turner - 04/21/23
Related Work
Peer-to-peer data dissemination»Snoeren et. al. – 2001»Fast Replica – Cherkasova & Lee – 2003»Kazaa and Bit Torrent
Epidemic data propagation»pbcast – Birman et. al. – 1999 »lbpcast – Eugster et. al. – 2001
Multicast»Scalable Reliable Multicast – Floyd et. al. – 1997»Narada – Chu et. al. – 2000»Overcast – Janotti et. al. – 2000
Content streaming»Split Stream – Castro et. al. – 2003 »CoopNet – Padmanabhan et. al. – 2003
4-21 - Jon Turner - 04/21/23
Conclusions Overlay mesh is superior to streaming over overlay
multicast tree. Bullet demonstrates this and includes some novel
features.»method of distributing data to subtrees to equalize
availability»scalable method of finding peers who can supply missing
data Large-scale performance evaluation
»supports claims for Bullet’s performance advantages»explores performance under variety of conditions
4-22 - Jon Turner - 04/21/23
Discussion Questions What are the contributions?
»what’s novel? what’s borrowed?»does it represent an improvement? how much?»are the authors’ claims justified?»for what applications might system be useful?
Are the authors’ design choices adequately justified?»method for distributing data over tree?»acquiring information about peer data?»use of summary tickets? use of Bloom filters? use of TFRC?
Is performance evaluation satisfactory?»what about other network topologies, link bandwidths?»what about different group sizes? tree characteristics?»why no detailed examination of specific design choices?»why isn’t cross traffic emulated?»what about processing requirements? what’s the average
number of cycles executed per packet received?»is it repeatable by others?