Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers,...
-
date post
22-Dec-2015 -
Category
Documents
-
view
215 -
download
1
Transcript of Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers,...
![Page 1: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/1.jpg)
Accessing Multiple Mirror Sites in Parallel:Using Tornado Codes to Speed Up Downloads
John Byers, Boston University
Michael Luby, Digital Fountain, Inc.
Michael Mitzenmacher, Harvard
INFOCOM 99
![Page 2: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/2.jpg)
The Problem
• Multicast: to save bandwidth.
Sender
Receivers
Senders
Receiver
• Parallel download: to improve speed.
![Page 3: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/3.jpg)
Many-to-many Distribution
• Heterogeneous environment of senders and receivers.• Senders broadcast. • Receivers gather data as fast as possible from as
many sources as possible.
Senders
Receivers
![Page 4: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/4.jpg)
Results
• A simple, robust, scalable solution for parallel downloads and many-to-many distribution using Forward Error Correction.
• Examination of tradeoffs– Speed vs. Goodput
![Page 5: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/5.jpg)
Applications• Internet
– Connect to many mirror sites simultaneously.
• Multiple access media– ISDN and modem simultaneously.
• Mobile clients– Multiple access points.– Listen to multiple frequencies.
• Satellite networks– Ground user receives from many satellites.
![Page 6: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/6.jpg)
Assumptions• Possible to create bottleneck-disjoint paths.
– Otherwise wasted bandwidth, more congestion.– Receiver should not be the bottleneck.– For people with big pipes.
Senders
Receiver
1 2
1 2
Senders
Receiver
1 2
2
1
dropped
BottleneckDisjoint
SharedBottleneck
![Page 7: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/7.jpg)
Solutions without Coding• A protocol without codes:
– Initially receiver tells each of s senders to send disjoint 1/s parts of the file.
– If one sender finishes early, re-negotiate packets to be sent.– Continue until all packets arrive.
• Problems– Significant feedback.– Unsuitable for many-to-many.– Complexity.– No protection against losses.– Wait for last packet.
![Page 8: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/8.jpg)
Forward Error Correction (FEC)
• Message of n packets encoded as cn packets.– A receiver decodes once enough packets arrive.
• FEC codes improve multicast scalability– Encoding packets can correct different losses
for different receivers.
– Reduces feedback, to even feedback-free solutions.
![Page 9: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/9.jpg)
Tornado Codes
• Tornado Codes are FEC codes that are– Very fast (linear time).– Better for large files.– Information-theoretically slightly suboptimal.
Requires 1.055n packets to decode n packet message.
![Page 10: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/10.jpg)
Ideal Solution: Digital Fountain
• Reconstruct file from any n packets, from any source.
• Feedback free: no need for receivers to acknowledge specific packets.
• Fountain metaphor: drink when the cup is full.• Approximate digital fountain solution using
Tornado codes.– Reception inefficiency due to overhead of Tornado
codes, duplicate packets.
![Page 11: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/11.jpg)
Feedback Free Solution• Senders encode message the same way.
• Senders cycle through permutation of encoding
• When receiver obtain any 211 distinct packets, it can decode to obtain the message.
1 - 200
1 - 600
Original Message
Encoded Message
17 485 238 12 311 411 512...
216 156 7 128 415 238 333...
397 188 25 315 275 499 12...
![Page 12: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/12.jpg)
Performance Metrics
• Speedup:
• Stretch factor (c): message of n packets encoded as cn packets.
• Reception inefficiency (z): zn packets arrive before decoding.– Code overhead– Duplicates
Download time now
Download time using single fastest sender
![Page 13: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/13.jpg)
Tradeoffs• Increasing stretch c:
– Lessens duplicates: senders have more packets to send, so random collisions less likely
– Increases encoding/decoding time, memory requirements, and complexity. (Grow linearly in c.)
17 485 238 12 311 411 512...
216 156 7 128 415 238 333...
397 188 25 315 275 499 12...
![Page 14: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/14.jpg)
Speedup vs. Stretch
0
0.5
1
1.5
2
2.5
3
3.5
4
2 3 4 5 6
Stretch Factor
Sp
eed
up
4 senders
3 senders
2 senders
![Page 15: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/15.jpg)
Reception Inefficiency vs. Stretch
1
1.05
1.1
1.15
1.2
1.25
1.3
1.35
1.4
2 3 4 5 6
Stretch Factor
Rec
epti
on
In
effi
cien
cy
4 senders
3 senders
2 senders
![Page 16: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/16.jpg)
Speedup vs. Stretch
1
1.5
2
2.5
3
2 3 4 5 6
Stretch Factor
Sp
eed
up
2 senders, 1:1
2 senders, 2:1
2 senders, 4:1
![Page 17: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/17.jpg)
Reception Inefficiency vs. Stretch
1
1.05
1.1
1.15
1.2
1.25
1.3
2 3 4 5 6
Stretch Factor
Rec
epti
on
In
effi
cie
ncy
2 senders, 1:1
2 senders, 2:1
2 senders, 4:1
![Page 18: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/18.jpg)
Feedback Free Solution• Pros
– Simple– Loss protection– Good download speedups– No feedback, coordination– Solves many-to-many
• Cons– Extra bandwidth for Tornado codes (5.5%)– Extra bandwidth from packet duplicates
• depends on c, number of senders, variation in rates
• additional 5-25+ %
153421
341253
421354
![Page 19: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/19.jpg)
Rare Feedback• Senders use same permutation of encoding.
• Receivers tell each of s senders to send 1/s of the encoding.– If c > s, each sender has 1 file worth of data.
• In rare cases, re-negotiate, or have senders send the rest in random order.
193672
193672
193672
![Page 20: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/20.jpg)
Rare Feedback• Pros
– Simple– Loss protection– Rare feedback, minimal coordination– Extra bandwidth for Tornado codes only
• Cons– Does not solve multi-multi– Extra bandwidth for Tornado Codes
193672
193672
193672
![Page 21: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d805503460f94a65082/html5/thumbnails/21.jpg)
Conclusions
• Fast parallel download and many-to-many distributions are practical.– Trade goodput for speed.
• FEC improves protocols.– Simpler.– Less feedback.– Loss protection.
• Deployment issues (fairness, bottleneck disjoint paths) still open.