Yongqiang Liu Center for Forest Disturbance Science USDA Forest Service, Athens, GA
On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with...
-
Upload
liliana-abigayle-cameron -
Category
Documents
-
view
216 -
download
0
Transcript of On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with...
![Page 1: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/1.jpg)
On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks
Meng ZHANG
with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG
Globecom 2006
![Page 2: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/2.jpg)
Outline
Background Related Work Problem Statement and Formulation Global Optimal Solution Distributed Algorithm Performance Evaluation Conclusion & Future Work
![Page 3: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/3.jpg)
Background
The Internet has witnessed a rapid growth in deployment of data-driven (swarming based) overlay/peer-to-peer network based IPTV systems during recent years.
These products are based on data-driven protocol
Facts of concurrent online users GridMedia: over 230,000, rate 310kbps (achieved by
one server) (developed by our lab) PPLive: 500,000, rate 300-500kbps QQLive: 1,460,000, rate 300-500kbps (not one server)
![Page 4: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/4.jpg)
Background - Data-Driven Protocol Review Aiming to enable large-scale live
broadcasting in the Internet environment Very simple and very similar to that of Bit-
Torrent Two steps in data-driven protocol
The overlay construction The block scheduling
![Page 5: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/5.jpg)
Background - Data-Driven Protocol Review The first step – overlay construction
All the nodes self-organize into a random graph
1 2 3
4 5
root
11 2 4 1 2 3
2 3
4 5 2 31 2
I have block 1,2,4 I have block 1,2,3
I have block 1,2 I have block 2,3
Request block 4 Request block 3
Request block 1 Request block 2
Send block 4 Send block 3
Send block 1 Send block 2
The second step – block scheduling The streaming is divided into blocks Each node has a sliding window containing all the
blocks it is interested in currently
![Page 6: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/6.jpg)
Related Work
To improve data-driven protocol, most recent efforts focus on optimizing overlay construction (i.e. the first step ): Vishnumurthy & Francis (INFOCOM2006):
random graph building under heterogeneous overlay
Liang & Nahrstedt (INFOCOM2006): propose RandPeer, a peer-to-peer QoS-sensitive membership management protocol
![Page 7: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/7.jpg)
Related Work
An problem not well addressed is how to optimize the second step, that is, how to do optimal block scheduling and
maximize the throughput of data-driven protocol under a constructed overlay
Most existent methods are straight forward and ad hoc Chainsaw: pure random way DONet: greedy local rarest-first PALS: round-robin method
![Page 8: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/8.jpg)
Problem Statement and Formulation
2
13 2
23 4
4
4 6 5
343
How to do optimal scheduling to maximize the throughput of the whole overlay?
The real situation is more complicated because different blocks may have different importance and the bottlenecks are not only at the last mile.
Our basic approach: Define priority to different blocks due to their importance Maximize the sum of priorities of all requested blocks
16 2
264
4
3
43 5
32
5
3
2
4
5
2
2
11
65
32
4
1
00
2
21 2 3 4
23 6 4 5Throughput is 4Optimal scheduling,
throughput gain is 25%
Some requestscongestion at node 1
Local Rarest First (LRF) strategy
![Page 9: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/9.jpg)
Problem Statement and Formulation - Priority Definition We use two factors to represent the
significance of a block: rarity factor emergency factor
We define the priority of block j∈Ai for node i∈R as follow: Pj
i = βPR(Σk Nbr(∈ i)hkj)+(1-β)PE(Ci+WT-dji),
Where 0≤β≤1, functions PR(*) (rarity factor) and PE(*) (emergency factor) are both monotonously non-increasing ones
![Page 10: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/10.jpg)
Problem Statement and Formulation - Formulation Decision variable
Global block scheduling problem:
s.t.
1, node should request
packet from nbr
0, otherwise
ikj
i
x j k
1max
i i
i ij kj kj
i j D k NBR
P h xN
a) , b) ,
c) , d) 1,
e) 0,1 .
i ikj i kj k
j A i k Nbr i i j A i
i ikj ki kj
j A i k Nbr i
ikj
x I x O
x E x
x
Notation Definition
N N+1 is the number of overlay nodes, where node 0 is the source node
Ii, the inbound bandwidth of node i
Oi, the outbound bandwidth of node i
Eik, the end-to-end available bandwidth between node i and node k
hkj {0,1}∈ Blocks availability: “akj=1” denotes
node k holds block j; otherwise, “akj=0”
NBRi set of neighbors of node i
τ period of requesting new blocks
WT the exchanging windows size
Ci the current play out time of node i
dji play out time of block j at node i
Di set of all absent blocks in the current exchanging window of node i
![Page 11: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/11.jpg)
Global Optimal Solution Convert the global block scheduling formulation
into an equivalent Min-Cost Flow Problem0
2 3
1
2 3 4 516 7 8 9 3 4 6 9
1 2 5 6 3 6 7 8
capacity, unit costnodeblock
v0s
v1s
v2s
v3s
vS vT
v01n
v21n
v31n
v02n
v32n
v03n
v23n
v12n
v13n
τ E01,0
τ E02,0
τ E23,0
τ E03,0
τ E31,0
τ E32,0
τ E21,0
τ E12,0
τ E13,0
end-to-endconstraints
τ O3,0
τ O1,0
τ O2,0
τ O0,0
outboundconstraints
v11b
v15b
v17b
v18b
v23b
v24b
v27b
v28b
v32b
v34b
v35b
v31b
1,0
1,0
1,01,0
1,0
1,0
1,0
1,0
1,0
1,0
1,0
1,01,0
1,0
1,0
1,0
1,0
1,0
1,0
1,0
1,01,0
1,0
1,0
1,0
block availability
v1r
v2r
v3r
1, -P11/N
1, -P51/N
1, -P71/N
1, -P81/N
1, -P32/N
1, -P42/N
1, -P72/N
1, -P82/N
1, -P13/N
1, -P23/N
1, -P43/N
1, -P53/N
duplicateconstraints
τ I1,0
τ I3,0
τ I2,0
inboundconstraints
∞ ,-M
![Page 12: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/12.jpg)
Global Optimal Algorithm
Proposition: The optimal goal of global block scheduling problem
has the same absolute value as the minimum flow amount of its corresponding min-cost network flow problem. The flow amount on arc (vki
n, vijb) {0, 1} is ∈
just the value of xkji, which is the solution to the optimal
block scheduling.
Algorithm complexity: O(nm(loglogU)log(nC)), where n and m are the number
of vertices and arcs while U and C is the largest magnitude of arc capacity and cost
![Page 13: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/13.jpg)
Distributed Algorithm
We first use a simple way to estimate the bandwidth that is available from each neighbor with historical information.
qki (m): the total number of blocks arrived at node i from neighbor k in the mth period.
Wki(m+1): the estimated bandwidth from node k to
node i
( 1) ( )
1
(3)m
m lki ki
l m M
W q M
![Page 14: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/14.jpg)
Distributed Algorithm With the estimated available
bandwidth, a local block scheduling is performed on each node
It can be also transformed into an equivalent min-cost network flow problem for local optimal request
max (2)i i
j kj kjj A i k Nbr i
P h x
s.t.
1, ,
, 0,1
i ikj kj i
k Nbr i j A i k Nbr i
i ikj ki kj
j A i
x x I
x W x
vTτ I1,0
vS τ W21,0
τ W01,0
τ W31,0
v01n
v21n
v14b
1,0
v16b
v17b
v1r
1, -P41
1, -P61
1,0
v18b
v31n 1,0
1, -P81
1,0
1,0
1,0
1,0
1,0
∞ ,-M
1, -P71
capacity, unit cost
node vertexblock vertex
![Page 15: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/15.jpg)
Distributed Algorithm
Heuristic distributed algorithm: Node i estimates the bandwidth Wki
(m+1) that its neighbor k can allocate it in the (m+1)th period with the traffic received from that neighbor in the previous M periods, as shown in equation (3);
Based on Wki(m+1), node i performs the local block
scheduling (2) using min-cost network flow model. The results xkj
i {0,1} represent whether node ∈ i should request block j from neighbor k;
Send requests to every neighbor.
![Page 16: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/16.jpg)
Performance Evaluation- Compared Scheduling Methods Random Strategy: each node will assign each desired
block randomly to a neighbor which holds that block. Chainsaw uses this simple strategy.
Local Rarest First (LRF) Strategy: A block that has the minimum owners among the neighbors will be requested first. DONet adopts this strategy.
Round Robin (RR) Strategy: All the desired blocks will be assigned to one neighbor in a prescribed order in a round-robin way. If there is multiple available senders, it is assigned to a sender that has the maximum surplus available bandwidth.
![Page 17: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/17.jpg)
Simulation Configuration For a fair comparison, all the experiments use the same
simple algorithm for overlay construction Delivery ratio: to represent the number of blocks that
arrive at each node before playback deadline over the total number of blocks encoded.
DSL nodes: Download bandwidth: 40% 512K, 30% 1M, 30% 2M Upload bandwidth: half of download bandwidth
500 nodes Each node has 15 neighbors Request period: 2 second
![Page 18: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/18.jpg)
Simulation Results
All are DSL nodes with exchanging window of 10 sec and bottlenecks only at the last mile. Group size is 500
250 300 350 400 450 5000.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Streaming Rate (Kbps)
Ave
rage
Del
iver
y R
atio
Global optimal algorithmProposed distributed algorithmDONet methodOne-layer PALS methodChainsaw method
![Page 19: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/19.jpg)
Simulation Results
All are DSL users with exchanging window of 10 sec and end-to-end available bandwidth 10~150Kbps. Group size is 500
250 300 350 400 450 5000.4
0.5
0.6
0.7
0.8
0.9
1
Streaming Rate (Kbps)
Ave
rage
Del
iver
y R
atio
Global optimal algorithmProposed distributed algorithmDONet methodOne-layer PALS methodChainsaw method
![Page 20: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/20.jpg)
Conclusion & Future Work The contributions of this paper are twofold.
First, to the best of our knowledge, we are the first to theoretically address the streaming scheduling problem in data-driven (swarming based) streaming protocol.
Second, we give the optimal scheduling algorithm under different bandwidth constraints, as well as a distributed asynchronous algorithm which can be practically applied in real system and outperforms existent methods by about 10%~80%
Future work How to do optimization over a horizon of several periods, taking
into account the inter-dependence between the periods.
How to do optimal scheduling with scalable video coding (such as layered video coding) or multiple description coding
![Page 21: On the Optimal Scheduling for Media Streaming in Data-driven Overlay Networks Meng ZHANG with Yongqiang XIONG, Qian ZHANG, Shiqiang YANG Globecom 2006.](https://reader036.fdocuments.in/reader036/viewer/2022062722/56649f335503460f94c50a6d/html5/thumbnails/21.jpg)
Thanks
Q&A