Pseudo-DHT: Distributed Search Pseudo-DHT: Distributed Search Algorithm for P2P Video StreamingAlgorithm for P2P Video Streaming

December 15, 2008December 15, 2008

Jeonghun NohJeonghun NohStanford UniversityStanford University

Sachin DeshpandeSachin DeshpandeSharp Laboratories of AmericaSharp Laboratories of America

22J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

P2P Live Video StreamingP2P Live Video Streaming• Relaying video using uplink bandwidth

Live video: no needs for locating video chunks

Video source

33J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

P2P Time-Shifted StreamingP2P Time-Shifted Streaming• Local storage to store fractions of the video• To locate video at arbitrary point, a query server may be used

3~6m

Seeking video of position 5m

7~11m

0~4m

A scalable distributed content search is desired

• The server can become a bottleneck as peer population increases• No dedicated server may be available

44J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

OutlineOutline• P2TSS system• Pseudo-DHT: Distributed Search• Performance evaluation

– Numerical analysis– Simulation study

55J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Introduction to P2TSSIntroduction to P2TSS• P2TSS (P2P Time-shifted Streaming System)

– Serves both live streaming and time-shifted streaming– Time-shifted stream (TSS) is the same as the original stream

except being delayed in time

• Peers store a fraction of video• Cached video chunks are later served to other peers

Source peer

Peer 1 stores Video [0m, 4m) from 9:00am

Peer 2 watches from 0m at 9:10am

[Deshpande et al., 2008]

66J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Peer3

Caching Live StreamCaching Live Stream

Time

Vid

eo

po

sit

ion

• Peers cache live stream with another video connection

Peer 2

Live streamx1

t1 t2

x2

t3

x3

Peer 1

Cached portion

Playback trajectory

77J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Chord: Distributed LookupChord: Distributed Lookup• Chord: a distributed lookup overlay [Stoica et al., 2001]

– Nodes (peers) are connected as a circle– Keys are mapped to successor node– Fast lookup by a finger table. Lookup latency: O( log(Np) )

• An example key/node space– Nodes and keys sorted by IDs– ID length: 6bits– Can be normalized to [0,1)

N: nodeK: key

88J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Building Lookup OverlayBuilding Lookup Overlay• Node is entered to overlay as peer joins

– Node ID: uniformly drawn between [0,1)

– Node is placed in a distributed manner

• (Key, Value) is entered as peer registers buffer status– Key (K): hashed video chunk ID ( 0 ≤ K < 1)

– Value (V): peer network address

– (K,V) is mapped to the successor node

• Example

0 1

99J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Pseudo-DHT: RegistrationPseudo-DHT: Registration• Register (K, V) may result in “collision”

1st attempt 2nd attempt

Chunk ID

Chunk IDii-1 empty

: Occupied ID

i

(i, Peer 1) (i, Peer 2)

• Repeat Register (K’, V) until there is no collision – K’ = K + (n-1)Δ (n=# attempts, Δ=offset base)

– Unlike original DHT, single key-multiple values is discouraged– Leads to better load balancing and low latency retrieval

1010J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Pseudo-DHT: RetrievalPseudo-DHT: Retrieval• Seeking Video Chunk I

– Retrieve(i) may return a “miss”– K’ = i - (n-1)Δ (n=# attempts, Δ=offset base)– Repeat Retrieve(K’) with different keys until a hit occurs– Best-effort search, different from original DHT

1st attempt2nd 3rd

Chunk ID

ii-1

: Video chunks available

i-2

: Video Chunk Header

Chunks delivered to the consumer peer

missmiss

1111J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

OutlineOutline• P2TSS system• Pseudo-DHT: Distributed Search• Performance evaluation

– Numerical analysis– Simulation study

1212J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Preliminaries for AnalysisPreliminaries for Analysis• Symbols

– Np: Number of peers (or nodes)– L: Video length (in secs)– Q: Video chunk size (in secs) : Ratio between Np and available slots M (=L/Q)

1313J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Registration LatencyRegistration Latency• Independence model for the number of collision, C

• More sophisticated model

where A denotes the number of peer arrival in Q seconds

1414J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Experimental SetupExperimental Setup• Pseudo-DHT implemented in PlanetSim [Garcia et al,

2004]

• Simulation setup– Maximum successive lookup: 4– Video length L: 7200s– Buffer size D: 240s – Chunk size Q: 5, 10, 15, 30s

1515J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Registration LatencyRegistration Latency• When is small, models match simulation results• Registration latency of both forward/backward key change is

identical

Video chunk size (Q): 5s

1616J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Retrieval LatencyRetrieval Latency• Empirical statistics

– Xk: 1 if video chunk Bk is occupied. 0 otherwise

– Conditional hit probability Pr (Xi-j=1| Xi=0), ( j: offset base )– With a larger offset base Δ, the correlation between successive retrievals

becomes weaker

• Modeling retrieval latency N:

1717J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Offset Base and Retrieval LatencyOffset Base and Retrieval Latency• Retrieval latency decreases as an offset base increases

Video chunk size Q: 5s

1818J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Video Chunk Size and Retrieval LatencyVideo Chunk Size and Retrieval Latency

• As chunk size Q increases, retrieval latency decreases

1919J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Overhead of Key StorageOverhead of Key Storage

0 1

Key/Node space

• Balls and bins problem– How many nodes (bins) hold k keys (balls)?

– Bins are created by nodes

– Random keys fall into [0,1) uniformly

– Statistically, larger bins will receive more balls

2020J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Overhead of Key StorageOverhead of Key Storage• One node’s probability of storing k keys

• Observations– Low overhead: keys are spread out on the overlay– 50% of nodes store no keys when N=K

N=300K=300

2121J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Conclusion and Future WorkConclusion and Future Work• Proposed Pseudo-DHT

– Allows peers to register / retrieve video chunk in a scalable way– Slightly different from original DHT due to video continuity– Spreads out (key, value) items over the overlay

• P2TSS and Pseudo-DHT– Application to a P2P system– Thorough evaluation with analysis and simulations

• Future research topics– Changing Q dynamically according to peer population size– Considering heterogeneous peer uplink capacity for registration

2222J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Thank you!

2323J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Backup SlidesBackup Slides

2424J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Preliminaries for AnalysisPreliminaries for Analysis• Video chunk ID space

0

X0

X1

X2 XM-1

M-1

2525J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Sample Key/Node SpaceSample Key/Node Space• Interval between nodes is not constant

Sample key/node space

Nodes (peers)

Keys (DSB info)

2626J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Chord: Distributed LookupChord: Distributed Lookup• Chord: a distributed lookup overlay [Stoica et al., 2001]

– Nodes (peers) and keys are mapped to a common space– Fast lookup by a finger table. Lookup time: O( log(Np) )

• An example key/node space

0

N: nodeK: key

2727J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Caching VideoCaching Video• Peers locally determine which portion to cache

• Distributed Stream Buffer (DSB)– Peer’s local buffer to hold a fraction of video– A finite size of cache (e.g., size of 2 to 4 minutes of video)– Independent of playback buffer

• Static contents in cache– No content change once the cache is full– Provides a bottom line performance

2828J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

A lookup on Chord OverlayA lookup on Chord Overlay• Fast search using a finger table. • Each node has more detailed knowledge about

nodes closer to them.

2929J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

RegistrationRegistration• Approach 2: Simplified dependence model

Ai : Number of insertions for key i ( Number of arrivals during the slot i)

3030J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Simultaneous RetrievalSimultaneous Retrieval

3131J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Retrieval AnalysisRetrieval Analysis• With larger offsets

– (+) The number of lookups decreases slightly.– (-) Peers have to switch earlier to another peer.– The simulation results match the model

• Parameters– Np = 500– L = 7200– D = 480s– Q = 5s– Number of slots: 1441

(M = 1440)

3232J. Noh : Pseudo-DHT: Distributed Search for P2P Streaming Dec. 15, 2008

Analysis: Overhead for Key StorageAnalysis: Overhead for Key Storage• Poisson process property:

– Given N(t) = N, N arrival times are independently uniformly distributed.

– In Poisson process, interarrival time between events is exponentially distributed.

– The converse is also true.

Finally,