1

Dynamic Streaming over HTTP

CSCU9YH - DASH

SettingtheContext

2

Streaming stored video:

CSCU9YH - DASH

1. videorecorded (e.g., 30 frames/sec)

2. videosent

streaming: at this time, client playing out early part of video, while server still sending laterpart of video

network delay(fixed in this

example)time

3. video received,played out at client(30 frames/sec)

Streaming stored video: challenges

CSCU9YH - DASH

v continuous playout constraint: once client playout begins, playback must match original timing § … but network delays are variable (jitter), so

will need client-side buffer to match playout requirements

v other challenges:§ client interactivity: pause, fast-forward,

rewind, jump through video§ video packets may be lost, retransmitted

3

constant bit rate video

transmission

time

variablenetworkdelay

client videoreception

constant bit rate video

playout at client

client playoutdelay

buffe

red

vide

o

Streaming stored video: revisted

• client-side buffering and playout delay: compensate for network-added delay, delay jitter

CSCU9YH - DASH

Client-side buffering, playout

CSCU9YH - DASH

variable fill rate, x(t)

client application buffer, size B

playout rate,e.g., CBR r

buffer fill level, Q(t)

video server

client

4

Client-side buffering, playout

CSCU9YH - DASH

variable fill rate, x(t)

client application buffer, size B

playout rate,e.g., CBR r

buffer fill level, Q(t)

video server

client

1. Initial fill of buffer until playout begins at tp

2. playout begins at tp, 3. buffer fill level varies over time as fill rate x(t) varies and playout rate r is constant

Client-side buffering, playout

playout buffering:averagefillrate(x),playout rate(r):• x<r:buffereventuallyempties(causingfreezingofvideoplayout until

bufferagainfills)• x>r:bufferwillnotempty,providedinitialplayout delayislarge

enoughtoabsorbvariabilityinx(t)– initialplayout delaytradeoff:bufferstarvationlesslikelywithlargerdelay,butlargerdelayuntiluserbeginswatching

CSCU9YH - DASH

variable fill rate, x(t)

client application buffer, size B

playout rate,e.g., CBR r

buffer fill level, Q(t)

video server

5

Streaming multimedia: UDP• server sends at rate appropriate for client

– often: send rate = encoding rate = constant rate– transmission rate can be oblivious to congestion

levels• short playout delay (2-5 seconds) to remove

network jitter• error recovery: application-level, time-

permitting• RTP [RFC 2326]: multimedia payload types• UDP may not go through firewalls

CSCU9YH - DASH

Real-Time Protocol (RTP)

• RTP specifies packet structure for packets carrying audio, video data

• RFC 3550• RTP packet provides

– payload type identification

– packet sequence numbering

– time stamping

• RTP runs in end systems• RTP packets

encapsulated in UDP segments

• interoperability: if two VoIP applications run RTP, they may be able to work together

CSCU9YH - DASH

6

RTP runs on top of UDP

CSCU9YH - DASH

RTP libraries provide transport-layer interface that extends UDP:

• port numbers, IP addresses• payload type identification• packet sequence numbering• time-stamping

RTP exampleexample:sending64kbpsPCM-encodedvoiceoverRTP• applicationcollectsencodeddatainchunks,e.g.,every20msec =160bytesinachunk

• audiochunk+RTPheaderformRTPpacket,whichisencapsulatedinUDPsegment

• RTP header indicates type of audio encoding in each packet– sender can change

encoding during conference

• RTP header also contains sequence numbers, timestamps

CSCU9YH - DASH

7

RTP and QoS

• RTP does not provide any mechanism to ensure timely data delivery or other QoS guarantees

• RTP encapsulation only seen at end systems (not by intermediate routers)– routers provide best-effort service, making no

special effort to ensure that RTP packets arrive at destination in timely matter

CSCU9YH - DASH

RTP header

CSCU9YH - DASH

payload type (7 bits): indicates type of encoding currently being used. If sender changes encoding during call, sender informs receiver via payload type field

Payload type 0: PCM mu-law, 64 kbpsPayload type 3: GSM, 13 kbpsPayload type 7: LPC, 2.4 kbpsPayload type 26: Motion JPEGPayload type 31: H.261Payload type 33: MPEG2 video

sequence # (16 bits): increment by one for each RTP packet sentv detect packet loss, restore packet sequence

payload type

sequence number

type

time stamp SynchronizationSource ID

Miscellaneous fields

8

RTP header

• timestamp field (32 bits long): sampling instant of first byte in this RTP data packet– for audio, timestamp clock increments by one for each

sampling period (e.g., each 125 usecs for 8 KHz sampling clock)

– if application generates chunks of 160 encoded samples, timestamp increases by 160 for each RTP packet when source is active. Timestamp clock continues to increase at constant rate when source is inactive.

• SSRC field (32 bits long): identifies source of RTP stream. Each stream in RTP session has distinct SSRC

CSCU9YH - DASH

payload type

sequence number

type

time stamp SynchronizationSource ID

Miscellaneous fields

RTSP/RTP programming assignment

• build a server that encapsulates stored video frames into RTP packets– grab video frame, add RTP headers, create UDP

segments, send segments to UDP socket– include seq numbers and time stamps– client RTP provided for you

• also write client side of RTSP– issue play/pause commands– server RTSP provided for you

CSCU9YH - DASH

9

Real-Time Control Protocol (RTCP)

• works in conjunction with RTP

• each participant in RTP session periodically sends RTCP control packets to all other participants

• each RTCP packet contains sender and/or receiver reports– report statistics useful to

application: # packets sent, # packets lost, interarrival jitter

• feedback used to control performance– sender may modify its

transmissions based on feedback

CSCU9YH - DASH

RTCP: multiple multicast senders

CSCU9YH - DASH

veach RTP session: typically a single multicast address; all RTP /RTCP packets belonging to session use multicast address

vRTP, RTCP packets distinguished from each other via distinct port numbers

vto limit traffic, each participant reduces RTCP traffic as number of conference participants increases

RTCPRTP

RTCPRTCP

sender

receivers

10

RTCP: packet types

receiver report packets:• fraction of packets lost, last

sequence number, average interarrival jitter

sender report packets: • SSRC of RTP stream,

current time, number of packets sent, number of bytes sent

source description packets:

• e-mail address of sender, sender's name, SSRC of associated RTP stream

• provide mapping between the SSRC and the user/host name

CSCU9YH - DASH

RTCP: stream synchronization

• RTCP can synchronize different media streams within a RTP session

• e.g., videoconferencing app: each sender generates one RTP stream for video, one for audio.

• timestamps in RTP packets tied to the video, audio sampling clocks– not tied to wall-clock time

• each RTCP sender-report packet contains (for most recently generated packet in associated RTP stream):– timestamp of RTP packet – wall-clock time for when

packet was created• receivers uses association to

synchronize playout of audio, video

CSCU9YH - DASH

11

RTCP: bandwidth scaling

RTCPattemptstolimititstrafficto5%ofsessionbandwidth

example:onesender,sendingvideoat2Mbps

• RTCPattemptstolimitRTCPtrafficto100Kbps

• RTCPgives75%ofratetoreceivers;remaining25%tosender

• 75 kbps is equally shared among receivers: – with R receivers, each receiver

gets to send RTCP traffic at 75/R kbps.

• sender gets to send RTCP traffic at 25 kbps.

• participant determines RTCP packet transmission period by calculating avg RTCP packet size (across entire session) and dividing by allocated rate

CSCU9YH - DASH

Streaming multimedia: HTTP• multimediafileretrievedviaHTTPGET• sendatmaximumpossiblerateunderTCP

• fillratefluctuatesduetoTCPcongestioncontrol,retransmissions(in-orderdelivery)

• largerplayout delay:smoothTCPdeliveryrate• HTTP/TCPpassesmoreeasilythroughfirewalls

CSCU9YH - DASH

variable rate, x(t)

TCP send buffer

videofile

TCP receive buffer

application playout buffer

server client

12

Internet Multimedia Protocol Stack

CSCU9YH - DASH

AAL3/4

IP Version 4, IP Version 6

UDP

Media encaps(H.264, MPEG-4)

RTP

ATM/Fiber Optics Ethernet/WiFi

TCP

SIP RTSP RSVP RTCP

AAL5

KE

RN

EL

AP

PLIC

ATION

Layer 4(Transport)

Layer 3(Network)

Layer 2(Link/MAC)

Layer 5(Session)

MPLS

DCCP

DASH

HTTP

Synchronization Service

Adaptive Streaming Concept• Adaptive Streaming technologies enable

– Optimal streaming video viewing experience for diverse range of devices over broad set of connection speeds

• Adaptive streaming technologies share– Production of multiple files from the same source file to

distribute to viewers watching on different powered devices via different connection speeds

– Distribution of files adaptively, changing stream that is delivered to adapt to changes in effective throughput and available CPU cycles on playback stations

– Transparent operation to the user so that the viewer clicks one button and all streams switch/adapt behind the scenes.

CSCU9YH - DASH

13

Adaptive HTTP Streaming System (Protocol)

• Server– Can be standard web

server– Media segment can be

prepared in-line or off-line

• Client – Sends series of HTTP GET

segment requests and receives segments

– Performs rate adaptation before sending a new GET segment request

CSCU9YH - DASH

Client-centric approach • Client has best view of network conditions• No session state in network

– Redundancy– Scalability

• Faster innovation and experimentation• But, relies on client for operational metrics

– Only client knows what really happens

CSCU9YH - DASH

14

Terms and Definitions of Adaptive HTTP Streaming

• Need – Media Presentation Description (MDP) which

provides metadata • For requesting (GET request) media segments• For rate adaptation purpose

– Segment which may include media data or metadata to decode

• Need DASH

CSCU9YH - DASH

Streaming multimedia: DASH• DASH: Dynamic, Adaptive Streaming over HTTP• server:

– divides video file into multiple chunks– each chunk stored, encoded at different rates – manifest file: provides URLs for different chunks

• client:– periodically measures server-to-client bandwidth– consulting manifest, requests one chunk at a time

• chooses maximum coding rate sustainable given current bandwidth

• can choose different coding rates at different points in time (depending on available bandwidth at time)

CSCU9YH - DASH

15

Streaming multimedia: DASH

• DASH: Dynamic, Adaptive Streaming over HTTP

• “intelligence” at client: client determines– when to request chunk (so that buffer

starvation, or overflow does not occur)– what encoding rate to request (higher quality

when more bandwidth available) – where to request chunk (can request from URL

server that is “close” to client or has high available bandwidth)

CSCU9YH - DASH

DASH – Dynamic Adaptive Streaming over HTTP

• Dash is NOT– System, protocol, presentation, codec, interactivity

• What is DASH – Enabler which provides formats to enable efficient and high-

quality delivery of streaming services over the Internet– Component of end-to-end service– Enabler to reuse existing technologies (containers, DRM

(Digital Rights Management), codecs)– Enabler for deployment on top of HTTP-CDNs– Enabler for very high user experience (low start-up, no re-

buffering)– Provides simple inter-operability points (profiles)

CSCU9YH - DASH

16

Internetmultimedia:StreamingApproach

• browser GETs metafile• browser launches player, passing metafile• player contacts server

• server streams audio/video to player

CSCU9YH - DASH

Streamingfromastreamingserver

• Thisarchitectureallowsfornon-HTTPprotocolbetweenserverandmediaplayer

• CanalsouseUDPinsteadofTCP.

CSCU9YH - DASH

17

DASH Client

Thomas Stockhammer, Qualcomm, “DASH – Design Principles and Standards , Presentation at MMSys 2011 CSCU9YH - DASH

Information Classification• DASH uses MPD (Media Presentation Descriptor) and

Index Information as metadata for DASH Access Client• Initialization and Media Segments for Media Engine

– Reuse of existing container format

CSCU9YH - DASHSource: MMSys’11

18

Media Presentation Data ModelMDP - description of accessible segments and corresponding timing

Source: Stockhammer, Qualcomm, “DASH – Design Principles and Standards , Presentation at MMSys 2011 CSCU9YH - DASH

MediaPresentationDescriptionFile<?xmlversion="1.0"?><MPDxmlns:xsi=http://www.w3.org/2001/XMLSchema-instance xmlns="urn:mpeg:DASH:schema:MPD:2011"xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011DASH-MPD.xsd"type="static"mediaPresentationDuration="PT3256S"minBufferTime="PT1.2S"profiles="urn:mpeg:dash:profile:isoff-on-demand:2011">

<BaseURL>http://cdn1.example.com/</BaseURL><Period><!-- Audio--><AdaptationSet mimeType="audio/mp4"codecs="mp4a.0x40"lang="en"subsegmentAlignment="true"subsegmentStartsWithSAP="1"><ContentProtectionschemeIdUri="urn:uuid:706D6953-656C-5244-4D48-656164657221"/><Representationid="1"bandwidth="64000"><BaseURL>7657412348.mp4</BaseURL></Representation><Representationid="2"bandwidth="32000"><BaseURL>3463646346.mp4</BaseURL></Representation></AdaptationSet>

<!-- Video--><AdaptationSet mimeType="video/mp4"codecs="avc1.4d0228"subsegmentAlignment="true"subsegmentStartsWithSAP="2"><ContentProtectionschemeIdUri="urn:uuid:706D6953-656C-5244-4D48-656164657221"/>

<Representationid="1"bandwidth="1024000"width="640"height="480"><BaseURL>streamer.harvard.edu/e139_vid_2_1mbps.mp4</BaseURL></Representation>

<Representationid="2"bandwidth="2048000"width="1280"height="720"><BaseURL>streamer.harvard.edu/e139_vid_2_2mbps.mp4</BaseURL></Representation></AdaptationSet></Period></MPD>

CSCU9YH - DASH

19

MDP Information • Includes redundant information of media streams

to initially select or reject groups or representations

• Includes access and timing information– Content addressing via HTTP-URLs– Byte range for each accessible segment– Segment availability start and end time in wall-clock time– Approximate media start time and duration– Instructions on starting playout (for live service)

• Includes switching relations across representations

CSCU9YH - DASH

Media Segments (1)

• Contain information to map segment into media presentation timeline for switching and synchronous presentation with other representations

• Can be short (~ 1-10seconds)• Can be long (~10sec – 2 hours)

CSCU9YH - DASH

20

Media Segments (2)

Media segmentduration advantages disadvantages

Shortduration Commonality withlivehighswitchinggranularityonsegmentlevel

- Largenumberoffiles- LargenumberofURLs- Fixedrequestsize- Switching granularityon

segmentlevel

Longduration - Smallnumberoffiles- SmallnumberofURLs- High switching

granularity- Flexiblerequestsizes- Improvedcache

performance

- Needforsegmentindex- Differencefromlive

CSCU9YH - DASH

DASHinAction

Qualcomm

21

AClearerPicture

StreamingMediaGlobal

ProblemSolved!Whataboutcompetingvideostreams?

Bitratestability,andlinkutilizationareknowntosuffer…SoWhat?

4YouTubeover6Mbps

S.Akhshabi,L.Anantakrishnan,A.C.Begen,C.Dovrolis,“WhathappenswhenHTTPadaptivestreamingplayerscompeteforbandwidth?”,inNOSSDAV’12,2012

22

ProblemSolved!Whataboutcompetingvideostreams?

Bitratestability,andlinkutilizationareknowntosuffer…SoWhat?

4YouTubeover6Mbps

*NOT*tobeconfusedwithtransmissionrate

Howabout‘fairness?’

23

Worse,is(un)Fairness!

Two streams compete over 4Mbps Four streams compete over 8Mbps

S.Akhshabi,L.Anantakrishnan,A.C.Begen,C.Dovrolis,“WhathappenswhenHTTPadaptivestreamingplayerscompeteforbandwidth?”,inNOSSDAV’12,2012

ContributingCauses

•VideoplayersrelyonTCPfairness

•Heterogeneousdevices

•Nostandardclientimplementation

47

24

APPLICATIONLAYER

Networkandsystems

APPLICATIONLAYER

PoorunderstandingNostandardsforofTCP(nothingnew)DASHimplementation

(new!)

IgnoreDevicesandAppl’n Heterogeneity,i.e.fairnessmeansequal

NetworkandSystems

25

APPLICATIONLAYER

PoorunderstandingNostandardsforofTCP(nothingnew)DASHimplementation

(new!)

IgnoreDevicesandAppl’n Heterogeneity,i.e.fairnessmeansequal

NetworkandSystems

✗

Latency?Loss?Throughput?

TCP‘TraditionalFlowsAre:

shortorlong

continuous.

StreamingMediaoverTCPis:

intermittent.

Note:TCPpreservesstate!

26

Latency?Loss?Throughput?

TCP‘TraditionalFlowsAre:

shortorlong

continuous.

StreamingMediaoverTCPis:

intermittent.

Note:TCPpreservesstate!