A Case for a Coordinated Video Control Plane - SIGCOMM · 2012-10-26 · A Case for a Coordinated...

A Case for a

Coordinated Video

Control Plane

Xi Liu, Florin Dobrian, Henry Milner,

Junchen Jiang, Vyas Sekar, Ion Stoica,

Hui Zhang

(Conviva, CMU, Intel, and UC Berkeley)

Video Is Dominating the Internet Traffic

Netflix traffic alone exceeds 20% of US traffic1

2011’s Cisco Visual Networking Index2

2011: video represents 51% of the Internet traffic

2016: all types of video will represent 86% of the Internet traffic

The Internet is becoming a Video Network

2http://web.cs.wpi.edu/~claypool/mmsys-2011/Keynote02.pdf

1http://blogs.cisco.com/sp/comments/cisco\_visual\_networking\_index\_forecast\_annual\_update

Video Ecosystem: Data-Plane

Video Source

Encoders & Video Servers

CMS and Hosting

Content Delivery Networks (CDNs)

ISP & Home Net

Screen

Video Player

Video Quality Matters [Sigcomm’11]

Quality has substantial impact on viewer engagement

Need to ensure uninterrupted streaming at high bitrates

Buffering ratio is most critical across video traffic types

Highest impact for live: 1% of buffering reduced play time by 3min

1% increase in buffering can lead to more than 60% loss in

audience over one month

Our Argument

CDN performance varies widely in time, geography, and

ISPs

Opportunity for significantly improving video Quality by

selecting best CDN (and bitrate) for each viewer

Hence, we argue for a logically centralized control plane to

dynamically select CDN and bitrate

Assumptions:• Content is encoded at multiple bitrates• Content is delivered by multiple CDNs

How do We Collect Data?

Streaming Module

UI Controller

Content Manager

Messaging & Serialization

Tobackend

HTTPS

Automatic Monitoring

Player Insight

Pla

yer

Ap

plic

atio

n

Automatic and continuous monitoring of video player

Flash: NetStream, VideoElement

Silverlight: MediaElement, SmoothStreamMediaElement

iOS: MPMoviePlayerElement

What Traffic do We See?

Close to two billions streams per month

Mostly premium content providers (e.g., HBO, ESPN, Disney) but also

User Generated Video sites (e.g., Ustream)

Live events (e.g., NCAA March Madness, FIFA World Cup, MLB), short

VoDs (e.g., MSNBC), and long VoDs (e.g., HBO, Hulu)

Various streaming protocols (e.g., Flash, SmoothStreaming, HLS), and

devices (e.g., PC, iOS devices, Roku, XBOX, …)

Traffic from all major CDNs, including ISP CDNs (e.g., Verizon, AT&T)

CDN Performance Varies Widely

CDNs Vary in Performance over Geographies and Time

There is no single best CDN across geographies,

network, and time

25%

50%

25%

CDN 1

CDN 2

CDN 3

• Metric: buffering ratio

• One month aggregated data-set– Multiple Flash (RTMP) customers

– Three major CDNs

• 31,744 DMA-ASN-hour with > 100 streams from each CDN– DMA: Designated Market Area

• Percentage of DMA-ASN-hour partitions a CDN has lowest buffering ratio

Washington DC (Hagerstown):

ASN-CXA-ALL

10% 20% 100%30% 40% 50% 60% 70% 80% 90%

Washington, DC viewer experience

differed greatly… Comcast viewers got the best

streams from CDN 1 51% of the time

and only 9% from CDN 2

Washington DC (Hagerstown):

VZGNI-TRANSIT (19262)

Verizon users got the best streams from CDN 1 only

17% of the time and 77% from CDN 2

There is no single best CDN in the same geographic

region or over time

CDN Streaming Failures Are Common Events

% of stream failures: % of streams that failed to start

Three months dataset (May-July, 2011) for a premium

customer using Flash

CDN Streaming Failures Are Common Events

% of stream failures: % of streams that failed to start

Three months dataset (May-July, 2011) for a premium

customer using Flash

CDN (relative) performance varies greatly over time

Opportunities for Improving Quality

Possible Actions to Improve Quality

Switch the bitrate

↓ Buffering, high frame drops, high start time, …

↑ High available bandwidth, …

Switch the CDN

↔ Connection error, missing content, buffering on low bitrate, ...

When to perform switching/selection?

Start time selection only

Start time selection & midstream switching

Potential Improvement Example: CDN Switching Only

For each CDN partition clients by

(ASN, DMA)

DMA: Designated Market Area



(ASN, DMA)


For each partition compute:

Buffering ratio

Failure ratio

Start time

….

CDN1 (buffering ratio)

DMA

ASN


DMA

ASN



(ASN, DMA)



Buffering ratio

Failure ratio

Start time

….


DMA

ASN


DMA

ASN

Avg. buff ratio of users in ASN[1]xDMA[1]

streaming from CDN1



(ASN, DMA)



Buffering ratio

Failure ratio

Start time

….


DMA

ASN


DMA

ASN


streaming from CDN1


streaming from CDN2


For each partition select best CDN

and assume all clients in the

partition selected that CDN


DMA

ASN


DMA

ASN






DMA

ASN


DMA

ASN

CDN1 >> CDN2






DMA

ASN


DMA

ASN





Essentially, pick partition with best

quality across CDNs CDN1 (buffering ratio)

DMA

ASN


DMA

ASN

Best CDN (buffering ratio)

DMA

ASN

Potential Improvements

Provider1: large UGV (User Generated Video) site

Provider2: large premium VoD content provider

Base-line: existing assignment of viewers (clients) to CDNs

Metric Provider1 (UGV) Provider2 (Premium)

Base

line

Start-

time

Selectio

n

Mid-

stream

Switching

Base

line

Start-

time

Selection

Mid-

stream

Switching

Buffering

ratio (%)

6.8 2.5 1 1 0.3 0.1

Between x2.7 and x10 improvement in buffering ratio

Coordinated Control Plane for High

Quality Video Delivery

Video Control Plane Architecture

Coordinator implementing a global optimization algorithm that

dynamically select CDN & bitrate for each client based on

Individual client

Aggregate statistics

Content owner policies

(CDN/ISP info)

Content owners (CMS & Origin)

CDN 1

CDN 2

CDN 3

Clients

Coordinator

Co

nti

nu

ou

s m

easu

rem

ents

Bu

sin

ess

Polic

ies

control

Example: Local vs. Global Optimization

0

20

40

60

80

100

0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000Ban

dw

idth

Flu

ctu

atio

n (

%)

Concurrent Viewers

Bandwidth fluctuation = (Max Bandwidth – Min Bandwidth)/(Average Bitrate)

Example: Local vs. Global Optimization

CDN1 DMA

ASN

DMA

ASN

DMA

ASN

CDN2

CDN3

0

20

40

60

80

100

0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000Ban

dw

idth

Flu

ctu

atio

n (

%)

Concurrent Viewers

0

10

20

30

40

0 5000 10000 15000 20000 25000 30000 35000

Ban

dw

idth

Flu

ctu

atio

n (

%)

Concurrent Viewers

ASN/DMA saturated on all CDNs Don’t switch CDN; cap bitrates, instead

Concluding Remarks (I)

Key transition of main-stream video to the Internet

Video quality presents opportunity and challenge

Premium video on big screens zero tolerance for poor quality

Video player continuous monitoring and global optimization

has best chance of delivering high quality video

Many challenges remain, e.g.,

Scalability

How do multiple coordinators interact?

…

Concluding Remarks (II)

The video traffic dominance in the Internet is growing

Over 51% Internet traffic today, will be more than 86% in the next 4

years

The Internet is becoming a Video Network

Managing video delivery and maximizing video quality must

be at the core of any future Internet architecture!

Backup Slides

Conviva Optimization in the Wild

… increased average bit-rate from

1.7 Mbps to 2.1 Mbps…

Reduced views impacted by

buffering from 16.13% to 5.56%

…

… and raised engagement

by 36%

0,00%

5,00%

10,00%

15,00%

20,00%

25,00%

1600

1700

1800

1900

2000

2100

2200

Views Impacted by Buffering

Average Bit Rate

Possible Coordinator ArchitectureContinuous real-time measurements from every client

Inference Engine

Bit Rates

CD

Ns

Decision Engine

Optimize viewer performance by selecting the best option within the set of bit rates and CDNs

Akamai

DMA

ASN

DMA

ASN

DMA

ASN

Limelight

Level3 Time of day

Localize issues by region, network, CDN, and time

Real-time Global Data Aggregation and Correlation

Historical Data Aggregation and

Analysis

Global Inference, Decision & Policy

Engine

Real-time global optimizations

Conviva Services Enhance the Viewer Experience

and Lift Engagement by Lifting Bit Rate and

Reducing Buffering

Increased average bit-rate from 1.6 Mbps to 2.1 Mbps …

1.5%

1.0%

0.5%

DMS LAUNCH EMS LAUNCH

… reduced buffering ratio from 1.5% to 0.5%

… and raised engagement by 36%

Potential Improvements

Customer1: large UGV site

Customer2: large premium content provider

Note: * denotes improvements when using mid-stream

switching

Metric Customer1 Customer2

Current Projected Current Projected

Buffering ratio

(%)

6.8 2.5 / 1* 1 0.3 / 0.1*

Start time (s) 6.41 2.91 1.36 0.9

Failure ratio (%) 16.57 2.4 1.1 0.7

Between x2.7 and x10 improvement in buffering ratio

Video Quality Matters [Sigcomm’11]

Quality has substantial impact on viewer engagement

Need to ensure uninterrupted streaming at high bitrates

Buffering ratio is most critical across genres

Highest impact for live: 1% of buffering reduced play time by 3min

1% increase in buffering leads to more than 60% loss in

audience

1% difference in buffering between two ISPs

68% monthly loss in uniques for ISPwith poor performance

Customer1: Start-time vs. Midstream CDN

Switching

78%

84%

90%

Provider1: Oracle vs. Historical

Base-lineOracle

Historic


Oracle:


and assume all clients in the partition

selected that CDN

Essentially, pick partition with best

quality across CDNsCDN1 (buffering ratio)

DMA

ASN


DMA

ASN

Best CDN (buffering ratio)

DMA

ASN


Details

If a partition has not enough clients

use a larger partition

?


DMA

ASN


Details

If a partition has not enough clients

use a larger partition

Use quality metric distribution to

predict quality of a client on new

CDNCDN1 (buffering ratio)

DMA

ASN


DMA

ASN

CDNs Vary in Performance over Geographies and Time

0,0%

10,0%

20,0%

30,0%

40,0%

50,0%

60,0%

70,0%

80,0%

90,0%

100,0%

Was

hin

gto

n D

C (

Hag

erst

ow

n):

CM

CS(

33

65

7)

Milw

auke

e:R

OA

DR

UN

NER

-CEN

TRA

L(2

02

31

)

Gre

en B

ay -

Ap

ple

ton

:SC

RR

-70

15

(70

17

)

Den

ver:

ASN

-QW

EST(

20

9)

Ch

arlo

tte:

SCR

R-1

14

26

(11

42

6)

Was

hin

gto

n D

C (

Hag

erst

ow

n):

ASN

-CX

A-A

LL-C

CI-

22

773

-…

Ph

ilad

elp

hia

:VZG

NI-

TRA

NSI

T(1

92

62

)

San

Die

go:S

BIS

-AS(

71

32

)

Las

Veg

as:A

SN-C

XA

-LV

-13

43

2-C

BS(

13

43

2)

Mad

iso

n:C

HA

RTE

R-N

ET-H

KY-

NC

(20

115

)

Ind

ian

apo

lis:S

BIS

-AS(

71

32

)

Pro

vid

ence

-N

ew B

edfo

rd:A

SN-C

XA

-ALL

-CC

I-2

277

3-…

Was

hin

gto

n D

C (

Hag

erst

ow

n):

VZG

NI-

TRA

NSI

T(1

92

62

)

Har

tfo

rd &

New

Hav

en:S

BIS

-AS(

71

32

)

Ho

ust

on

:SB

IS-A

S(7

13

2)

Gra

nd

Rap

ids

-K

alam

azo

o -

Bat

tle

Cre

ek:S

BIS

-AS(

71

32

)

Atl

anta

:BEL

LSO

UTH

-NET

-BLK

(638

9)

Ho

no

lulu

:HA

WA

IIA

N-T

ELC

OM

(36

149

)

Atl

anta

:CO

MC

AST

-77

25

(77

25)

Was

hin

gto

n D

C (

Hag

erst

ow

n):

SPC

S(1

05

07

)

Orl

and

o -

Day

ton

a B

each

-M

elb

ou

rne:

EMB

AR

Q-…

Den

ver:

CM

CS(

33

65

2)

No

rfo

lk -

Po

rtsm

ou

th -

New

po

rt N

ews:

ASN

-CX

A-A

LL-C

CI-

…

Sain

t Lo

uis

:CH

AR

TER

-NET

-HK

Y-N

C(2

01

15

)

Cin

cin

nat

i:FU

SE-N

ET(6

18

1)

Ph

oen

ix:A

SN-Q

WES

T(2

09

)

Dal

las

-Fo

rt W

ort

h:V

ZGN

I-TR

AN

SIT(

19

26

2)

Pit

tsb

urg

h:C

CC

H-3

(70

16

)

Sain

t Lo

uis

:SB

IS-A

S(7

13

2)

San

Fra

nci

sco

-O

akla

nd

-Sa

n J

ose

:SB

IS-A

S(7

13

2)

San

Die

go:R

OA

DR

UN

NER

-WES

T(2

00

01

)

Gre

envi

lle -

Spar

tan

sbu

rg -

Ash

evill

e -…

Kan

sas

Cit

y:SB

IS-A

S(7

13

2)

Co

lum

bu

s -

OH

:SC

RR

-10

79

6(1

07

96

)

Lou

isvi

lle:I

NSI

GH

T-C

OM

MU

NIC

ATI

ON

S-C

OR

P-A

S1(3

67

27)

Ch

icag

o:A

TT-I

NTE

RN

ET3(

64

78

)

Kan

sas

Cit

y:A

SN-C

XA

-ALL

-CC

I-2

277

3-R

DC

(22

77

3)

Mia

mi -

Fort

Lau

der

dal

e:B

ELLS

OU

TH-N

ET-B

LK(6

38

9)

Cle

vela

nd

:NEO

-RR

-CO

M(1

10

60

)

Ch

icag

o:V

ZGN

I-TR

AN

SIT(

19

262

)

Co

lum

bia

-SC

:SC

RR

-11

42

6(1

14

26

)

Dal

las

-Fo

rt W

ort

h:S

BIS

-AS(

71

32

)

Det

roit

:SB

IS-A

S(7

13

2)

Kan

sas

Cit

y:SC

RR

-11

95

5(1

19

55

)

Los

An

gele

s:C

HA

RTE

R-N

ET-H

KY-

NC

(201

15

)

Mia

mi -

Fort

Lau

der

dal

e:C

OM

CA

ST-2

02

14

(20

21

4)

Cle

vela

nd

:SC

RR

-10

79

6(1

07

96

)

Wes

t P

alm

Bea

ch -

Fort

Pie

rce:

CO

MC

AST

-20

21

4(2

02

14)

San

Die

go:A

SN-C

XA

-ALL

-CC

I-22

77

3-R

DC

(22

773

)

Seat

tle

-Ta

com

a:A

SN-Q

WES

T(2

09)

New

Yo

rk:C

AB

LE-N

ET-1

(61

28

)

Po

rtla

nd

-O

R:A

SN-Q

WES

T(2

09

)

Okl

aho

ma

Cit

y:SB

IS-A

S(7

13

2)

Ph

oen

ix:A

SN-C

XA

-ALL

-CC

I-2

277

3-R

DC

(22

77

3)

Ch

icag

o:S

BIS

-AS(

71

32

)

Gre

ensb

oro

-H

igh

Po

int

-W

inst

on

-Sal

em:S

CR

R-…

Alb

any

-Sc

hen

ecta

dy

-Tr

oy:

RR

-NYS

REG

ION

-ASN

-01

(11

35

1)

Tyle

r -

Lon

gvie

w (

Lufk

in &

Nac

ogd

och

es):

SUD

DEN

LIN

K-…

Gra

nd

Rap

ids

-K

alam

azo

o -

Bat

tle

Cre

ek:C

MC

S(3

36

68

)

Ho

ust

on

:CM

CS(

33

66

2)

Har

tfo

rd &

New

Hav

en:C

OM

CA

ST-7

01

5(7

01

5)

Euge

ne:

ASN

-QW

EST(

209

)

San

Fra

nci

sco

-O

akla

nd

-Sa

n J

ose

:CM

CS(

33

65

1)


Oracle:


and assume all clients in the partition

selected that CDN

Historical:

For each partition select best CDN in

previous epoch, and assign clients to

that CDN in next epoch


DMA

ASN


DMA

ASN

A Case for a Coordinated Video Control Plane - SIGCOMM · 2012-10-26 · A Case for a Coordinated...

Documents

Transcript of A Case for a Coordinated Video Control Plane - SIGCOMM · 2012-10-26 · A Case for a Coordinated...