Scaling Video Infrastructure

Dr. Alex GouaillardCTO Temasys Communications

Limits of P2P

Other interesting configurations / use cases

MCU internals

Open Source MCUs

Limits of P2P

Other interesting configurations / use cases

MCU internals

Open Source MCUs

Limits of P2P: client side scalability

Alice Bob

Alice Bob

Chris

Alice Bob

Chris David

2

6

12

Alice

Bob

Chris

EdDave

20

n(n-1) O(n^2)

Business as usual MCU: multiplexing streams - wow, super bandwidth ! - Hum, wait, what about latency? - What happen if peers are far away …. ?

- Interface layout ??=> CP, VAS

Ok, that’s great but ….

- Chat? I mean, if only to pass a link- Collaboration (white board?)- Services other than A/V?

- Usage beyond the boardroom model: BYOD device, work from home, ….

Alice

Bob

Chris

EdDave

MCU

5

n O(n)

Smarter topology ?

Alice

Bob

Chris

EdDave

broadcast

12,5

(n^2)/2 O(n^2)

typeStreams

INStreams

OUT

p2p N-1 N-1

MCU 1 1

Hybrid N-1 1

Smarter topology ? - bandwidth optimization WITH layout freedom

2 7 12 17 220

100

200

300

400

500

600

typeStreams

INStreams

OUT

p2p N-1 N-1

MCU 1 1

Hybrid N-1 1

Limits of P2P

Other interesting configurations / use cases

MCU internals

Open Source MCUs

Broadcast / seminar

Prof

Broadcast

Broadcast

Broadcast

Broadcast

Broadcast

S

S

S

S

S

S

S

SS

S

S

S

SS

S

S

Heterogeneous – Bandwidth, CPU, Display - Concept

Desk.

Desk.

Cr.Book

Mobile

Mobile

MCU

HDHD

SD

QVGA

QVGA

Heterogeneous – Case 1: simulcast

The user send the same stream at different resolutions

Pro: easy on the server side (switch)Cons: overhead on client (still better than p2p)

Desk.

HD

MCUSD

QVGA

Cr. Book

Mobile

Desk.

HD

SD

QVGA

Heterogeneous – Case 2: video Proc. In MCU

Scale down and/or drop frames in the MCU

Pro: no overhead on clientCon: full overhead on server (need access to raw frame)

Desk. MCUHD Cr. Book

Mobile

Desk.

HD

SD

QVGA

Heterogeneous – Case 3: streams management In MCU (SVC)

- PRO: Only 30% overhead on client.- PRO: No need for raw media on server

Limits of P2P

Other interesting configurations / use cases

MCU internals

Open Source MCUs

MCU Internals – The Onion

decryptunstream unload

UDPTCPTLS

SRTP RTP VP8

Decode

MAGIC

Raw Media

loadencode encrypt

VP8 RTP SRTP

stream

UDPTCPTLS

MCU Internals – Variation 1: old school MCU

decryptunstream unload

UDPTCPTLS

SRTP RTP VP8

Decode Scale & Rotate

Raw Media

loadencode encrypt

VP8 RTP SRTP

stream

UDPTCPTLS

decryptunstream unload

UDPTCPTLS

SRTP RTP VP8

Decode Scale & Rotate

Raw Media

decryptunstream unload

UDPTCPTLS

SRTP RTP VP8

Decode Scale & Rotate

Raw Media

MCU Internals – Variation 2: relay / TURN

decryptunstream unload

UDPTCPTLS

SRTP RTP VP8

Decode

MAGIC

Raw Media

loadencode encrypt

VP8 RTP SRTP

stream

UDPTCPTLS

MCU Internals – Variation 3: Broadcast

unstream

UDPTCPTLS

stream

UDPTCPTLS

multicopy

stream

UDPTCPTLS

stream

UDPTCPTLS

stream

UDPTCPTLS

stream

UDPTCPTLS

MCU Internals – Variation 4: Recording

decryptunstream unload

UDPTCPTLS

SRTP RTP VP8

Decode

record

Raw Media

MCU Internals – Variation 5: Transcoding

decryptunstream unload

UDPTCPTLS

SRTP RTP VP8

Decode

Raw Media

loadencode encrypt

H264 RTP SRTP

stream

UDPTCPTLS

Limits of P2P

Other interesting configurations / use cases

MCU internals

Open Source MCUs

Foreword on Open Source MCUs● All Chrome/Mozilla ready (except Jitsi)● No data channel (Janus is the closest to have it)● None gave huge CPU saving but expect bandwidth saving

Kurento (Total files:210/LoC:25000) - JAVA

PROS CONS

● Licensed under L-GPL License.● 2$M EUR support from EU● Using GStreamer● Developers answering quickly

● Large project, Complicated design● no low level API ● 3 entities: Web Server -> Connector-

>Media Server● Academic driven.

● Webrtc Loopback to Http player. Http player = possibility to integrate into IE or Safari but latency of 5 secs. Currently there is a bug on the IE/Safari support.

● Audio + Video recorder into webm or mp4 format (one user only)Currently the javascript API is “one user only”. This means that it is not possible to mix source of different session

(availability expected: end of the month).Available with the JAVA API:

● Video conferencing up to 4 peers

Licode/ Lynckia (Total files:110/LoC:26000) – C++

PROS CONS

● MIT License ● No working example/demo● Project dead? (last commit on the

MCU 2 months ago: https://github.com/ging/licode/tree/master/erizo)

Personal Comment: Even the demos on their website are down: http://chotis2.dit.upm.es/

Jitsi (Total files:15/LoC:8000) - JAVA

PROS CONS

● Licensed under MIT license● Small code base● Bandwidth improvement● XMPP (federation, presence, …)

● Installation can be difficult● No real CPU improvement

What I did with it/Available demos tested:● Video conferencing https://meet.jit.si/

Janus (Total files:30/LoC:14,000) - C

PROS CONS

● Low level. CPU efficient.● Easy to install/use● Bandwidth management● Easy Extension through plugin arch.● Developer answering FAST.● License: GPL

• Each function needs to be a plugin

Personal Comment: I have been able to test many functionalities and easily. After one day studying the source code I’m able to understand the main functionalities

What I did with it/Available demos tested:● Video/Audio conferencing with control for the number of publisher (1 publisher = broadcasting) and number of

person in the room● Screensharing● Streaming Audio file: on demand (different streaming contexts for each client) or live (same streaming context for

each client)● Streaming rtp source using a rtp source generator (gstreamer = audio+video -> gateway -> clients)● Audio conferencing: mix all audio of other peers into one audio stream. Possible to record the audio into wav.● Audio recording into .opus format● NOT TESTED Sip Gateway (works with Asterisk)

QUESTIONS

THANK YOU!