Encoding For Adaptive Streaming
-
Upload
trankhuong -
Category
Documents
-
view
241 -
download
0
Transcript of Encoding For Adaptive Streaming
1
Encoding forAdaptive Streaming
Jan Ozer
www.streaminglearningcenter.com
276-238-9135
Copyright © 2011 Jan Ozer, All Rights Reserved
Encoding for Adaptive Streaming
Introduction to adaptive streaming
How they work RTMP (server-based)
HTTP (no server)
Generic issues How many streams/what data rates
Encoding for adaptive streaming
Transmuxing
2
Copyright © 2011 Jan Ozer, All Rights Reserved
For Additional Reference
The material in this presentation was derived from Jan Ozer’s book, Video Compression for Flash, Apple Devices and HTML5.
For further explanation of the concepts presented in this presentation, check out the book at bit.ly/ozerbook1
Copyright © 2011 Jan Ozer, All Rights Reserved
Technology Overview
Streams switched to adapt to factors like: Changing delivery bandwidths (avoid hard stops) CPU utilization at client (avoid frame drops)
Information is gathered by player Server-based systems (RTMP Flash) deliver a different
stream when change is required, switching at key frame
HTTP-based systems (HTTP Flash, iOS) use 2-10 second file chunks Player retrieves chunk from different source file to
effectuate stream switch (more later)
3
Copyright © 2011 Jan Ozer, All Rights Reserved
RTMP Flash - Overview
Server driven system Uses files as encoded (no chunking) MP4, F4V, FLV, whatever
Server switches as necessary at key frame
Copyright © 2011 Jan Ozer, All Rights Reserved
Apple HTTP Live Streaming: How it works
Encoding Encode as normal, send
to segmenter Files chunked, inserted
into transport stream (.tsextension)
Manifest file (M3U8) created
Uploaded to server
Client Monitors heuristics Changes retrieved stream
as necessary
4
Copyright © 2011 Jan Ozer, All Rights Reserved
The Manifest File Structure
Copyright © 2011 Jan Ozer, All Rights Reserved
HTTP Flash - How it Works
5
Copyright © 2011 Jan Ozer, All Rights Reserved
To Boil it Down-HTTP Flash
MP4
.F4F
.F4M
Into Packager
Media file chunks
Manifest files that tellplayer where to findfile chunks
Player driven system Player monitors heuristics Retrieves different stream as necessary
Copyright © 2011 Jan Ozer, All Rights Reserved
Smooth Streaming – How It Works
Manifest file
Chunked content file
6
Copyright © 2011 Jan Ozer, All Rights Reserved
Summary
RTMP Server driven
No special encoding requirement
HTTP No server
Files must be chunked; metadata created
Player does all the work
Copyright © 2011 Jan Ozer, All Rights Reserved
Generic Encoding Issues
How many streams What configuration
What data rates
7
Copyright © 2011 Jan Ozer, All Rights Reserved
Inputs for Analysis
Multiple case studies (Harvard, Turner, MTV, NBC, Indiana University) StreamingMedia here: bit.ly/ozeradaptive
White papers and other guides as identified throughout
Several consulting projects
Copyright © 2011 Jan Ozer, All Rights Reserved
How Many Streams?
Considerations Enough to provide relevant coverage SD – 3-4 (320x240 – 640x480)
HD – need additional streams to provide full spectrum
MTV At least one stream for each window size served
8
Copyright © 2011 Jan Ozer, All Rights Reserved
How Many Streams?
Considerations Number of window sizes served
MTV.com
Copyright © 2011 Jan Ozer, All Rights Reserved
How Many Streams?
MTV Schema Have at least one for each window size served
(optimum quality when stream size=window size)
Never switch to stream larger than current viewing size So, if playing in 640x360 window, don’t switch to
768x432 unless viewer expands window or goes full screen
9
Copyright © 2011 Jan Ozer, All Rights Reserved
How Many Streams?
Other considerations Subscription – if paid, more streams for better
service Major League Baseball – 11 streams
More if entertainment
MTV with 8, though Turner (PGA golf) with 4 Fewer if education or corporate
Indiana - 3, Harvard, 5
Copyright © 2011 Jan Ozer, All Rights Reserved
Stream Count – Bottom Line
There are no hard and fast rules More streams required for HD than SD Just a function of size
More for entertainment than education/business Entertainment – about the experience
Business – it’s about making sure the viewer can watch the stream
More for subscription than general entertainment Provide more options when viewer is paying
10
Copyright © 2011 Jan Ozer, All Rights Reserved
What Resolution?
Other considerations Never encode at larger than source Scaling upwards degrades quality
Very few producers adhere to mod-16 (where height/width are divisible by 16) But all favor at least mod-8, all at least mod-4
SD adaptive - many producers use the same resolution for all streams (640x480)
Copyright © 2011 Jan Ozer, All Rights Reserved
A Word About Frame Rates
At extremely low data rates, often more effective to drop frame rate than resolution So: 640x480@ 15 fps is better than
320x240@ 30 fps
In particular, Apple recommends dropping frame rate on lowest data rate streams
11
Copyright © 2011 Jan Ozer, All Rights Reserved
Format Specific: HTTP Live Streaming
Bit.ly/bestpracticehttplive
Copyright © 2011 Jan Ozer, All Rights Reserved
Format Specific: HTTP Live Streaming
Adobe white paper Cluster around window
sizes Switches least apparent
w/in same window size
bit.ly/Levkovhttp
12
Copyright © 2011 Jan Ozer, All Rights Reserved
Format Specific: Smooth Streaming
Copyright © 2011 Jan Ozer, All Rights Reserved
Format Specific: RTMP/HTTP Flash
Adobe Media Encoder Presets
http://bit.ly/adaptivepresets
13
Copyright © 2011 Jan Ozer, All Rights Reserved
What Data Rates?
Considerations Must be sufficiently far apart to avoid
constant switching
Will vary with stream size Smaller gaps at lower bandwidths
Larger gaps at higher bandwidths
Copyright © 2011 Jan Ozer, All Rights Reserved
What Data Rates?
MTV Schema
14
Copyright © 2011 Jan Ozer, All Rights Reserved
What Data Rates?
http://alexzambelli.com/WMV/MBRCalc.html For Smooth Streaming, but good starting point
Enter parameters, calculates rez/data rate for you
Attempts to keep quality consistent at all data rates
Copyright © 2011 Jan Ozer, All Rights Reserved
How should H.264 encoding parameters change?
Now we know stream count, resolution and data rate
How do we customize encoding for adaptive? Key frame settings
Bit rate control
Audio parameters
15
Copyright © 2011 Jan Ozer, All Rights Reserved
Key Frame Interval
Why important - RTMP Flash – stream switches occur at key frame Key frame location must be identical in all streams Use same interval
Disable scene change detection
Typically shorter (2-5 seconds) to enable more responsive switching
Copyright © 2011 Jan Ozer, All Rights Reserved
Key Frame Interval
Why important – HTTP technologies - key frame must be first frame of every chunk Key frame interval must divide evenly into chunk
duration If 9-sec. chunks, use key frame interval of 3 sec
As with RTMP, key frame location must be identical in all streams, so:
Use same interval Disable scene change detection
16
Copyright © 2011 Jan Ozer, All Rights Reserved
In General - VBR vs CBR
CBR more conservative Produces smoothest stream, which is easiest
to consistently deliver
But, quality is typically inferior to VBR Sometimes noticeably so
Copyright © 2011 Jan Ozer, All Rights Reserved
VBR vs. CBR
17
Copyright © 2011 Jan Ozer, All Rights Reserved
In General - VBR vs CBR
VBR Variable data rate
Could introduce stream switches that relate to bit rate control rather than change in conditions
Too many stream switches degrade perceived quality
Copyright © 2011 Jan Ozer, All Rights Reserved
VBR vs CBRVideo bitrate
A B C
Player buffer
A
B
Buffer OKStatus quoNo change
Ruh, roh!Buffer too low
Switch to lowerbitrate
med chunk Good delivery
large chunk Slow delivery
CSmall chunk Very fast delivery
Life is good!Switch to higher
bitrate
18
Copyright © 2011 Jan Ozer, All Rights Reserved
What’s the Answer?
Most conservative Use CBR
Many real world producers Use constrained VBR,
Less constrained at high data rates, where difference in bandwidths is great (MTV uses 2X constrained)
Copyright © 2011 Jan Ozer, All Rights Reserved
In General - Audio Parameters
Most conservative - use same parameters for all files Popping can occur if audio parameters change But, doesn’t optimize experience at higher bit rates
If you do switch audio parameters Switch from stereo to mono at same per channel
sampling rate and bit rate From 128 kbps/44 kHz/16-bit/stereo to 64 kbps/44 kHz/16-
bit/mono
Test to ensure no artifacts when switching streams
19
Copyright © 2011 Jan Ozer, All Rights Reserved
H.264 Encoding Parameters
Adapt to target playback device Adobe white paper/encoder AME - Baseline for lowest data rate streams, on up to
High
White paper – Main to High
Adapt to lowest common denominator If streaming to Apple devices Baseline – iPhone/iPod touch – through 4G
Main – iPhone 4/iPod touch 4G/iPad
Copyright © 2011 Jan Ozer, All Rights Reserved
WebM-based Adaptive Streaming
Nascent market; Some technology providers Anevia
Quavlive
HTML5 standard coming in a DASH Dynamic Adaptive Streaming over HTTP (DASH)
http://mpeg.chiariglione.org/working_documents.htm
No encoding recommendations at this point
20
Copyright © 2011 Jan Ozer, All Rights Reserved
Transmuxing Technologies
Most producers must serve two clients Flash (or Silverlight) AND
iOS (and now Android)
In the past, that meant two separate encoding and delivery workflows
Now, multiple technologies for: “Transmuxing” H.264 stream
Using correct protocol to distribute to target
Copyright © 2011 Jan Ozer, All Rights Reserved
Transmuxing
Not re-encoding “re-wrapping” file into a different container format
Creating necessary manifest files
Delivering files to proper server
All in real time
21
Copyright © 2011 Jan Ozer, All Rights Reserved
Transmuxing Technologies
Options Technology providers - Wowza, Microsoft, Adobe Service providers - Akamai (in the network repackaging)
Key point: If serving multiple targets, you must produce using lowest
common denominator H.264 encoding parameters
Copyright © 2011 Jan Ozer, All Rights Reserved
New Feature
Wowza Transcoder Single H.264 file in
Re-encode into different adaptive streams
Transmux as necessary
Shipped in the last couple of weeks
Sorenson Squeeze Live
22
Copyright © 2011 Jan Ozer, All Rights Reserved
RTMP Flash Sources
Abhinav Kapoor, “Live dynamic streaming with Flash Media Server 3.5,” (adobe.ly/kapoorlivefms)
Maxim Levkov, “Video encoding and transcoding recommendations for HTTP Dynamic Streaming on the Flash Platform,” (adobe.ly/Levkovhttp)
Larry Bouthillier, “How to do Dynamic Streaming with Flash Media Server,” website (bit.ly/fmsdynamic)
“Tutorial: On-demand HTTP Dynamic Streaming,” Adobe website (bit.ly/ondemanddynamic)
“Encoding Guidelines Dynamic Streaming for Flash over HTTP,” Akamai website (bit.ly/akamaiwhitepaper) .
Copyright © 2011 Jan Ozer, All Rights Reserved
HTTP Flash Resources
Best sources: Maxim Levkov - Adobe - very detailed
recommendations adobe.ly/Levkovhttp
Akamai - White paper:”Encoding Guidelines for Dynamic Streaming for Flash over HTTP” bit.ly/akamaiwhitepaper
23
Copyright © 2011 Jan Ozer, All Rights Reserved
HLS Resources
Sources Apple Tech Note: “Best Practices for Creating and
Deploying HTTP Live Streaming Media for the iPhone and iPad,” (bit.ly/bestpracticehttplive)
Apple Tech Note: “HTTP Live Streaming Overview,”(bit.ly/httpliveoverview)
Apple Tech Note: “Using HTTP Live Streaming,“(bit.ly/usinghttplive)
Copyright © 2011 Jan Ozer, All Rights Reserved
Silverlight Resources
“Akamai HD for Microsoft Silverlight On-Demand Encoding Recommendations,”bit.ly/akamaiwhitepapers
Alex Zambelli, “IIS Smooth Streaming Technical Overview,” bit.ly/smoothstreamzambelli
Zambelli webcast – bit.ly/smtranscoding
24
Copyright © 2011 Jan Ozer, All Rights Reserved
Questions?
For more information, check out the book Available on Amazon
Some copies available today US $29.95 retail
Show special: 10 pounds or Euros