Scalable Video Coding Dept Talk Ankit Bhurane

Scalable Video Coding: An Overview

Ankit A. Bhurane

TIDSP, EE, IITB

Why Video?

• Annual global IP traffic will surpass 1.3 zettabytes/ month threshold by the end of 2016!

• 1, 000, 000, 000, 000, 000, 000, 000 bytes!

• Who is responsible?

…

• Complete list at http://en.wikipedia.org/wiki/List_of_video_hosting_services

Forecast by CISCO: http://www.cisco.com/

April 3, 2013 Department Talk | Ankit Bhurane | TIDSP-IITB 2

http://en.wikipedia.org/wiki/List_of_video_hosting_services

http://en.wikipedia.org/wiki/List_of_video_hosting_services

http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-481360_ns827_Networking_Solutions_White_Paper.html



Raw Images

• Store data from an image sensor

• Typical image sensors capture only red, green, or blue for a pixel.


Courtesy: Adobe Systems

http://news.cnet.com/8301-30685_3-57371809-264/adobe-offering-new-reasons-to-get-dng-religion/

Video?

… …

Source: http://ivpl.eecs.northwestern.edu/files/general/Public_Files587/MC_Frame_Rate_Upconversion/frame_motion.png


n-1 n n+1

http://ivpl.eecs.northwestern.edu/files/general/Public_Files587/MC_Frame_Rate_Upconversion/frame_motion.png



How Hard can it be?

• Size: HD: 1280×720;

• Color

• Frame rate: 30 fps

• 1280×720×3×30×8 bits/sec

• 663.552000 Mbits/sec!


Raw Video Specs

• Specification:

– Spatial Resolution (aspect ratio)– 4:3, 16:9, …

– Color space: RGB, YUV, …

– Framerate: 30 fps, 50 fps, 60 fps

– ?


Resoltion Chart: Source: Wikipedia

http://upload.wikimedia.org/wikipedia/en/9/9b/Vector_Video_Standards2.png


http://upload.wikimedia.org/wikipedia/en/9/9b/Vector_Video_Standards2.png

Frame rate


Color Space


RGB to YUV


RGB YUV

𝑌𝑈𝑉

=0.299 0.587 0.114−0.147 −0.289 0.4360.615 −0.515 −0.100

𝑅𝐺𝐵

How to achieve compression?

• Take advantage of human visual system (HVS)!

• Remove redundancy.


Human Eye Sensitivity


Shamelessly Taken from: Slides by Yao Wang, Polytechnic University, Brooklyn :P

Courtesy: Yao Wang

Croma Sampling


Shamelessly Taken from: Slides by Yao Wang, Polytechnic University, Brooklyn :P

Courtesy: Yao Wang

Most video is 4:2:0


Taken from: http://people.xiph.org/~tterribe/pubs/lca2008/anatomy.pdf

http://people.xiph.org/~tterribe/pubs/lca2008/anatomy.pdf


Spot the difference

Department Talk | Ankit Bhurane | TIDSP-IITB April 3, 2013 15

4:4:4 4:2:2

Sequences available at: http://samples.mplayerhq.hu/yuv/

4:2:2 requires 2/3rd the bandwidth of 4:4:4!

http://samples.mplayerhq.hu/yuv/

http://samples.mplayerhq.hu/yuv/


• Take advantage of human visual system!



Spatial Redudndancy: Correlated data


Transform


Taken from: http://people.xiph.org/~tterribe/pubs/lca2008/anatomy.pdf http://www.vcodex.com/files/videocoding_intro_1.pdf



http://www.vcodex.com/files/videocoding_intro_1.pdf



Spatial-Temporal redundancy

n n+1


MEMC


Taken from: John G. Apostolopoulos, Image and Video Compression, MIT 6.344, Spring 2002

Motion

Motion Estimation and

Compensation


Taken from: Thyagarajan, Still Image and Video Compression.


Chapter11_MultipleReferenceFramesME.swf

MEMC-Example



• Take advantage of human visual system!



Campbell-Robson

Contrast Sensitivity Chart


Sinusoidally modulated Luminance

Co

ntr

ast

fro

m 1

00

% t

o a

bo

ut

0.5

%

http://neurovision.berkeley.edu/Demonstrations/VSOC/izumi/CSF/A_What_is_CSF.html

http://neurovision.berkeley.edu/Demonstrations/VSOC/izumi/CSF/A_What_is_CSF.html

Quantization

• The only lossy step!

• Broadly categorized:

– Scalar

– Vector


Core of Data Compression

Transform

• Removes spatial redundancy

Quantize

• Removes insignificant data

Entropy coding

• Removes statistical redundancy


Core of Video Compression


MEMC

• Removes temporal redundancy

Transform

• Removes spatial redundancy

Quantize

• Removes irrelevant data

Entropy coding

• Removes statistical redundancy

Transform

• Goal: A sparse representation of the pixel data.

– By removing redundancy of highly correlated data.

• Optimal transform: Karhunen-Loève Transform (KLT).

– Data-dependent

– Need to transmit eigen vectors

• More appropriate: DCT


Transform


Taken from: http://people.xiph.org/~tterribe/pubs/lca2008/anatomy.pdf http://www.vcodex.com/files/videocoding_intro_1.pdf





Entropy Coding

• Inputs:

– transform coefficients

– motion vectors and

– ‘side’ information (headers, synchronization markers, etc.)

• Techniques:

– Huffman coding

– Arithmetic coding


Entropy Coding

• Huffman coding

– Optimal– but only as symbol codes

– Needs integer-valued codeword length

• Arithmetic coding

– Codes group/ block of symbols

– Near-optimal for long data


error.swf

How it appears?


Taken from: http://www.w3.org/2008/01/video-container.png http://cdn.reelstatic.com/wp-content/uploads/encoding-video.png

http://www.w3.org/2008/01/video-container.png




http://cdn.reelstatic.com/wp-content/uploads/encoding-video.png






How it comes to us?


Internet

www.youtube.com 208.65.153.251

Server

Internet Service Provider

http://www.youtube.com/

How does it look in Market?



Todays Need


Encode: Once at highest resolution, Decode: Different rates depending on application

Single bit stream adaptable to heterogeneous end users!

Scalability, What’s that?


Scalable Bitstream

Courtesy: Nikola Adami et al., “State-of-the art and trends in Scalable video compression with wavelet-based approaches”


SVC System



Scalable Bitstream



Spatial Scalability


Temporal Scalability


Quality Scalability


Performance


SVC demo

Courtesy: Georgia Tech.

Scalable Video Coding Dept Talk Ankit Bhurane

Documents

Transcript of Scalable Video Coding Dept Talk Ankit Bhurane