Current Work on Video Coding Raj Kumar. Overview Fine-Grained Scalability (FGS) What is it and why...
-
date post
21-Dec-2015 -
Category
Documents
-
view
212 -
download
0
Transcript of Current Work on Video Coding Raj Kumar. Overview Fine-Grained Scalability (FGS) What is it and why...
Overview Fine-Grained Scalability (FGS)
What is it and why do we need it What are its drawbacks
Our new scheme (FGS+) B-Frame only scheme All-Frame Scheme
Future Directions
Fine Grained Scalability What is it What are the pros and cons
I PB B B Base Layer
Enhancement Layer
Problem with Temporal FGS Reference Frames are poor quality Doesn’t use motion-prediction well Compression ratio suffers
compared to traditional encoders (approx 2-2.5 DB)
Insight FGS provides two
degrees of freedom SNR Temporal
Is there an optimal path that will maximize overall quality. How do we
determine it How do we take
advantage of it
Equal-bandwidth lines
Non-optimal paths Max overall quality
Equal-quality lines
(A) The Video SNR-Temporal Plane
SNR bandwidth
Temporal bandwidth
Idea Can we use a part of the enhancement
layer, in addition to the base layer, to predict motion?
We can if we know how much of the base layer will be present at the decoder when the temporal layers are introduced
So if we build in rules about when temporal layers are introduced relative to spatial quality, we can!
Implementation Increase SNR
quality to a predetermined point
Then improve temporal quality
Further improve SNR quality at the new frame-rate
And so on….
Enhanced-reference scheme
Scheme using fixed reference rate
(B) SNR quality of enhanced-scheme
Bandwidth
SNR dB 2T fps
T fps
4T fps
3T fps
Two issues When (at what SNR quality) do we
introduce new temporal-frames? How much extended-reference do
we use
Introducing Temporal Frames We conducted a
study where users chose preferred frame-rate at different bit-rates:
As the bit-rate goes up, people prefer better frame-rates
High-motion videos (Stefan, Coastguard) require a quicker transition
0102030405060708090
100
0 5
200300500
Spatio-Temporal Preferences
0
2
4
6
8
10
12
0 500 1000 1500
Bitrate (kbps)
Fram
e-R
ate
(fps) Foreman
Stefan
Coastguard
Mobile
Determining the size of the reference
Plot SNR performance as a function of enhancement at various bit-rates
Choose enhancement corresponding to best quality
SNR as function of enhancement
27
29
31
33
35
37
39
0 1 2 3 4 5 6bitplanes
SN
R (
dB
)
300 kbps 500 kbps1000 kbps 1500 kbps
Performance Improvement over
FGS varies from 0.19 dB to 1.28 dB
At low bit-rates simple videos benefit (Coastguard)
At high bit-rates complex videos benefit (Mobile)
Improvement of FGS+ (B-frames) over FGS
0
0.2
0.4
0.6
0.8
1
1.2
1.4
300 800 1300kbps
SN
R (
dB)
Foreman StefanCoastguard Mobile
FGS+ for All-Frames We can improve
performance further by using enhancement for base-layer P-frames also
But P-frames are present at all bit-rates. How much enhancement do we use?
(B) Effect of Reference Bandwidth T on Performance
T
SNR (dB)
P(r) = probability(R)
Transmission bandwidth R
Original Compression
New Compression Negative effect
of choice of T
Positive effect of choice of T
Improvement in Performance At 3 bit-planes Performance
varies from –0.03 dB to 0.59 dB
Performance will degrade at low bit-rates, due to incomplete references
Improvement of All-frames FGS+ over B-Frames only FGS+
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
300 800 1300
SNR (dB)
kbps
Foreman StefanCoastguard Mobile
Conclusion FGS+ uses a spatio-temporal
notion of video-quality Improves performance of
traditional FGS from 0.16 to 1.61 dB
Current Issues Spatio-Temporal
preference curves vary for different videos
How do we choose the correct curve automatically
Use Motion-Vectors, Image-Complexity
0102030405060708090
100
0 5
200300500
Spatio-Temporal Preferences
0
2
4
6
8
10
12
0 500 1000 1500
Bitrate (kbps)
Fra
me-
Rat
e (f
ps) Foreman
Stefan
Coastguard
Mobile
Future Directions Factors other than video-characteristics
do affect spatio-temporal preferences Environment (resolution, display-type,
lighting) User preferences (moods, some prefer
spatial-quality always) How do we account for these other
dimensions ? Large dimensional space requires many
(thousands) of sample points