Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding Hua Yang and Kenneth...
-
date post
21-Dec-2015 -
Category
Documents
-
view
216 -
download
1
Transcript of Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding Hua Yang and Kenneth...
Rate-Distortion Optimized Motion Estimation for Error Resilient Video Coding
Hua Yang and Kenneth Rose
Signal Compression Lab
ECE Department
University of California Santa Barbara, USA
Mar. 2005
Mar. 2005 ICASSP 2005 2
Outline
Motion estimation (ME) for coding efficiency
– Conventional ME
– Rate-constrained ME & rate-distortion (RD) optimized ME
Motion estimation for error resilience
Proposed end-to-end distortion based RDME
– Intuition behind
– End-to-end distortion analysis
Simulation results
Conclusions
Mar. 2005 ICASSP 2005 3
Motion Estimation for Coding Efficiency
Motion compensated prediction (MCP)– To remove inherent temporal redundancy of video signal – Both the motion vector and the prediction residue are encoded.
Coded frame n-1 Original frame n
Mar. 2005 ICASSP 2005 4
21ˆminmin
MBi
mvin
in
mvres
mvffD
Motion Estimation for Coding Efficiency
Conventional motion estimation
– ME Criterion: minimize prediction residue
• Ignoring the motion vector bit-rate cost
Mar. 2005 ICASSP 2005 5
• However, not yet the ultimate rate-distortion optimization for the best overall coding performance.
Motion Estimation for Coding Efficiency
Motion estimation in low bit rate video coding– In low bit rate video coding, motion vectors may occupy a
significant portion of total bit rate.
– Efficient bit allocation between motion vector and prediction residue coding is necessary for better overall coding efficiency.
mvresmv
RD min : Lagrange multiplier
– Rate-constrained motion estimation
Mar. 2005 ICASSP 2005 6
headerresmvQPmv
RRRD },{
min
Motion Estimation for Coding Efficiency
Motion estimation for low bit rate video coding (cont’d)
– Rate-distortion optimized motion estimation (RDME)
QMBi
in
in
MBi
in
in DeeffD
22ˆˆ
– Some references• [Girod `94] Theoretical analysis of rate-constrained ME
• [Sullivan `98] Summary of rate-constrained ME
• [Chung `96] Low complexity RDME for each MB using RD modeling
• [Schuster `97] Joint RDME for multiple MB’s
Mar. 2005 ICASSP 2005 7
Motion Estimation for Error Resilience
In the presence of packet loss:– Packet loss & error propagation
• Internet – no QoS guarantee
Wireless – inherent error-prone channel
• Error propagation due to MCP
No mv for Inter-mode!
– Error resilient video coding• RD optimization with end-to-end distortion
• Coding mode selection: {Intra/Inter, QP}
Error resilience via motion compensation – Multi-frame motion compensation (MFMC) [Budagavi `01]
– Reference picture selection (RPS) [H.263+]
– Error resilient rate-constrained ME [Wiegand `00]
Not comprehensively attack the RD optimization problem!
Mar. 2005 ICASSP 2005 8
Motion Estimation for Error Resilience
We propose end-to-end distortion based RDME
[accounting for packet loss]
The exact RD optimal ME solution for error resilience
Critical:
accurate pixel-level end-to-end distortion estimation• Build on: recursive optimal per-pixel estimate (ROPE) [R. Zhang, S. Regunathan, and K. Rose `00]
Mar. 2005 ICASSP 2005 9
Conventional motion estimation completely ignores the error resilience information.
– This error resilience information should be exactly considered for each pixel.
Proposed RDME
Intuition for “error resilience via ME”
For coding efficiency For error resilience
I
I
P1
P3
P4
P2
P2
P1
P1
Best trade-off
Mar. 2005 ICASSP 2005 10
– DEP is explicitly affected by mv, whose minimization favors mv’s that point to reference areas with less encoder-decoder mismatch.
Proposed RDME
ROPE-based end-to-end distortion analysis
Error concealment
Error propagated distortion
ECQEP
ECMBi MBi
in
in
mvin
mvin
MBi
in
in
in
mvin
in
MBi
in
in
DpDpDp
DpeeffEp
ffEpeffEp
ffEDE
11
ˆ})~ˆ{(1
})~
{(})ˆ~
{(1
})~
{(}{
2211
21
21
2
MBi
in
in
in
in
MBi
in
in fEfEffffEDE }])
~{(}
~{2)[(})
~{(}{ 222
ROPE
Mar. 2005 ICASSP 2005 11
Proposed RDME
The proposed RDME solution
– Comparing with existent RDME• Source coding distortion end-to-end distortion
• mv affects not only the Rmv vs. Rres trade-off, but also more importantly, the coding efficiency vs. error resilience trade-off.
Packet loss impact
headerresmvQPmv
RRRDE }{min},{
– Comparing with existent RD optimized coding mode selection• Extended Inter mode with the mv parameter
• Further optimize the Inter-mode performance
headerresmvQEP
QPmvRRR
pDD
1min
},{
Mar. 2005 ICASSP 2005 12
Simulations
Objective: to check upper-bound performance– Joint {mv, QP} optimization
– RD calculation via actual encoding
Simulation settings– UBC H.263+
– Encoding: I-P-P-……
– Transmission: independent packet loss, with a uniform p
– Decoding: 50 different packet loss realizations for each p
– Performance: average luminance PSNR
Mar. 2005 ICASSP 2005 13
Simulations
Simulation settings (cont’d)– Testing methods
• Conventional ME (cME)• The proposed RDME (RDME)
– Testing scenarios• Random Intra updating (rI):
arbitrarily assigns MB’s to 1/p groups, and cycles through them updating one group per frame.
• Optimal Intra updating (oI):
RD optimized Intra/Inter mode selection.
Mar. 2005 ICASSP 2005 14
Simulation Results Random Intra
PSNR vs. Packet loss rate [QCIF, 10f/s, 48kb/s]
0 5 10 15 20 25 3030
31
32
33
34
35
36
37
38
39
40
Packet los s rate (%)
PS
NR
(dB
)
cMERDME
0 5 10 15 20 25 3021
22
23
24
25
26
27
28
29
30
Packet los s rate (%)P
SN
R (
dB)
cMERDME
Miss_am Foreman
Mar. 2005 ICASSP 2005 15
Simulation Results Optimal Intra
0 5 10 15 20 25 3031
32
33
34
35
36
37
38
39
40
Packet los s rate (%)
PS
NR
(dB
)
RDME-rIcME-oIRDME-oI
0 5 10 15 20 25 3021
22
23
24
25
26
27
28
29
30
Packet los s rate (%)P
SN
R (
dB)
RDME-rIcME-oIRDME-oI
PSNR vs. Packet loss rate [QCIF, 10f/s, 48kb/s]
Miss_am Foreman
Mar. 2005 ICASSP 2005 16
Simulation Results Random Intra
30 40 50 60 70 80 90 100 110 12031
32
33
34
35
36
37
38
Total bit rate (kb/s )
PS
NR
(dB
)
cMERDME
30 40 50 60 70 80 90 100 110 12021.5
22
22.5
23
23.5
24
24.5
25
25.5
26
26.5
Total bit rate (kb/s )P
SN
R (
dB)
cMERDME
PSNR vs. Total bit rate [QCIF, 10f/s, p=10%]
Miss_am Foreman
Mar. 2005 ICASSP 2005 17
Simulation Results Optimal Intra
30 40 50 60 70 80 90 100 110 12033.5
34
34.5
35
35.5
36
36.5
37
37.5
38
Total bit rate (kb/s )
PS
NR
(dB
)
RDME-rIcME-oIRDME-oI
30 40 50 60 70 80 90 100 110 12023
23.5
24
24.5
25
25.5
26
26.5
27
27.5
Total bit rate (kb/s )P
SN
R (
dB)
RDME-rIcME-oIRDME-oI
PSNR vs. Total bit rate [QCIF, 10f/s, p=10%]
Miss_am Foreman
Mar. 2005 ICASSP 2005 18
Simulation Results
Miss_am: QCIF, 10f/s, 48kb/s, p=10%, random Intra
Conventional ME [29.58dB]
RDME[33.83dB]
Mar. 2005 ICASSP 2005 19
Simulation Results
Foreman: 1st 200f, QCIF, 10f/s, 112kb/s, p=10%, random Intra
Conventional ME[23.92dB]
RDME[26.92dB]
Mar. 2005 ICASSP 2005 20
Besides Intra updating, RDME presents another good alternative for error resilience.
Conclusions
Identify the new opportunity of achieving error resilience via motion estimation.
Propose an RD optimal ME solution, which further optimizes the Inter-mode performance.
Investigate the upper-bound performance.– With random Intra: substantial gain– With optimal Intra: significant gain at low bit rates.
Mar. 2005 ICASSP 2005 21
Conclusions
Future work I: more comprehensive tests– Inaccurate p, bursty loss, or over actual networks, etc.
Future work II: complexity reduction– RD modeling, separate mv and QP optimization,
sophisticated ME strategies, etc.
Originally, the power of Intra coded MB’s is only recognized as stopping past error propagation, while the proposed RDME reveals their new potential on reducing future error propagation.
Mar. 2005 ICASSP 2005 22
References
[Girod `94] B. Girod, ``Rate-constrained motion estimation,'' Nov. 1994. [Sullivan `98] G. J. Sullivan and T. Wiegand, ``Rate-distortion optimization for video
compression,’’ Nov. 1998. [Chung `96] W. C. Chung, F. Kossentini, and M. J. T. Smith, ``An efficient motion
estimation technique based on a rate-distortion criterion,'' May 1996. [Schuster `97] G. M. Schuster and A. K. Katsaggeslos, ``A theory for the optimal bit
allocation between displacement vector field and displaced frame difference,'' Dec. 1997.
[Budagavi `01] M. Budagavi and J. D. Gibson, ``Multiframe video coding for improved performance over wireless channels,'' Feb. 2001.
[H.263+] ITU-T, Rec. H,263, ``Video codeing for low bitrate communications'', version 2 (H.263+), Jan. 1998.
[Wiegand `00] T. Wiegand, N. Farber, K. Stuhlmuller and B. Girod, ``Error-resilient video transmission using long-term memory motion-compensated prediction,'' June 2000.
[Zhang `00] R. Zhang, S. L. Regunathan, and K. Rose, ``Video coding with optimal intra/inter mode switching for packet loss resilience,'' June 2000.
Mar. 2005 ICASSP 2005 23
The End