Post on 21-Dec-2015
Source-Channel Prediction in Error Resilient Video Coding
Hua Yang and Kenneth RoseSignal Compression Laboratory
ECE Department
University of California, Santa Barbara
7/8/03 ICME 2003 2
Outline
Introduction
Source-channel prediction
Simulation results
Conclusions
7/8/03 ICME 2003 3
Introduction Existent error resilient approaches on the prediction
mechanism Slice coding
limit prediction within certain non-overlapping spatial regions Video redundancy coding
Multiple independently predicted “threads” Multi-frame motion compensation
Multiple reference frames for prediction
Feature in common: assume the same underlying conventional prediction framework
Framework: separate source-channel coding Prediction: past encoder reconstructed frames Motion estimation criterion: minimum prediction error
7/8/03 ICME 2003 4
Introduction
Via considering packet loss effects during encoding, joint source-channel coding usually achieves better error resilience than that of separate coding.
Our proposed approach Prediction is based on expected decoder reconstruction of the previous
frames. Novelty
Unlike all the other existent error resilient prediction schemes and all the other existent source-channel coding schemes, our proposed method is actually a source-channel prediction scheme.
7/8/03 ICME 2003 5
Introduction0 p1 p2 p3
pi packet loss rate of video packet i.
p1 p2
p2
p3
p3
p3
p3
1-p1 1-p2 1-p3
1-p2
1-p3
1-p3
1-p3
Encoder reconstruction, i.e. “best possible” decoder reconstruction: quantization loss only.
Other possible decoder reconstructions:different transmission loss patterns.
Expected decoder reconstruction:quantization loss & transmission loss.
7/8/03 ICME 2003 6
Introduction Expected decoder reconstruction
Encoder’s estimate of the decoder reconstruction. Given the packet loss rate, it can be accurately computed with the
ROPE method.
Recursive optimal per-pixel estimate (ROPE)
Basic idea: ROPE accurately computes these unknown quantities in a recursive manner for all
the pixels of every frame. Accurate & Low complexity Frequently used to estimate end-to-end distortion in various RD optimization
scenarios. Now we use these expectations for source-channel prediction.
})~
{(}~
{2)(})~
{( 222 in
in
in
in
in
in
in fEfEffffEd
Random variableunknown
7/8/03 ICME 2003 7
Source-channel Prediction
Conventional prediction
Source-channel prediction
Prediction residue
jin
in ff
1ˆ
jin
in fEf
1~
in
in
in ffres
infi
nf Original and predicted values
For pixel i in frame n:
jnf 1
~
jnf 1
ˆ
Encoder and decoder reconstruction values of pixel j in frame n-1 to predict pixel i in frame n.
inres Prediction error to be quantized
7/8/03 ICME 2003 8
Source-channel Prediction Source-channel prediction is the optimal prediction in
the sense of minimum MSE end-to-end distortion.
Pending problem: motion estimation criterion ?
Criterion in the conventional scheme
jinfE 1
~
MBi
mvin
in
mvMBi
in
in
mvffff
2
12 ˆminmin
MBi
mvin
in
mvfEf
2
1 }~
{minCriterion I
Constant value:
Not the actual predictor of the decoder ji
ni
n fEf 1
~Plug in:
7/8/03 ICME 2003 9
Source-channel Prediction
Pending problem: motion estimation criterion? (cont.)
Criterion II
MBi
mvin
in
mvffE }~
{min2
1
Random variable:
Actual predictor of the decoder
Criterion II is superior than Criterion I in that it explicitly accounts for the randomness of the decoder’s actual predictor.
7/8/03 ICME 2003 10
Source-channel Prediction Another interpretation of Criterion II
DR
mv
MBi
mvin
mvin
mvin
in
mv
MBi
mvin
in
mv
DDp
fEfEfEf
ffE
)1(min
}~
{}){(}~
{min
~min
122
12
1
21
MBi
in
inR ffD
2
MBi
in
inD ffED
2~
While Criterion II considers the properly weighted impacts of both DR and DD , in contrast, Criterion I only considers DR . In this sense, Criterion II is more “comprehensive”.
7/8/03 ICME 2003 11
Simulation Results Simulation conditions
H.263+ video codec
System performance: average luminance PSNR 50 different packet loss patterns
Testing scenarios No INTRA Updating
Periodic INTRA Updating
For packet loss rate p, coding a MB in INTRA mode once for every 1/p frames.
R-D optimized INTRA Updating
For each MB, select its coding mode as INTER or INTRA with the R-D criterion.
7/8/03 ICME 2003 12
Foreman (300kbps)
13
18
23
28
33
1 51 101 151 201 251 301 351
Frame No.
PS
NR
(dB
)
EP SCP_CI SCP_CII
Foreman (200kbps)
23
24
25
26
27
28
29
5 10 15 20 25 30
Packet Loss Rate (%)
PS
NR
(d
B)
EP SCP_CI SCP_CII
Foreman (200kbps)
24
25
26
27
28
29
30
31
5 10 15 20 25 30
Packet Loss Rate (%)
PS
NR
(d
B)
EP SCP_CI SCP_CII
(a) No INTRA updating ( p = 10%)
(b) Periodic INTRA updating. (c) RD optimal INTRA updating.
7/8/03 ICME 2003 13
Simulation Results Observations
The proposed “SCP_CII” method consistently offers the best performance, which proves our previous analysis.
When INTRA updating is more effectively performed, smaller gains are achieved by “SCP_CII” over “EP”. Hence, the gain depends on how much damage of packet loss is not accounted for in the conventional scheme.
Similar results also hold for other testing sequences, e.g., carphone, miss_am, salesman, etc.
7/8/03 ICME 2003 14
Demo
Conventional prediction based on encoder reconstruction
(PSNR = 25.06dB)
Foreman, QCIF, 30f/s, 300kb/s, packet loss rate = 10%, periodic Intra update.
Source-channel predictionbased on expected decoder reconstruction
(PSNR = 26.72dB)
7/8/03 ICME 2003 15
Conclusions
Novelty: the proposal of further enhancement of error resilience via fundamental modification of the conventional prediction structure.
Source-channel prediction based on expected decoder reconstruction, which uses ROPE to get accurate estimate of decoder quantities.
In spite of the loss in source coding gain due to the lower source prediction quality, our scheme achieves better overall R-D tradeoff than the conventional scheme.
We identify the subtle points in selecting the motion estimation criterion, and shows that it is advantageous to use the criterion of minimizing the expected prediction error.
7/8/03 ICME 2003 16
Thanks!