Optimum Bit Optimum Bit Allocation and Allocation and

Rate Control for Rate Control for H.264/AVCH.264/AVC

Wu Yuan, Shouxun Lin, YongdWu Yuan, Shouxun Lin, Yongdong Zhang, Wen Yuan, and Haong Zhang, Wen Yuan, and Ha

iyong Luoiyong Luo

CSVT 2006

OutlineOutline• Introduction• Rate-Distortion Modeling• Rate Distortion Optimization• Macroblock-Layer Rate Control• Experimental Results

– Comparisons with JVT-G012

Introduction (1)Introduction (1)• H.264/AVC Rate Controller

http://www.pixeltools.com/rate_control_paper.html#bas

Intra/Interprediction

Residual

Introduction (2)Introduction (2)• The Chicken and Egg Dilemma

Rate Control

QP

RDO

MAD

Coding

(Intra/Inter mode selection)

!?

(Residual calculation)

Solution: Guess!MAD = MADprev +

MADprev

MAD

Rate-Distortion Modeling (1)Rate-Distortion Modeling (1)• Overhead Bit-Rate Prediction

– Overhead bits: QP, MV, MB mode, …– Using history

• Coding Complexity Prediction– Complexity: Residual (MAD)– Using history

• R-D Behavior Prediction– Using overhead bits, residual, and initial QP

• Distortion Prediction– Using history

Residual(org - pred)

Overhead(MV, QP, …)

Encoded frame

Entropycoding

Rate-Distortion Modeling (2)Rate-Distortion Modeling (2)• Overhead Bit-Rate Prediction

– JVT-G012: Hi = Hiave

– Proposed: Hi = Hiprev

n-1 n

Rate-Distortion Modeling (3)Rate-Distortion Modeling (3)• Coding Complexity Prediction

– MAD = MADprev + • Data points are first selected by the spatial and t

emporal distance, and then removing outliers.

%100MAD

|MAD-MAD|

pred

realpred

JVT-G012

n-1 n

RPE < 3%

QP JVT-G012 Proposed

32 20.58% 23.05%

34 47.98% 58.12%

Rate-Distortion Modeling (4)Rate-Distortion Modeling (4)• R-D Behavior Prediction

– Assumption: DCT coefficients of residual can be approximated by

– Distortion

xe

xP

x

where,2

)(

||

(Laplace distribution)

||),( xxxxD D

DDR 0 where,)ln()(

* A. Viterbi and J. Omura, Principles of Digital Comuunicatin and Coding. New York: McGraw-Hill Electrical Engineering Series,1979

*

)()1(2

1)1()( 3

2 QstepRQstepQstep

QstepR

2

2MADMAD)(

Qstep

b

Qstep

aQstepR

Taylor expansion

Residual rate

Let /D = 1+x

Rate-Distortion Modeling (5)Rate-Distortion Modeling (5)• R-D Behavior Prediction

– Taylor expansion of R(Qstep) at Qstepave

– – –

• Rate of the ith MB:

QPmax

QPaveQPmin

)()(2

)("

)()(')()(~

ave32

aveave

aveaveave

QstepQstepRQstepQstepQstepR

QstepQstepQstepRQstepRQstepR

CQstepBQstepAQstepR 2)(

previiiiiii HCQstepBQstepAR 2

2

)(" aveQstepRA

aveaveave )(")(' QstepQstepRQstepRB

ave2aveave)( QstepBQstepAQstepRC

(Rate of residual)

Rate of residual Rate of overhead

Rate-Distortion Modeling (6)Rate-Distortion Modeling (6)• Distortion Prediction

– Assumption 1: Distortion of DCT coefficients is uniformly distributed

• D = 2 = |x-y|2/12 (Qstep2/12)

– Assumption 2: Qstepi Qstepiprev

• Qstepi (Diprev/Qstepi

prev)

• Di Qstepi2 (Qstepi Di

prev/Qstepiprev)

Di = iQstepi, where i = Diprev/Qstepi

prev

Set as 1

pred

real

D

D

(For scalability)

Rate Distortion OptimizationRate Distortion Optimization

• Consider Di = iQstepi for the ith MB

– subject to

N

iii

QstepQstepQstep

NN 1,...,

1min1

TTN

ii

1

][1

min11

,,...,1

N

ii

N

iii

QstepQstepTTQstep

NN

])([1

min1

prev2

1,,...,1

N

iiiiiii

N

iii

QstepQstepTHCQstepBQstepAQstep

NN

N

ki

k

N

k

N

k kk

kN

k k

i

i

i

ii

A

CA

BHT

AA

BQstep

1

2

1 1

2

1

prev

* 4

2

(Lagrange theory)

Rate of the ith MB

Macroblock-Layer Rate Macroblock-Layer Rate Control (1)Control (1)

Macroblock-Layer Rate ControMacroblock-Layer Rate Control (2)l (2)

Macroblock-Layer Rate ControMacroblock-Layer Rate Control (3)l (3)

• Rate Controller1. Initialization: QPave = QPstart, QPmin = max(QPave-3, 0), and QPmax = min(Q

Pave+3, 51). Let i = 02. Optimum Bit Allocation for ith MB:

1) k = i.2) R-D modeling:

3) Optimum Computations:

4) k = k+1. If k N, jump to 2).5) Compute optimal QPi

*:

prevMADMAD kk

2max

2

max

kmax

MADMAD)(

Qstep

b

Qstep

aQstepR kkk

kH

N

ki

k

N

k

N

k kk

kN

k k

i

i

i

ii

A

CA

BHT

AA

BQstep

1

2

1 1

2

1

prev

* 4

2

2

)(" aveQstepRAk aveaveave )(")(' QstepQstepRQstepRBk

ave2aveave)( QstepBQstepAQstepRC kkk kkk CQstepBQstepAQstepR 2)(

(overhead bits)

(check if > T when adding with Hk)

2max

2

maxmax

MADMAD)(

Qstep

b

Qstep

aQstepR kkkk

(by linear regression)

(by linear regression)

Macroblock-Layer Rate ControMacroblock-Layer Rate Control (4)l (4)

• Rate Controller3. Adjust QPi

*: QPi* = max{QPi-1

*-1, min{QPi*,QPi-1

*+1}}. Then QPi*

= max{1,QPave-3, min{51,QPave+3,QPi*}}

4. Encoding5. Update

(Reducing blocking effect)

(For smoothness)

Experimental Results (1)Experimental Results (1)

CIF: Mobile, Paris

%100rateTarget

rate RealRate Prediction Error Ratio:

Experimental Results (2)Experimental Results (2)

Experimental Results (3)Experimental Results (3)