2015/6/15VLC 2006 PART 1 Introduction on Video Coding StandardsVLC 2006 PART 1 Variable Length...

VLC 2006 PART 1112/04/18 VLC 2006 PART 1 <1>Introduction on Video Coding Standards

Variable Length Coding Information entropy

Huffman code vs. arithmetic code Arithmetic coding Why CABAC? Rescaling and integer arithmetic coding Golomb codes Binary arithmetic coding CABAC


Information Entropy Information entropy: Claude E. ShannonClaude E. Shannon 1948, “A Mathematical

Theory of Communication”

The information contained in a statement asserting the occurrence of an event depends on the probability p(f), of occurrence of the event

f. - lg p(f)

The unit of the above information quantity is referred as a bit, since it is the amount of information carried by one (equally likely) binary digit.

Entropy H is a measure of uncertainty or information content

- Very uncertain high information content

p

AffpfpH )(lg)(


Entropy Rate Conditional entropy H(F|G) between F and G: uncertainty of F given

G

Nth order entropy

Mth order conditional entropy

Entropy rate (lossless coding bound)

gAg G gFHgpGFH )|()()|(

),...,()( 1 NN FFHFH

NCN

NN

HHN

H ,lim1

lim

),...,|()( 11, FFFHFH MMNC


Bound for Lossless Coding Scalar coding: could differ from the entropy by up to 1 bit/symbol

Vector (block) coding: assign one codeword for each group of N symbols

Conditional coding (predictive coding, context-based coding): The codeword of the current symbol depends on the pattern (context) formed by the previous M symbol

1111 HRH

HR

NNHRNH

HRH

NN

NNN

NNN

lim

/1//

1

1lim ,

HRH MCM


Huffman Coding• Huffman coding for pdf: (a1, a2, a3) = (0.5, 0.25, 0.25)

– –lg0.5 = 1, –lg 0.25 = 2

– a1 = 0, a2 = 10, a3 = 11

• If the self information is not integer?– pdf: (a1, a2, a3, a4) = (0.6, 0.2, 0.125, 0.075)– –lg 0.6 = 0.737, –lg 0.2 = 2.32,

–lg 0.125 = 3, –lg 0.075 = 3.74

– a1 = 0, a2 = 10, a3 = 110, a4 = 111

a1 = 0.5

a2 = 0.25

a3 = 0.25

0

10

1

a1 = 0.6

a2 = 0.2

a3 = 0.125

a4 = 0.075

0

1

0

1

0

1


Huffman vs. Arithmetic Coding

Huffman coding: convert a fixed number of symbols into a variable length codeword

Efficiency

The usage of fixed VLC tables does not allow an adaptation to the actual symbol statistics.

Arithmetic Coding: convert a variable number of symbols into a variable length codeword

Efficiency

Process one symbol at a time

Easy to adapt to changes in source statistics

Integer implementation is available

NNHRNH NN /1//

NNHRNH NN /2//


Arithmetic Coding• The bits allocated for each symbol can be non-integer

– If pdf(a) = 0.6, then the bits to encode ‘a’ is 0.737

• For the optimal pdf, the coding efficiency is always better than or equal to the Huffman coding

• Huffman coding for a2 a1 a4 a1 a1 a3, total 11 bits:

• Arithmetic coding for a2 a1 a4 a1 a1 a3, total 11.271 bits:

2 + 1 + 3 + 1 + 1 + 3

2.32 +0.737+ 3.74 +0.737+0.737+ 3The exact probs are preserved


Arithmetic Coding• Basic idea:

– Represent a sequence of symbols by an interval with length equal to its probability

– The interval is specified by its lower boundary (l), upper boundary (u) and length d (= probability)

– The codeword for the sequence is the common bits in binary representations of l and u

• The interval is calculated sequentially starting from the first symbol– The initial interval is determined by

the first symbol– The next interval is a subinterval of

the previous one, determined by the next symbol

)()(

)1()(

symbol observedth the:

)(

)(

; ;

,

1 ,1 ,0

symbolth the toupy probabilit cumulative the,

)(

..., ,2 ,1 ,

)1()1()1()(

)1()1()1()(

111

1111

1111

000

1

nXnnnn

nXnnnn

n

lnnnn

lnnnn

nnnlnnnlnn

lth

l

kll

ll

l

xFlulu

xFlull

nx

qlulu

qlull

dluqdllpdd

a symbolthe nreceiving On

dandul

lpq

app

Lla

)( 11

1111

lln

lnlnnn

pqd

pdqdlu


An Example Any binary value between l and u canunambiguously specify the input message.

½=(10…)=(01…1…)

¼ =(010…)=(001…1…)

d(ab)=1/8


Why CABAC?

The first standard that uses arithmetic entropy coder is given by Annex E of H.263

Drawbacks:

1. Annex E is applied to the same syntax elements as the VLC elements of H.263

2. All the probability models are non-adaptive that their underlying probability as assumed to be static.

3. The generic m-ary arithmetic coder used involves a considerable amount of computational complexity.


CABAC: Technical OverviewCABAC: Technical Overview

Context modeling

Binarization Probability estimation

Coding engine

update probability estimation

Adaptive binary arithmetic coder

Chooses a model conditioned on past

observations

Maps non-binary symbols to a binary

sequence

Uses the provided model for the actual encodingand updates the model


CABAC


Context-based Adaptive Binary Arithmetic Code(CABAC)

Usage of adaptive probability models

Exploiting symbol correlations by using contexts Non-integer number of bits per symbol by using

arithmetic codes Restriction to binary arithmetic coding

• Simple and fast adaptation mechanism• But: Binarization is needed for non-binary symbols• Binarization enables partitioning of state space


Implementation of Arithmetic Coding

Rescaling and Incremental coding Integer arithmetic coding Binary arithmetic coding

Hoffman Trees

Exp-Golomb Codes


Issues Finite precision (underflow & overflow): As n gets larger, these

two values, l(n) and u(n) come closer and closer together. This means that in order to represent all the subintervals uniquely we need to increase the precision as the length of the sequence increases.

Incremental transmission: transmit portions of the code as the sequence is being observed.

Integer implementation


Rescaling & Incremental Coding


Incremental Encoding

U

L

L

L

U

U


Question for Decoding How do we start decoding? decode the first symbol

unambiguously

How do we continue decoding? mimic the encoder

How do we stop decoding?


Incremental Decoding

Top 18% of [0,0.8)

U

0.80.82

1.0


Issues in the Incremental Coding


Solution


Solution (2)


Incremental Encoding


Incremental Decoding


Integer Implementation


Integer Implementation • nj: the # of times the symbol j

occurs in a sequence of length Total Count.

• FX(k) can be estimated by• Define

we have

• E3: if (E3 holds)Shift l to the left by 1 and shift 0 into LSBShift u to the left by 1 and shift 0 into LSBComplement (new) MSB of l and uIncrement Scale3

TotalCount

nkF

k

i iX

1)(

k

i inCountCum1

_

1)(_)1(

)1(_)1(

)1()1()1()(

)1()1()1()(

TotalCount

xCountCumlulu

TotalCount

xCountCumlull

nnn

nn

nnn

nn


Golomb Codes Golomb-Rice code: a family of codes designed to encode integers with

the assumption that the larger an integer, the lower its probability of occurrence.

An example (the simplest, unary code): for integer n, codes as n 1s followed by a 0. This code is the same as the Huffman code for {1, 2, …} with probability model

Golomb code with m: code n > 0 using two numbers q and r: Q is coded by unary code of q; r is represented by binary code using

bits. the first – m values, uses bits the rest values: uses bits

Golomb code for m = 5:

.2

1][

kkp

qmnrm

nq

mlg

mlg2 mlg

mr m lg2

n q r code n q r code

0 0 0 000 5 1 0 1000

1 0 1 001 6 1 1 1001

2 0 2 010 7 1 2 1010

3 0 3 0110 8 1 3 10110

4 0 4 0111 9 1 4 10111

3+3=110


Golomb Codes Golomb code is optimal for the probability model

exp-Golomb code: variable length codes with regular construction: [m zeros] [1] [info].

code_num: index

info: is an m-bit field carrying information Mapping types: ue, te, se, and me

designed to produce short codewords for frequently-occurring values and longer codewords for less common parameter values.

.lg

1

1 ,)( 1

pm

pqqpnP n


exp-Golomb Codes exp-Golomb code

Decode:

1. Read in m leading zeros followed by 1.2. Read m -bit info field.3.

mcode_num

numcodem

21info

)1_lg(

﹝m zeros 1 info﹞﹝ ﹞﹝ ﹞

1info2code_num m


exp-Golomb Entropy Coding

A parameter k to be encoded is mapped to code_num in one of the following ways:


exp-Golomb Entropy Coding


H.264 Coding Parameters

2015/6/15VLC 2006 PART 1 Introduction on Video Coding StandardsVLC 2006 PART 1 Variable Length...

Documents

Transcript of 2015/6/15VLC 2006 PART 1 Introduction on Video Coding StandardsVLC 2006 PART 1 Variable Length...