05-Quantization

Bernd Girod: EE398A Image and Video Compression Quantization no. 1

Quantization

Q q

Q -1

Sometimes, this

convention is used:

M representative levels

Q

Input-output characteristic of a scalar quantizer

M-1 decision thresholds

input signal x

Output

t q+1 t q+2 t q

x x

x2

qx

1

qx

qx

x

q x


Example of a quantized waveform

Original and Quantized Signal

Quantization Error


Lloyd-Max scalar quantizer

Problem : For a signal x with given PDF find a quantizer with

M representative levels such that

( )Xf x

Solution : Lloyd-Max quantizer

[Lloyd,1957] [Max,1960]

M-1 decision thresholds exactly half-way between representative

levels.

M representative levels in the centroid of the PDF between two

successive decision thresholds.

Necessary (but not sufficient)

conditions

1

1

1

1 1, 2, , -1

2

( )

0,1, , -1

( )

q

q

q

q

q q q

t

X

t

q t

X

t

t x x q M

x f x dx

x q M

f x dx

2

min.d MSE E X X


Iterative Lloyd-Max quantizer design

1. Guess initial set of representative levels

2. Calculate decision thresholds

3. Calculate new representative levels

4. Repeat 2. and 3. until no further distortion reduction

0,1,2, , -1qx q M

1

1

( )

0,1, , -1

( )

q

q

q

q

t

X

t

q t

X

t

x f x dx

x q M

f x dx

11

1,2, , -12

q q qt x x q M


Example of use of the Lloyd algorithm (I)

X zero-mean, unit-variance Gaussian r.v.

Design scalar quantizer with 4 quantization indices with

minimum expected distortion D*

Optimum quantizer, obtained with the Lloyd algorithm Decision thresholds -0.98, 0, 0.98

Representative levels 1.51, -0.45, 0.45, 1.51

D*=0.12=9.30 dB

Boundary

Reconstruction


Example of use of the Lloyd algorithm (II)

Convergence

Initial quantizer A:

decision thresholds 3, 0 3 Initial quantizer B:

decision thresholds , 0,

After 6 iterations, in both cases, (D-D*)/D*


Example of use of the Lloyd algorithm (III)

X zero-mean, unit-variance Laplacian r.v.

Design scalar quantizer with 4 quantization indices with

minimum expected distortion D*

Optimum quantizer, obtained with the Lloyd algorithm Decision thresholds -1.13, 0, 1.13

Representative levels -1.83, -0.42, 0.42, 1.83

D*=0.18=7.54 dB

Threshold

Representative


Example of use of the Lloyd algorithm (IV)

Convergence

Initial quantizer A,

decision thresholds 3, 0 3 Initial quantizer B,

decision thresholds , 0,

After 6 iterations, in both cases, (D-D*)/D*


Lloyd algorithm with training data


2. Assign each sample xi in training set T to closest representative

3. Calculate new representative levels

4. Repeat 2. and 3. until no further distortion reduction

; 0,1,2, , -1qx q M

x

1 0,1, , -1

q

q

Bq

x x q MB

qx

B

q x T :Q x q q 0,1,2,, M -1


Lloyd-Max quantizer properties

Zero-mean quantization error

Quantization error and reconstruction decorrelated

Variance subtraction property

0E X X

0E X X X

2

2 2

XXE X X


High rate approximation

Approximate solution of the "Max quantization problem,"

assuming high rate and smooth PDF [Panter, Dite, 1951]

Approximation for the quantization error variance:

3

1( )

( )Xx x const

f x

Distance between two

successive quantizer

representative levels

Probability density

function of x

3

23

2

1 ( )12

X

x

d E X X f x dxM

Number of representative levels


High rate approximation (cont.)

High-rate distortion-rate function for scalar Lloyd-Max quantizer

Some example values for

2 2 2

3

2 2 3

2

1with ( )

12

R

X

X X

x

d R

f x dx

2

92

uniform 1

Laplacian 4.5

3Gaussian 2.721

2


High rate approximation (cont.)

Partial distortion theorem: each interval makes an (approximately)

equal contribution to overall mean-squared error

2

1 1

2

1 1

Pr

Pr for all ,

i i i i

j j j j

t X t E X X t X t

t X t E X X t X t i j

[Panter, Dite, 1951], [Fejes Toth, 1959], [Gersho, 1979]


Entropy-constrained scalar quantizer

Lloyd-Max quantizer optimum for fixed-rate encoding, how can we

do better for variable-length encoding of quantizer index?

Problem : For a signal x with given pdf find a quantizer with rate

such that

Solution: Lagrangian cost function

2

min.d MSE E X X

( )Xf x

1

2

0

logM

q q

q

R H X p p

2

min.J d R E X X H X


Iterative entropy-constrained scalar quantizer design


and corresponding probabilities pq

2. Calculate M-1 decision thresholds

3. Calculate M new representative levels and probabilities pq

4. Repeat 2. and 3. until no further reduction in Lagrangian cost

; 0,1,2, , -1qx q M

1

1

( )

0,1, , -1

( )

q

q

q

q

t

X

t

q t

X

t

xf x dx

x q M

f x dx

-1 2 1 2

-1

log log= 1,2, , -1

2 2

q q q q

q

q q

x x p pt q M

x x


Lloyd algorithm for entropy-constrained

quantizer design based on training set


and corresponding probabilities pq

2. Assign each sample xi in training set T to representative minimizing Lagrangian cost

3. Calculate new representative levels and probabilities pq

4. Repeat 2. and 3. until no further reduction in overall Lagrangian cost

: 0,1,2, , -1qB x Q x q q M T

; 0,1,2, , -1qx q M

x

1 0,1, , -1

q

q

Bq

x x q MB

qx

2

2 log

ix i q qJ q x x p

2

2logi ii i

x i i q xx x

J J x Q x p


Example of the EC Lloyd algorithm (I)

X zero-mean, unit-variance Gaussian r.v.

Design entropy-constrained scalar quantizer with rate R2 bits, and

minimum distortion D*

Optimum quantizer, obtained with the entropy-constrained Lloyd

algorithm 11 intervals (in [-6,6]), almost uniform

D*=0.09=10.53 dB, R=2.0035 bits (compare to fixed-length example)

Threshold

Representative level


Example of the EC Lloyd algorithm (II)

Initial quantizer B, only 4 intervals

(11) in [-6,6], with the same length

5 10 15 20-6

-4

-2

0

2

4

6

Qu

an

tizati

on

Fu

ncti

on

Iteration Number

0 5 10 15 20

6

8

10

12

SN

R [

dB

]

SNRfinal

= 8.9452

0 5 10 15 201

1.5

2

2.5

R [

bit

/sym

bo

l]

Rfinal

= 1.7723

Iteration Number

0 5 10 15 20 25 30 35 40

6

8

10

12

SN

R [

dB

]

SNRfinal

= 10.5363

0 5 10 15 20 25 30 35 401

1.5

2

2.5

R [

bit

/sym

bo

l]

Rfinal

= 2.0044

Iteration Number

5 10 15 20 25 30 35 40-6

-4

-2

0

2

4

6

Qu

an

tizati

on

Fu

ncti

on

Iteration Number


Example of the EC Lloyd algorithm (III)

X zero-mean, unit-variance Laplacian r.v.

Design entropy-constrained scalar quantizer with rate R2 bits

and minimum distortion D*

Optimum quantizer, obtained with the entropy-constrained Lloyd

algorithm 21 intervals (in [-10,10]), almost uniform

D*= 0.07=11.38 dB, R= 2.0023 bits (compare to fixed-length example)

Decision threshold

Representative level


Example of the EC Lloyd algorithm (IV)

Initial quantizer B, only 4 intervals

(21 & odd) in [-10,10], with the

same length

Convergence in cost faster than convergence of decision thresholds

5 10 15 20-10

-5

0

5

10

Qu

an

tizati

on

Fu

ncti

on

Iteration Number

0 5 10 15 20

5

10

15

SN

R [

dB

]SNR

final = 7.4111

0 5 10 15 201

2

3

R [

bit

/sym

bo

l]

Rfinal

= 1.6186

Iteration Number

0 5 10 15 20 25 30 35 40

5

10

15

SN

R [

dB

]

SNRfinal

= 11.407

0 5 10 15 20 25 30 35 401

2

3

R [

bit

/sym

bo

l]

Rfinal

= 2.0063

Iteration Number

5 10 15 20 25 30 35 40-10

-5

0

5

10

Qu

an

tizati

on

Fu

ncti

on

Iteration Number


High-rate results for EC scalar quantizers

For MSE distortion and high rates, uniform quantizers (followed by

entropy coding) are optimum [Gish, Pierce, 1968]

Distortion and entropy for smooth PDF and fine quantizer interval

Distortion-rate function

is factor or 1.53 dB from Shannon Lower Bound

222

2

12d d

2 logH X h X

2 21

2 212

h X Rd R

6

e

2 21

2 22

h X RD Re


Comparison of high-rate performance of

scalar quantizers

High-rate distortion-rate function

Scaling factor

2 2 22 RXd R

2

2

Shannon LowBd Lloyd-Max Entropy-coded

6Uniform 0.703 1 1

e

9Laplacian 0.865 4.5 1.232

2 6

3Gaussian 1 2.721 1.423

2 6

e e

e


Deadzone uniform quantizer

Quantizer

input

Quantizer

output x

x


Embedded quantizers

Motivation: scalability decoding of compressed bitstreams at different rates (with different qualities)

Nested quantization intervals

In general, only one quantizer can be optimum

(exception: uniform quantizers)

Q0

Q1

Q2

x


Example: Lloyd-Max quantizers for Gaussian PDF

2-bit and 3-bit optimal

quantizers not embeddable

Performance loss for

embedded quantizers

0 1

0

0

0

1

1

0

1

1

0

0

0

0

0

1

0

1

0

0

1

1

1

0

0

1

0

1

1

1

0

1

1

1

-.98 .98

-1.75 -1.05 -.50 .50 1.05 1.75


Information theoretic analysis

Successive refinement Embedded coding at multiple rates w/o loss relative to R-D function

Successive refinement with distortions and can be achieved iff there exists a conditional distribution

Markov chain condition

1D 2 1D D

1 1

2 2

( , )

( , )

E d X X D

E d X X D

1 1

2 2

( ; ) ( )

( ; ) ( )

I X X R D

I X X R D

1 2 2 1 2 1 2 2 1 2 ,

( , , ) ( , ) ( , )X X X X X X X

f x x x f x x f x x

2 1 X X X

[Equitz,Cover, 1991]


Embedded deadzone uniform quantizers

Q0

Q1

Q2

x

8

4

2

x=0

4

2

Supported in JPEG-2000 with general for quantization of wavelet coefficients.


Vector quantization


LBG algorithm

Lloyd algorithm generalized for VQ [Linde, Buzo, Gray, 1980]

Assumption: fixed code word length

Code book unstructured: full search

Best representative vectors

for given partitioning of training set

Best partitioning of training set

for given representative

vectors


Design of entropy-coded vector quantizers

Extended LBG algorithm for entropy-coded VQ [Chou, Lookabaugh, Gray, 1989]

Lagrangian cost function: solve unconstrained problem rather than

constrained problem

Unstructured code book: full search for

2

min.J d R E X X H X

The most general coder structure:

Any source coder can be interpreted as VQ with VLC!

2

2 log

ix i q qJ q x x p


Lattice vector quantization

Amplitude 1

Am

plit

ude 2

cell

pdf

representative vector


8D VQ of a Gauss-Markov source

r 0.95

SN

R [

dB

]

18

12

6

0

0 0.5 1.0

Rate [bit/sample]

scalar

E8-lattice

LBG variable CWL

LBG fixed CWL


8D VQ of memoryless Laplacian source S

NR

[dB

]

Rate [bit/sample]

scalar

E8-lattice LBG variable CWL

LBG fixed CWL

9

6

3

0.5 1.0

0

0


Reading

Taubman, Marcellin, Sections 3.2, 3.4

J. Max, Quantizing for Minimum Distortion, IEEE Trans. Information Theory, vol. 6, no. 1, pp. 7-12, March 1960.

S. P. Lloyd, Least Squares Quantization in PCM, IEEE Trans. Information Theory, vol. 28, no. 2, pp. 129-137,

March 1982.

P. A. Chou, T. Lookabaugh, R. M. Gray, Entropy-constrained vector quantization, IEEE Trans. Signal Processing, vol. 37, no. 1, pp. 31-42, January 1989.

05-Quantization

Documents

Transcript of 05-Quantization