New Multiresolution and subband coding - Stanford University · 2000. 10. 26. · Filtering...

1

Bernd Girod: EE368b Image and Video Compression Multiresolution & Subband no. 1

Multiresolution and subband coding

n Predictive (closed-loop) pyramids

n Open-loop (“Laplacian”) pyramids

n Subband coding

n Perfect reconstruction filter banks

n Quadrature mirror filter banks

n Discrete Wavelet Transform (DWT)

n Embedded zerotree wavelet (EZW) coding

n Transform coding as a special case of subband coding


Interpolation error coding, I

Interpolator

InterpolatorSubsampling

• • •

• • •

Input picture

Reconstructed picture

Q

Q

+

+

-

- +

+

+

+


• • • • • •

Coder includes

Decoder

Sample encoded in current stage

Previously coded sample

2


Interpolation error coding, II

signals to be encoded

original image


Predictive pyramid, I

Interpolator

Subsampling

Q

Q

+

+

-

- +

+

+

+

InterpolatorInterpolator

Filtering

Filtering

Subsampling

Input picture


• • •

• • •• • • • • •

Coder includes

Decoder

Sample encoded in current stage

3


Predictive pyramid, II

Number of samples to be encoded =

1+1N

+1

N 2+ ...

=

N

N −1 x number of original image samples

subsampling factor


Predictive pyramid, III

original image

signals to be encoded

4


Comparison:interpolation error coding vs. pyramid

n Resolution layer #0, interpolated to original size for display

Interpolation Error Coding Pyramid





5







n Resolution layer #3

=(original)


6


Open-loop pyramid (Laplacian pyramid)

Interpolator

Interpolator

Subsampling

Q+

Q+ -

-

Filtering

Filtering

Interpolator


Input picture


• • • • • • • • • • • •

Receiver Transmitter


When multiresolution coding was a new idea . . .

This manuscript is okay if compared to some of the weaker papers.[. . .] however, I doubt that anyone will ever use this algorithm again.

Anonymous reviewer of Burt and Adelson‘s original paper, ca. 1982

7


Embedded multiresolution coding

Φ (ω)uu

ω

preserved spectrum

reconstruction errorspectrum low rate

Φ (ω)vv

reconstruction errorspectrum high rate

Low frequency components have to berefined for optimal resolution/noise balance


Optimum bit allocation for open-loop pyramid

i=0

i=1

i=2

variance

number of samples

ri =12

log2σi

2

Ni+ f (R) bits /sample

depends on bit-rate R, but not not layer i

Assume layer m.s.e distortion-rate function:

di ri

= σ i2 2

−2ri

8


ri =

12

log2

ασi2(N

i−1− Ni)σ i−1

2 (Ni − Ni+1)

bits/sample

i=0

i=1

i=2 variance number of samples

power transfer

factor

does not depend on R !!

remaining bits

r0 = R −

Ni rii=1

L−1

∑

Nii=0

L−1

∑bits /sample

Optimum bit allocation for closed-loop pyramid


Two-layer open- vs. closed-loop pyramid

30

32

34

36

38

0.2 0.4 0.6 0.8 1.0

PS

NR

[dB

]

Rate R [bits/sample]

28

30

32

34

36

0.2 0.4 0.6 0.8 1.0

PS

NR

[dB

]

Rate R [bits/sample]

Theory Actual Coder

closed loop

open loop open loop

closed loop

9


Subband coding

n Number of degrees of freedom is preserved:

“Critically sampled filter bank”

n Perfect reconstruction filter bank required

Input signal G (ω)k Q k

k Q k

k Q k

• • •

Analysis filterbank

Synthesis filterbank

Reconstructed signal

F (ω)

G (ω)

G (ω)

1

M

1

F (ω)0 0 0 0

1F (ω)1

MMM

+

Transmitter Receiver

1k0

+1k1

+ ...+1

kM= 1


Two-channel filterbank

n Aliasing cancellation if :

2 2

2 2

F (ω)0

F (ω)1

G (ω)

G (ω)

0

1

X(ω)X(ω)

G0 (ω ) = F1 (ω + π )

−G1(ω ) = F0 (ω + π)

ˆ X (ω) =12

F0 (ω )G0 (ω) + F1 (ω )G1(ω )[ ]X(ω )

+12

F0 (ω + π )G0 (ω) + F1(ω + π )G1 (ω)[ ]X(ω + π )Aliasing

10


Example: two-channel filter bank with perfect reconstruction

n Analysis filter impulse responses:

l Lowpass band:

l Highpass band:

n Synthesis filter impulse responses:

l Lowpass band:

l Highpass band:

n Frequency responses:

|G |0

|F |0

1|F |

1|G |

π2

π

1

2

00

FrequencyF

requ

ency

res

pons

e

14

−1, +2, +6, +2, −1( )

14

+1, −2, +1( )

14

+1, +2, −6, +2, +1( )

14

+1, +2, +1( )


Quadrature mirror filters (QMF)

n QMFs achieve aliasingcancellation by choosing

n Highpass band is the mirror image of the lowpass band in the frequency domain

F1(ω ) = F0 (ω + π )

= −G1 (ω ) = G0 (ω + π )

frequency ω

Example:16-tap QMF filterbank

11


Cascaded analysis / synthesis filterbanks


Discrete Wavelet Transform

n Recursive application of a two-band filter bank to thelowpass band of the previous stage yields octave band splitting:

n Same concept can be derived from wavelet theory: Discrete Wavelet Transform (DWT)

frequency

12


2-d Discrete Wavelet Transform

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

ωx

ωy

...etc


2-d Discrete Wavelet Transform example

13


Embedded zero-tree wavelet algorithm

X X X XX X X XX X X XX X X X

X X X X

X

X

X X X X


X

X X X X


n Idea: Conditional coding of all descendants (incl. children)

n Coefficient magnitude > threshold: significant coefficients

n Four cases

l ZTR: zero-tree, coefficient and all descendants are not significant

l IZ: coefficient is not significant, but some descendants are significant

l POS: positive significant

l NEG: negative significant

„Parent“

„Children“

„Descendants“


Embedded zero-tree wavelet algorithm (cont.)

n For the highest bands, ZTR and IZ symbols are merged into one symbol Z

n Successive approximation quantization and encodingl Initial „dominant“ pass

• Set initial threshold T, determine significant coefficients• Arithmetic coding of symbols ZTR, IZ, POS, NEG

l Subordinate pass• Refine magnitude of coefficients found significant so far by one bit

(subdivide magnitude bin by two)• Arithmetic coding of sequence of zeros and ones.

l Repeat dominant pass• Set previously found significant coefficients to zero• Decrease threshold by factor of 2, determine new significant

coefficients• Arithmetic coding of symbols ZTR, IZ, POS, NEG

l Repeat subordinate and dominate passes, until bit budget is exhausted.

14


Embedded zero-tree wavelet algorithm (cont.)

n Decoding: bitstream can be truncated to yield a coarser approximation: „embedded“ representation

n Further details: J. M. Shapiro, „Embedded image coding using zerotrees of wavelet coefficients,“ IEEE Transactions on Signal Processing, vol. 41, no. 12, pp. 3445-3462, December 1993.

n Enhancement SPIHT coder: A. Said, A., W. A. Pearlman, „A new, fast, and efficient image codec based on set partitioning inhierarchical trees,“ IEEE Transactions on Circuits and Systemsfor Video Technology, vol. 63 , pp. 243-250, June 1996.

n JPEG-2000 standard similar to SPIHT


Subband coding vs. transform coding, I

n Transform coding is a special case of subband coding with:l Number of bands = order of transform N

l Subsampling factor K = N

l Length of impulse responses of analysis/synthesis filters � N

n Filters used in subband coders are not in general orthogonal.

15


Subband coding vs. transform coding, II

re-order

Original image

8x8 DCT

8-channel subband decomposition (using DCT filters)


Frequency response of a DCT of order N=8

0 π/2 π −30

−20

−10

0

10

Frequency

Fre

quen

cy r

espo

nse

[dB

]

i = 0

0 π/2 π −30

−20

−10

0

10

Frequency

Fre

quen

cy r

espo

nse

[dB

]

i = 1

0 π/2 π −30

−20

−10

0

10

Frequency

Fre

quen

cy r

espo

nse

[dB

]

i = 2

0 π/2 π −30

−20

−10

0

10

Frequency

Fre

quen

cy r

espo

nse

[dB

]

i = 3

0 π/2 π −30

−20

−10

0

10

Frequency

Fre

quen

cy r

espo

nse

[dB

]

i = 4

0 π/2 π −30

−20

−10

0

10

Frequency

Fre

quen

cy r

espo

nse

[dB

]

i = 5

0 π/2 π −30

−20

−10

0

10

Frequency

Fre

quen

cy r

espo

nse

[dB

]

i = 6

0 π/2 π −30

−20

−10

0

10

Frequency

Fre

quen

cy r

espo

nse

[dB

]

i = 7

16


Summary:multiresolution and subband coding

n Resolution pyramids with subsampling 2:1 horizontally and vertically

n Predictive pyramids: quantization error feedback („closed loop“)

n Transform pyramids: no quantization error feedback („open loop“)

n Pyramids: overcomplete representation of the image

n Critically sampled subband decomposition: number of samples not increased

n Quadrature mirror filters: aliasing cancellation

n Discrete Wavelet Transform = cascaded 2:1 subband splits

n Exploit statistical dependencies across subbands by zero-trees

n Transform coding is a special case of subband coding

New Multiresolution and subband coding - Stanford University · 2000. 10. 26. · Filtering...

Documents

Transcript of New Multiresolution and subband coding - Stanford University · 2000. 10. 26. · Filtering...