Image Compression Overviewinst.eecs.berkeley.edu/~ee290t/sp04/lectures/imgcompress.pdf · Bernd...

Bernd Girod: EE398A Image Communication I Image Compression Overview no. 1

Image Compression Overview

Goal of this chapter: provide a first introduction of some key terms and ideas of image compression, without rigorous treatmentLossless vs lossy compressionMeasuring distortion and compressionStatistical redundancy and entropyCompression as a vector quantization problemQuantizationTransformsCompression with predictive feedback


Digital images

A digital image is a two-dimensional sequence of samples

[ ]1 2 1 1 2 2, , 0 , 0 x n n n N n N≤ < ≤ <

2 1N −

1 1N −

01

0 1 2 2n

2

[ ]1 2,x n n1n


Discrete image intensities

Unsigned B-bit imagery

Signed B-bit imageryMost common: B=8, but larger B are used in medical, military, or scientific applications.Useful interpretation

[ ] { }1 2, 0,1, , 2 1Bx n n ∈ −…

[ ] { }1 1 11 2, 2 , 2 1, , 2 1B B Bx n n − − −∈ − − + −…

[ ] [ ]1 2 1 2, 2 ,Bx n n x n n′=

Real-valued intensities, range 0…1

or –1/2…+1/2

Rounding to nearest

integer


Multiple image components

Color images typically represented by three values per sample location, for red, green and blue primary components

General multi-component image

Examples:Color printing: cyan, magenta, yellow, black dyes, sometimes moreHyperspectral satellite imaging: 100s of channels

[ ] [ ] [ ]1 2 1 2 1 2, , , , , R G Bx n n x n n x n n

[ ]1 2, , 1, 2, , Cx n n c C= …


Some classes of imagery

“Natural” image

Text image

Graphics


Lossless compression

Minimize number of bits required to represent original digital image samples w/o any loss of information.All B bits of each sample must be reconstructed perfectly.Achievable compression usually rather limited.Applications

Binary images (facsimile)Medical imagesMaster copy before editingPalettized color images


Lossy compression

Some deviation of decompressed image from original(“distortion”) is often acceptable:

Human visual system might not perceive loss, or tolerate it.Digital input to compression algorithm is imperfect representation of real-world scene

Much higher compression than with lossless.Lossy compression used widely for natural images (e.g. JPEG) and motion video (e.g. MPEG).


Lossy compression: measuring distortion

Most commonly employed: Mean Squared Error

. . . or, equivalently, Peak Signal to Noise Ratio

AdvantagesEasy calculationMathematical tractability in optimization problems

DisadvantageNeglects properties of human vision

[ ] [ ]( )1 2

1 2

1 12

1 2 1 20 01 2

1 ˆMSE= , ,N N

n nx n n x n n

N N

− −

= =

−∑ ∑

( )2

10

2 1PSNR=10log dB

MSE

B −

Gamma correction


Display devices (e.g. CRTs) highly nonlinear

Cameras compensate by γ-predistortion circuitryIEC 61966-2-1 standard for γ-predistorted color space sRGB (“standard RGB”), each normalized linear RGB component mapped by

~ , 1.8 2.8L xγ γ = …Screen

luminance

( ) 1/

if 01 if 1

lin lin

lin lin

gx xx

x xγ

εβ β ε

≤ ≤⎧′ = ⎨ + − ≤ ≤⎩

( ) ( )2.4, 0.055, ,

111 1g

γ

β βγ β εε γ

βγ

⎛ ⎞⎜ ⎟⎜ ⎟= = = =⎜ ⎟ −⎛ ⎞

− −⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠⎝ ⎠


sRGB γ-characteristic

linx

x′

Linearpiece


Effect of a small error

linx

x′

dx′

lindx

( )( )

1

1 1/

lin

lin

dx x dx

x dx

γ

γ

γ

γ

−

−

′ ′=

′=

Roughly matchesWeber’s Law.

PSNR distortionmeasure only usefulin the γ-correcteddomain.


Measures of compression

Image represented by “bit-stream” c of length ||c||.Compare no. of bits w/ and w/o compression

Alternatively

For lossy compression, bit-rate more meaningful than compression ratio, as B is somewhat arbitrary.

1 2compression ratio = N N Bc

1 2

bit-rate = bits/pixelN N

c


Typical bit-rates after compression

Substantially dependent on image content: consider typical natural imagesLossless compression: (B-3) bpp (bits per pixel)Assume viewing on computer monitor, 90 pixels/inch.Lossy compression,

high quality: 1 bppmoderate quality: 0.5 bppusable quality: 0.25 bpp

Perceived distortion depends on sampling density and contrast


How does compression work?

Exploit statistical redundancy.Take advantage of patterns in the signal.Describe frequently occuring events efficiently.Lossless coding: only statistical redundancy

Introduce acceptable deviations.Omit “irrelevant” detail that humans cannot perceive.Match the signal resolution (in space, time, amplitude) to the applicationLossy coding: exploit statistical and visual redundancy


Statistical redundancy

Trivial example: Given two B-bit integers (e.g., representing two adjacent pixels)

Assume that only takes on values {0,1}Compression to 1 bpp

Further assume, that Compression to 0.5 bpp

Hope: bit-rate increases only slightly, as long as the above assumptions hold with high probability

{ }1 2, 0,1, , 2 1Bx x ∈ −…

1 2,x x

1 2x x=


Joint histogram of two horizontally adjacent pixels

relative frequency

of occurence

amplitude of adjacent pixel

amplitude of current pixel


Entropy

Statistical redundancy is formalized by the concept of entropy of Shannon’s information theoryMinimum bit-rate for encoding or separately

Minimum bit-rate for encoding and jointly

Entropies are determined entirely by the statistical distribution of values and

1x 2x

( ) ( )1 2 H x H x

1x 2x

( ) ( ) ( )1 2 1 2, H x x H x H x≤ +

( ) ( ) ( )1 2 1 2, , , H x H x H x x1x 2x


Visual Irrelevance

For images to be viewed by humans, no need to represent more than the visible resolution in

spacetimebrightnesscolor

Required resolution might depend on image content (“masking”)For some applications, only a specific region of the image might be relevant, e.g., in

medical imagingmilitary imaging


Irrelevance in Color Imagery

Human visual system has much lower acuity for color hue and saturation than for brightnessUse color transform to facilitate exploiting that property

Note

Cb and Cr often sub-sampled 2:1 relative to Y.

0.299 0.587 0.1140.169 0.331 0.50.5 0.419 0.0813

Y R

Cb G

Cr B

x xx xx x

⎛ ⎞ ⎛ ⎞ ⎛ ⎞⎜ ⎟ ⎜ ⎟ ⎜ ⎟= − − ⋅⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟− −⎝ ⎠ ⎝ ⎠ ⎝ ⎠

RGB components (γ-predistorted)

Chrominancecomponents

( ) ( )0.564 0.713Cb B Y Cr R Yx x x x x x= − = −

Luminancecomponent


Compression as a global mapping

( )compressor

M=c x ( )1

decompressor

ˆ M −=x c

image x bit-stream c

reconstructed ˆimage x

1 2

lookup table2 entriesN N B

lookup table

2 entriescfixed length cLookup table

interpretation

Lossless compressionLossy compression

Leads to idempotent compression system

1 1M M− −=

( ) ( )( )1arg min ,M D M −

′′= =

cc x x c

( )( )1M M − =c c


Compression as a global mapping (cont.)

Variable length encoding: need formulation of trade-off between rate and distortionLookup table formulation corresponds to approach taken by fixed-length vector quantizationSize of lookup tables for entire image is impractical (both for encoder and decoder)Impose structure on mappings and

Avoid exponential increase of complexity with image size and size of bit-streamMaintain achievable performance close to unconstrained system

1M −M


Typical structured compression system

( )transform

T=y x ( )quantizer

Q=q y ( )encoder

C=c qsamples yimage x indices q

( )1

inversetransform

ˆ ˆT −=x y ( )1

dequantizer

ˆ Q−=y q ( )1

decoderC−=q c

indices qˆsamples yreconstructed

ˆimage x

bit-stream c

Transform T(x) usually invertibleQuantization not invertible, introduces distortionCombination of encoder and decoder lossless

( )Q y( )C q ( )1C− c


Variable length coding

Goal: exploit non-uniformity of probability distribution of quantization indicesSimple example: Probabilities Code words

Average code word length

Often much bigger gains for image compression

1/ 2 if 0 11/ 4 if 1 01

( ) 1/ 8 if 2 0011/ 8 if 3 000

q

qq

P qqq

=⎧⎪ =⎪= ⎨ =⎪⎪ =⎩

1 1 11 bits 2bits 2 3bits 1.75 bits2 4 8⋅ + ⋅ + ⋅ ⋅ =


Quantization

Goal: reduce the number of possible amplitude values for codingSimple scalar quantizer with four output indices

Dequantization

Interval 1 Interval 2 Interval 3 Interval 4

y

q=3

Interval 1 Interval 2 Interval 3 Interval 4

2y 3y4y1y


Transforms

Goal: represent the image samples in a different form, such that statistical dependencies are greatly reducedSimple example:

-- -

imagesamples

x[m,n]

transformsamplesy[m,n]

[ ][ ]

}{

[ ]

, 0,1

1 , if , both even4

,, 2 ,2 otherwise

2 2

i j

x m i n j m n

y m nm nx m n y

∈

⎧ + +⎪⎪= ⎨

⎡ ⎤⎢ ⎥ ⎢ ⎥⎪ − ⎢ ⎥⎢ ⎥ ⎢ ⎥⎪ ⎣ ⎦ ⎣ ⎦⎣ ⎦⎩

∑[ ]

[ ] [ ]}{

[ ]

, 0,1

ˆ ˆ2 , , if , both even

ˆ ,ˆ ˆ, 2 ,2 otherwise

2 2

i j

y m n y m i n j m n

x m nm ny m n y

∈

⎧ − + +⎪⎪=⎨ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎪ − ⎢ ⎥⎢ ⎥ ⎢ ⎥⎪ ⎣ ⎦ ⎣ ⎦⎣ ⎦⎩

∑


[ ]indices q k

Compression with predictive feedbackin the transform domain

( )transform

T=y x [ ] [ ]( )quantizer

Q e=q k k ( )encoder

C=c q[ ]

samples y kimage x

[ ]indices q k

( )1

inversetransform

ˆ ˆT −=x y ( )1

decoderC−=q c

reconstructedˆimage x

bit-stream c

Σ [ ] [ ]( )1

dequantizer

e Q q−=k k

Σ

[ ] [ ]( )1 2

causal predictorˆ , 1 ,py P y k k= −k …

[ ] [ ]( )1

dequantizer

e Q q−=k kΣ

[ ] [ ]( )1 2

causal predictorˆ , 1 ,py P y k k= −k …

- [ ]residual e k

[ ]e k[ ]y k

[ ]e k[ ]y k

[ ]p

prediction y k

[ ]p

prediction y k


Compression with predictive feedbackin the image domain

residual

e ( )

transformT=y e ( )

quantizerQ=q y ( )

encoderC=c q

image x indices q

( )1

decoderC−=q c

reconstructedˆimage x

bit-stream c

Σ( )1

dequantizer

ˆ Q−=y q

Σ

( )causal predictor

ˆP=px x

Σ

-

exprediction

px

( )1

inversetransform

ˆ ˆT−=e yy

prediction

px( )

causal predictorˆP=px x

( )1

dequantizer

ˆ Q−=y qe

( )1

inversetransform

ˆ ˆT−=e yy indices q


Reading Assignment

Taubman+Marcellin, Chapters 2.1 and 2.2

Image Compression Overviewinst.eecs.berkeley.edu/~ee290t/sp04/lectures/imgcompress.pdf · Bernd...

Documents

Transcript of Image Compression Overviewinst.eecs.berkeley.edu/~ee290t/sp04/lectures/imgcompress.pdf · Bernd...