JPEG and JPEG2000 · BASIC JPEG NAIVE ALGORITHM I Given an N ×N image A (A square and N is...

JPEG AND JPEG2000

Catherine Bénéteau, Caroline Haddad,David Ruch and Patrick J. Van Fleet

Center for Applied MathematicsUniversity of St. Thomas

St. Paul, MN USA

PREP - Wavelet Workshop, 2008

SATURDAY, 7 JUNE, 2008 (LECTURE 10) JPEG/JPEG2000 PREP 2008 1 / 11

TODAY’S SCHEDULE

9:00-10:15 Lecture Nine: The Lifting Method10:15-10:30 Coffee Break (OSS 237)10:30-11:45 ⇒Lecture Ten: JPEG and JPEG200011:45-12:00 Evaluations/Wrap Up12:00-1:00 Lunch (Cafeteria)

TODAY’S SCHEDULE

OUTLINE

TODAY’S SCHEDULE

BASIC JPEGNaive AlgorithmAn Example

JPEG2000Features and EnhancementsBasic AlgorithmThe Cohen/Daubechies/Feauveau (CDF97) FilterThe Quantization ProcessAn Example (Reprise)

BASIC JPEG NAIVE ALGORITHM

I Given an N × N image A (A square and N is divisible by 8 forsimplicity):

I Subtract 127 from each element in A.I Partition A into 8× 8 blocks.I Transform each block using the Discrete Cosine Transform (DCT).I Quantize the elements in each block.I Encode using Huffman encoding (more sophisticated version).

I Note that the quantization step makes JPEG an example of lossycompression.

BASIC JPEG AN EXAMPLE

Consider the moon surface image stored in A. The dimensions are200× 200.

We partition A into 8× 8 blocks. There are 25× 25 blocks. We havehighlighted one block (call it A23) for use as a running example:

The elements in A23 are:

140 142 127 126 124 129 135 129142 142 132 125 125 124 130 133143 135 122 134 140 115 124 139121 93 104 159 209 166 117 130

96 58 83 150 220 224 142 12381 37 68 124 203 231 161 11388 29 56 101 166 201 141 118

126 72 57 92 131 135 135 133

Shifting by 127 gives

13 15 0 −1 −3 2 8 215 15 5 −2 −2 −3 3 616 8 −5 7 13 −12 −3 12−6 −34 −23 32 82 39 −10 3−31 −69 −44 23 93 97 15 −4−46 −90 −59 −3 76 104 34 −14−39 −98 −71 −26 39 74 14 −9−1 −55 −70 −35 4 8 8 6

The DCT is the 8× 8 matrix

cos( π16 ) cos( 3π

16 ) cos( 5π16 ) cos( 7π

16 ) cos( 9π16 ) cos( 11π

16 ) cos( 13π16 ) cos( 15π

cos( π8 ) cos( 3π

8 ) cos( 5π8 ) cos( 7π

8 ) cos( 9π8 ) cos( 11π

8 ) cos( 13π8 ) cos( 15π

cos( 3π16 ) cos( 9π

16 ) cos( 15π16 ) cos( 21π

16 ) cos( 27π16 ) cos( 33π

16 ) cos( 39π16 ) cos( 45π

cos( π4 ) cos( 3π

4 ) cos( 5π4 ) cos( 7π

4 ) cos( 9π4 ) cos( 11π

4 ) cos( 13π4 ) cos( 15π

16 ) cos( 25π16 ) cos( 35π

16 ) cos( 45π16 ) cos( 55π

16 ) cos( 65π16 ) cos( 75π

8 ) cos( 15π8 ) cos( 21π

8 ) cos( 27π8 ) cos( 33π

8 ) cos( 39π8 ) cos( 45π

16 ) cos( 35π16 ) cos( 49π

16 ) cos( 63π16 ) cos( 77π

16 ) cos( 91π16 ) cos( 105π

I Row j of U, j = 2, . . . , 7 is formed by evaluating cos t at 8uniformly spaced points on the interval [0, jπ).

I We won’t talk much about U today, but we will note:I U is orthogonal so U−1 = UT .I U maps the vector (1, 1, 1, 1, 1, 1, 1, 1)T to (2

√2, 0, 0, 0, 0, 0, 0, 0)T

I To apply the DCT to A23, we compute UAUT .

√2, 0, 0, 0, 0, 0, 0, 0)T

We have

T23 = UA23UT

2.875 −136.615 −80.091 131.114 52.125 −7.580 20.592 2.797

55.579 121.818 48.461 −92.981 −18.519 −9.120 −25.666 −1.5478−50.705 41.645 95.027 −35.707 −27.598 15.933 −12.217 −0.976

13.668 −64.300 −58.929 50.519 −24.407 5.640 19.388 −5.37513.125 10.890 −7.207 −0.237 20.375 −7.510 −6.621 −1.543−8.119 −4.627 −12.092 2.622 8.589 −1.813 4.358 1.207

1.384 8.555 16.283 −7.630 −3.587 3.495 −12.277 2.213−4.652 −2.066 7.586 −1.387 −3.124 2.021 3.110 −4.523

Perhaps a plot is more informative:

The DCT tends to store information about all 64 values into a fewvalues and “shove” them to the upper left of the output. The remainingvalues are either 0 or approximately 0.

Here is a picture of the DCT of the entire image A:

The quantization step involves dividing element-wise the individual8× 8 blocks by the matrix

16 11 10 16 24 40 51 6112 12 14 19 26 58 60 5514 13 16 24 40 57 69 5614 17 22 29 51 87 80 6218 22 37 56 68 109 103 7724 35 55 64 81 104 113 9249 64 78 87 103 121 120 10172 92 95 98 112 100 103 99

and then rounding the results to the nearest integer. Note that therounding step is irreversible.

For our 8× 8 block T23 we have

0 −12 −8 8 2 0 0 05 10 3 −5 −1 0 0 0

−4 3 6 −1 −1 0 0 01 −4 −3 2 0 0 0 01 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0

I We then add 127 to each element of each block, and then applyHuffman encoding to the result.

I The compressed image can be represented using 57231 bits or1.43078 bits per pixel.

I That is quite a substantial savings over the original 320000 bitsneeded to represent the image.

I There are some drawbacks:I the compression is lossyI the uncompressed image is “blocky”I the transform doesn’t handle edge effects

Compressed:

Original:

Here is a blow-up of the original:

Here is a blow-up of the compressed image:

JPEG2000 FEATURES AND ENHANCEMENTS

I Work on the new standard began in the late 1990’s (1997)I Here are some features of the new standards:

I Lossy and lossless compression The new standard has bothoptions.

I Better compression rates than JPEG.I Progressive Signal Transmission - JPEG2000 can reconstruct the

digital image as it is received via the internet.I Tiling - a generalization of the JPEG practice of partitioning images

into 8× 8 blocks.I Regions of interest - JPEG2000 allows the user to identify ROIs

and compress them at higher rates than other parts of the images.I Larger image size - JPEG could handle images of size

64, 000× 64, 000 or smaller while the maximum image size forJPEG2000 is 232 − 1× 232 − 1.

I Multiple channels - JPEG could handle color images (3 channels)while JPEG2000 can handle up to 256 channels.

JPEG2000 BASIC ALGORITHM

I Subtract 127 from each element of input grayscale image A.I Apply i iterations of the biorthogonal wavelet transform using the

LeGall filter for lossless compression and theCohen/Daubechies/Feauveau (9, 7) filter (more later) for lossycompression.

I For lossy compression, apply a quantizer (more later) much likethat used for the wavelet shrinkage algorithm in denoisingapplications.

I Use an arithmetic coding system called Embedded Block Codingwith Optimized Truncation (EBCOT) to the quantized transform.

JPEG2000 THE COHEN/DAUBECHIES/FEAUVEAU (CDF97) FILTER

I The JPEG2000 committee decided they wanted to use waveletsinstead of the DCT.

I Application of the wavelet transform to the entire image reducesthe block effects caused by applying the DCT to 8× 8 blocks ofthe partitioned image.

I Insisting the filters be symmetric helps remove boundary effects.I Committee members wanted long filters but with minimal

difference in filter lengths.I Decided on lengths 9 and 7 but rejected the (9, 7) biorthogonal

spline filter.I The Fourier series for the filters are unbalanced at ω = π.

The Fourier series H(ω) for the 7-term filter (built from the binomialcoefficients) h satisfies H(j)(π) = 0 for j = 0, . . . , 5:

But Fourier series H(ω) for the 9-term filter h satisfies H(j)(π) = 0 foronly j = 0, 1:

The CDF97 filter comes from the following results:

THEOREM

If h and h are symmetric, odd-length biorthogonal filters, then theirFourier series H(ω) and H(ω) can be written as

H(ω) =√

2 cos2˜(ω

2)p(cos(ω))

andH(ω) =

√2 cos2`(

2)p(cos(ω))

where ˜and ` are nonnegative integers and p and p are polynomialswith p(−1) 6= 0, p(−1) 6= 0 and p(1) = p(1) = 1.

THEOREM

The polynomials p and p satisfy

p(cos(ω))p(ω) =K−1∑j=0

(K − 1 + j

)sin2j(

where K = ˜+ `.

For the biorthogonal spline filters, p(t) = 1.

H(ω) =√

2 cosN(ω

H(ω) =√

2 cosN(ω

K−1∑j=0

(K − 1 + j

)sin2j(

I The CDF97 filter is constructed by splitting the factors of thepolynomial

P(t) =K−1∑j=0

(K − 1 + j

I We use ˜= ` = 2 so that K = ˜+ ` = 4 and consider the cubicpolynomial

P(t) =3∑

(3 + j

)t j = 1 + 4t + 10t2 + 20t3

I This polynomial has one real root r1 = −0.342484 and twocomplex roots c1 = −0.078808− .373391i ,c2 = −0.078808 + .373391i .

P(t) =K−1∑j=0

(K − 1 + j

P(t) =3∑

(3 + j

)t j = 1 + 4t + 10t2 + 20t3

P(t) =K−1∑j=0

(K − 1 + j

P(t) =3∑

(3 + j

)t j = 1 + 4t + 10t2 + 20t3

I Thus we can rewrite P(t) = 1 + 4t + 10t2 + 20t4 as

P(t) = 20(t − r1)(t − c1)(t − c2)

I We set

p(t) = a(t − r1) and p(t) =20a

(t − c1)(t − c2)

I To make the Fourier series H(ω) and H(ω) satisfyH(0) = H(0) =

√2, we must have a = 2.920696.

P(t) = 20(t − r1)(t − c1)(t − c2)

I We set

p(t) = a(t − r1) and p(t) =20a

(t − c1)(t − c2)

√2, we must have a = 2.920696.

P(t) = 20(t − r1)(t − c1)(t − c2)

I We set

p(t) = a(t − r1) and p(t) =20a

(t − c1)(t − c2)

√2, we must have a = 2.920696.

Substituting t = sin2(ω2 ), a = 2.020696 and expanding into Fourier

series gives

H(ω) =3∑

k=−3

hkeikω

where (to 6 digits)

h0 = 0.788486h−1 = h1 = 0.418092h−2 = h2 = −0.040689h−3 = h3 = −0.064539

Substituting t = sin2(ω2 ), a = 2.020696 and expanding into Fourier

series gives

H(ω) =4∑

k=−4

hkeikω

where (to 6 digits)

h0 = 0.852699h−1 = h1 = 0.377403h−2 = h2 = −0.110624h−3 = h3 = −0.023850h−4 = h4 = 0.037829

The plots of |H(ω)| and |H(ω)| look more like those representinglowpass filters.

|H(ω)| for the seven-term filter with H(j)(π) = 0, for j = 0, 1, 2, 3.

The plots of |H(ω)| and |H(ω)| look more like those representinglowpass filters:

|H(ω)| for the nine-term filter with H(j)(π) = 0, for j = 0, 1, 2, 3.

JPEG2000 THE QUANTIZATION PROCESS

I For lossy compression, a quantization function is applied to eachportion of the biorthogonal wavelet transformation.

I The quantization function is

q(t) = sgn(t)b|t |/dc

I The value d is called the quantization step size and this valuechanges depending on (1) the iteration and (2) the portion of thetransform being quantized.

I Other factors that influence the choice of d :I The number of bits needed to represent the values of the original

image.I The number of bits needed to represent the exponent and mantissa

of values in the blur portion of the transform.I The analysis gain bits for each portion of the transformation.

JPEG2000 AN EXAMPLE (REPRISE)

We return to the 200× 200 image

and perform lossy compression using JPEG2000.

We perform 3 iterations of the biorthogonal wavelet transformationusing the CDF97 filter and exploiting symmetry:

Here is the modified biorthogonal wavelet transformation after thequantization step:

I We then add 127 to each element of each block, and then applyEBCOT encoding to the result.

I That is quite a substantial savings over the original 320000 bitsneeded to represent the original image and better than a 50%improvement over JPEG.

I Recall that JPEG needed 57231 bits or 1.43078 bits per pixel.I Let’s look at the compressed images:

Original:

PSNR = 39.54711669.

PSNR = 34.68773088.

JPEG2000:

PSNR = 39.54711669.

Corner enlargement - original:

Corner enlargement - JPEG:

Corner enlargement - original:

Corner enlargement - JPEG2000:

TODAY’S SCHEDULE

9:00-10:15 Lecture Nine: The Lifting Method10:15-10:30 Coffee Break (OSS 237)10:30-11:45 Lecture Ten: JPEG and JPEG200011:45-12:00 ⇒Evaluations/Wrap Up12:00-1:00 Lunch (Cafeteria)

JPEG and JPEG2000 · BASIC JPEG NAIVE ALGORITHM I Given an N ×N image A (A square and N is...

Documents

Transcript of JPEG and JPEG2000 · BASIC JPEG NAIVE ALGORITHM I Given an N ×N image A (A square and N is...