Lossless JPEG transcoding

ECE 533 Project Report Fall 2006

Daniel Sanchez

1

Index Index....................................................................................................... 1 Introduction.............................................................................................. 2

The problem of transcoding ...................................................................... 2 Performed work ...................................................................................... 2

Full-featured JPEG encoder/decoder ............................................................. 2 Code base.............................................................................................. 3 Implemented functions ............................................................................ 3 Functionality .......................................................................................... 4 Comparing the codec with the MATLAB imread/imwrite functions ................... 4

Lossless JPEG transcoder ............................................................................ 5 Results of lossless transcoding .................................................................. 6

Lossless JPEG rotation................................................................................ 7 Results of the lossless rotation.................................................................. 7 Other Applications................................................................................... 9

References ............................................................................................... 9

2

Introduction This is the final report of the “Lossless JPEG transcoding” project, created by Daniel Sanchez for the “ECE533: Digital Image Processing” course, taught at the University of Wisconsin-Madison in the fall 2006 semester. The original goal of the project, as stated in the preliminary proposal, was to develop a method to achieve lossless JPEG transcoding. However, the requirements of the project have caused the final project to be much broader in scope. Nevertheless, the lossless JPEG transcoding algorithm has been successfully implemented.

The problem of transcoding

Some image processing and image transmission systems require images to be decoded and re-encoded again, with little or no change at all. With lossy compression algorithms such as JPEG, this process results in a loss of quality, even if the recompressed image is not modified.

Performed work

The initial approach for the project was to reuse the code of an existing JPEG codec. Unfortunately, I could not find a full-featured baseline JPEG codec written in MATLAB that could be modified as desired. MATLAB provides support for reading and writing JPEG images through the imread/imwrite commands, but does not allow the algorithm itself to be modified. Other programming languages were considered, with little success: very few usable things were available in the Java language (as with MATLAB, Sun provides standard routines to access JPEG files but their source code is not available). In the C programming language, the predominance of the IGJ developed codec is remarkable. This codec is open source, but is focused on being fast and reliable, not educational, so using it as a base would have been difficult. Finally, I divided the work into these stages:

1. Extend the MATLAB JPEG Toolbox developed by Phil Sallee (http://redwood.ucdavis.edu/phil/) to create a full-featured JPEG codec.

2. Modify the codec to perform lossless transcoding on arbitrary images. 3. Develop a program to perform lossless rotation of images, taking advantage

of the previously developed software. These three points will be explained in detail, and results that support the correctness of the developed algorithms will be presented. Full-featured JPEG encoder/decoder When dealing with the JPEG algorithm, it is always useful to have its basic block diagram in mind:

RGB->YCbCr

Level shifting DCT Q

DPCM DC Huffman

ZigZag AC Huffman

JFIF

File

RGB

Image

JPEG Encoder

DC

AC

3

Code base

As I mentioned before, I have not implemented the complete JPEG coding/decoding process. Instead, I have extended an existing set of functions: the MATLAB JPEG Toolbox, created by Phil Sallee, which can be downloaded at http://redwood.ucdavis.edu/phil/. These functions implement the ‘hard’ part of the codec: they can partially encode and decode up to the quantization stage. The information of a partially encoded/decoded JPEG image is held in a so-called JPEG object, a MATLAB structure that holds:

• General information about the image (size, color model, comments). • The quantized DCT coefficients. • The AC and DC quantization matrices. • The Huffman tables used. • Information about each color channel (e.g. its vertical and horizontal

subsampling ratios, among other things). • Coding flags to allow optimized Huffman coding and progressive encoding.

The set of functions provided by the toolbox are: • jpeg_read: Reads a JPEG file and returns its corresponding JPEG object. • jpeg_write: Encodes a JPEG object into a file. • quantize/dequantize: Perform the quantization/dequantization stage on a

block, given the quantization matrices. • Some other functions that won’t be used.

It is worth mentioning that jpeg_read and jpeg_write are not written from scratch either: they are heavily based on the IJG code, and so those two functions are written in C and used from MATLAB via MEX files. I have also used the MATLAB “colorspace” function written by Pascal Getreuer (http://www.math.ucla.edu/~getreuer/) to perform the conversions between the YCbCr and RGB color models in a fast and accurate way.

Implemented functions

Based on the aforementioned functions, I have implemented the following two ones:

• image=jpeg2im(jObj) is passed a JPEG object as a parameter and completes the decoding process, returning a RGB image.

• jObj=jpeg2im(image,quantAC,quantDC,sample) generates a JPEG object from an image, the AC and DC quantization tables and the subsampling factor for the Cr and Cb channels (e.g. if sample=2, the subsampling follows the 4:2:2 format, if 1 it follows the 4:4:4 format)

Level shifting

YCbCr->RGB IDCT IQ

IDPCM DC Huffman

DeZigZag AC Huffman

JPEG Decoder

DC

AC

RGB

Image

JFIF

File

4

Functionality

The jpeg2im and im2jpeg functions can be combined with the jpeg_read and jpeg_write ones to fully encode and decode JPEG images. The codec implements most of the features of the baseline JPEG specification:

• Reads/writes images of arbitrary size (width and height don’t need to be a multiple of 8)

• Reads/writes true color images (8-bits/channel) • Writes files using optimized Huffman coding by default, yielding a smaller

filesize than with the standard coding. However, the codec has some limitations:

• It only decodes images encoded in the YCbCr color model (this model is almost always used in practice, but the JPEG format still allows the usage of other color models)

• It only works with images with the following color subsampling modes: 4:4:4 (i.e. no subsampling) and 4:2:2 (i.e. vertical and horizontal subsampling of Cb and Cr by a factor of 2). Note that 4:2:2 is used in most cases.

Comparing the codec with the MATLAB imread/imwrite functions

To see how the codec performs, I have tested it against the standard imread/imwrite functions provided by MATLAB. This test can be run by executing the compareAlgorithms.m batch file. A JPEG file is decoded, reencoded and redecoded by both algorithms, and the difference between the first and the second decoded images is considered as the noise. The results are the following:

Original image Noise with MATLAB imread/write (x15)

Noise with implemented codec (x15)

Size: 490x733px Filesize of original image: 27.7KB

SNR = 33.3dB Filesize of reencoded image: 27.7KB

SNR = 25.1dB Filesize of reencoded image: 26.5KB

It can be observed that my implementation leads to a more noisy reencoding process. However, we will address this problem later with the lossless transcoding process.

5

Finally, note that the resulting filesize is smaller when reencoding with the implemented codec. This is due to the use of optimized Huffman coding and does not imply a loss of detail in general. Lossless JPEG transcoder Once we have the implemented the codec, it is quite straightforward to modify it to achieve lossless transcoding. In a transcoder, the first block is the decoder, which can be followed by another system, and the last one is the encoder. For now, we will restrict our scenario by assuming that no intermediate processing is done between decoder and encoder. Therefore, the goal is to achieve the same file as the original at the end of the encoder. This might seem useless at a first glance, but it still has some real applications (e.g. in transmission systems). One key issue that is still left to explain is why a regular decoding/reencoding process is lossy. We have presented an example that shows that this is the case, however, a theoretical explanation might not be obvious to see. The key here are the color model conversion processes (YCbCr->RGB and RGB->YCbCr). They introduce some quantization, and so the decoding/reencoding process is subject to the loss of quality. The modified block diagram that has been implemented to achieve lossless transcoding is:

In the decoder, the image is fully recoded and then partially reencoded. The coefficients of the reencoded image will have a small quantization error. These errors are calculated by subtracting the original coefficients to the reencoded ones. The errors are encoded and output like the decoded image. The encoder will then use the errors to correct the reencoded coefficients and achieve a lossless

Level shifting

YCbCr->RGB IDCT IQ

IDPCM DC Huffman

DeZigZag AC Huffman

JPEG Lossless Transcoder - Decoder

DC

AC

RGB

Image

RGB->YCbCr

Level shifting DCT Q Lossless

Compression +

-

RGB->YCbCr

Level shifting DCT Q

RGB

Image

JPEG Lossless Transcoder - Encoder

Difference

coefficients Lossless

Decompression

Difference

coefficients

JFIF

File

DPCM DC Huffman

ZigZag AC Huffman

JFIF

File

6

compression. This scheme has some formal similarities with the linear predictive coders (although its application is entirely different): a reencoded block can be seen as an estimation of the block, and the error is transmitted to correct that estimation. As regards to the difference (or error) block coefficients, we will encode them losslessly and in an efficient way. To do that, I have chosen to use the JPEG file format with optimized Huffman encoding. This has the following advantages:

• Very good compression ratios (up to 100:1) are achieved, because the error coefficients have limited, small values.

• The JPEG format is used to carry extra information about the original image, such as the original quantization matrices and comments (which can be valuable or necessary to keep, e.g. EXIF tags).

Note that we are using the JPEG file format and the Huffman encoding, nothing else from the JPEG algorithm, therefore the compression scheme is lossless. The lossless transcoder has been implemented via two functions, lDec and lEnc, which implement the decoder and the encoder respectively. The difference coefficients can either be kept in memory (in a JPEG object) or saved to disk (in a JPEG file).

Results of lossless transcoding

A demo file (losslessTranscodeDemo.m) has been created to test the lossless transcoder. This demo reencodes a previously encoded JPEG image using the lossless transcoder and checks that it is identical to the original one. In addition, it degrades an image by reencoding with imwrite/imread to see the effects. The results are the following:

Original image imread/imwrite: Noise when reencoding 50 times

(x15)

Lossless transcoder: Generated file with the

error coefficients (contrast was increased)

Image size: 512x512px Original filesize: 43.1KB

SNR=33.2dB Fileisize of the reencoded image: 43.1KB

Filesize of the transcoded image: 42.3KB Filesize of the difference coefficients: 2.1KB

With the lossless transcoder, SNR = ∞ , since the original and reencoded images are exactly the same. Again, note that the filesize of the transcoded image is smaller because of the optimized Huffman coding. This does not imply a loss of quality.

7

Lossless JPEG rotation The third and final part of the project consists on an implementation of a lossless rotation algorithm for JPEG images. In this case, we will deal with the quantized blocks directly. This approach is much faster and efficient that decoding the image, rotating it and reencoding it losslessly as in the previous section. I will first describe how to rotate an image by manipulating its blocks in the transformed domain directly. The lossless rotator implements a 90º counterclockwise rotation, which can be seen mathematically as:

g(x,y) f(M y,x)= −

where g(x,y) is the rotated image and f(x,y) is the original, MxN image. The 8x8 blocks have to be reorganized in the same way, so that they correspond to the same part of the image as before. It is left what transformation we need to do rotate the contents of a particular block. Recall that the DCT transform is:

7 7

x 0 y 0

7 7

x 0 y 0

1f(x,y) u v 0

4F(u,v)

1 (2x 1)u (2y 1)vf(x,y)cos cos 0 u,v 7,u v 0

8 16 16π π

= =

= =

⎧= =⎪

⎪= ⎨+ +⎪ ≤ ≤ + >

⎪⎩

∑∑

∑∑

Ignoring the scaling of the DC coefficient, we can write:

7 7

x 0 y 0

1 (2x 1)u (2y 1)vF(u,v) f(x,y)cos cos

8 16 16π π

= =

+ += ∑∑

Taking into account the transformation, 7 7

x 0 y 0

1 (2x 1)u (2y 1)vG(u,v) f(7 y,x)cos cos

8 16 16π π

= =

+ += −∑∑

7 7

x 0 n 0

1 (2x 1)u ( 2n 14 1)vG(u,v) f(n,x)cos cos

8 16 16π π

= =

+ − + += ∑∑

7 7

x 0 n 0

1 (2x 1)u ( 2n 1 16)vG(u,v) f(n,x)cos cos

8 16 16π π

= =

+ − − += ∑∑

7 7

x 0 n 0

1 (2x 1)u (2n 1)vG(u,v) f(n,x)cos cos cos(v )

8 16 16π π π

= =

+ += ∑∑

And finally, vG(u,v) F(v,u) ( 1)= ⋅ −

Applying this transformation to each block effectively rotates its contents. The lossless rotation algorithm has been implemented with one function, losslessRotate(inFile,outFile), that performs a 90º counterclockwise rotation by:

• Reordering the blocks • Performing the derived operation in each block to rotate their contents • Transposing the quantization matrices (they are not used when transcoding,

but will be used when decompressing)

Results of the lossless rotation

The demo file to test the lossless rotation is name losslessRotateDemo.m. It uses the following image throughout the test:

8

The image is first rotated 4 times and resaved at each rotation using the imread/imwrite commands. This yields a degraded image, with the difference between this and the original being (amplified 15 times):

The signal to noise ratio is only SNR = 78.1 = 18.9 dB this time. It is clear that the degradation that an image suffers when being rotated is significantly greater than when it is just reencoded. Following this experiment, the losslessRotate function is applied 4 times to the image. The resulting image is loaded and compared with the original one, verifying that both are the same. Finally, a single rotation is performed to check that the operation works correctly. The rotated image is:

9

Other Applications

Lossless JPEG transcoding has many other relevant applications besides reencoding and rotating. For example, it can be used by editing software to avoid a quality loss in the unedited parts of the image. With some additional modifications, it can also be used to perform other simple geometric transformations on JPEG compressed images, like cropping or mirroring. Finally, note that most of the images that involve transformations (including rotation) require that the input image has dimensions that are a multiple of 8, because they operate on undecoded blocks. References

1) R. C. Gonzalez, R.E. Woods, “Digital Image Processing”, 2nd Edition. 2) Independent JPEG Group - http://www.ijg.org/ : Their widely used software

package includes an utility that performs lossless transcoding, jpegtran. 3) JPEGClub - http://jpegclub.org/ : Offers some modifications to jpegtran

program that allow lossless rotation/crop on JPEG compressed images. 4) MATLAB JPEG Toolbox by Phillip Sallee -

http://redwood.ucdavis.edu/phil/demos/jpegtbx/jpegtbx.htm : The developed JPEG codec extends this toolbox.

5) colorspace function by Pascal Getreuer - http://www.math.ucla.edu/~getreuer/ : The developed JPEG codec uses this function to convert between the YCbCr and RGB color models.