MMN lec4-Image v2.ppt - ce.sharif.educe.sharif.edu/courses/89-90/2/ce873-1/resources/root/Class...

Image (Fundamentals, Compression Techniques & Standards)

Hamid R. RabieeMostafa Salehi, Fatemeh Dabiran, Hoda Ayatollahi

Spring 2011

Digital Media Lab - Sharif University of Technology2

Outlines

²Image Fundamentals² Sampling, Quantization

²Image Compression Techniques

²Image Coding Standards² JPEG² JPEG2000

Fundamentals of an Image

² An image consists of pixels (picture elements)

² Each pixel has typically 8-bits and represents luminance (and

color)

² Image Representation² Color spaces

²RGB (red-green-blue)

²CMY (cyan-magenta-yellow)

²YUV

² Greyscale (e.g. 8-bit per pixel)

² Binary (1-bit per pixel)3 Digital Media Lab - Sharif University of Technology


² A TV frame is about 640x480 pixels; each pixel has 8-bits =>

Total image size: 640×480*3=921,600 bytes or ≈7.4Mbits

At 30 frames per second, this would be ≈ 220Mbits/second

Do we need all these bits? No!

4 Digital Media Lab - Sharif University of Technology


² An example of eye’s sensitivity to luminance:

²We can build a perceptual model² Only code what is important to the Human Visual System (HVS)

²Usually a function of spatial frequency

5

An image with 8-bits/pixel An image with 6-bits/pixel An image with 4-bits/pixel

Digital Media Lab - Sharif University of Technology

Sampling and Quantization

² Computer handles “discrete” data.² Sampling² Sample the value of the image at the nodes of a

regular grid on the image plane.

² A pixel (picture element) at (i, j) is the image intensity value at grid point indexed by the integer coordinate (i, j).

² Quantization² Is a process of transforming a real valued sampled

image to one taking only a finite number of distinct values.

² Each sampled value in a 256-level grayscale image is represented by 8 bits.

6

020

4060

80

0

50

1000

50

100

150

200

250

columnsrows

inte

nsity

y x

I(x,y)

y

x


Examples of Sampling

256x256

64x64

16x16


Examples of Quantizaion

8 bits / pixel

4 bits / pixel

2 bits / pixel


IMAGE COMPRESSION PRINCIPLES

Winter 2011


Image Compression

What is image compression?

² Image compression is the art/science of finding efficient

representations for digital images in order to:

² Reduce the memory required for their storage,

² Reduce required bandwidth or time for their transfer across

communication channels

² Increase the effective data transfer rate when reading from storage

devices


Image Compression

Types of Image Compression² Lossless (Reversible, Bit-preserving)

²The image after compression/ decompression is numerically

identical to the original image on a pixel-by-pixel basis.

²Only the statistical redundancy is exploited to achieve compression

² Lossy (Irreversible)

²The reconstructed image contains degradations relative to the

original image

²The statistical redundancy as well as the perceptual irrelevancy of

the image data are exploited


Requirements for Compression Algorithms

² Objectives

² Minimize the complexity of the encoding and decoding process

² Ensure a good quality of decoded images

² Achieve high compression ratios

² Other general requirements

² Independence of specific size and frame rate

² Support various data rates


Some Coding Techniques

² Only for review² Run-Length Encoding

² Huffman Coding

² Arithmetic Coding


Huffman Coding

² Huffman coding uses longer code-words for symbols with

smaller probabilities and shorter code-words for symbols

that often occur.

² Example:

s1

s2

s3

s4

0.125

0.125

0.25

0.5

0.25

0.51

0

0

1

0

1

(01)

(1)

(001)

(000)


Run-Length Encoding

² a data compression algorithm that is supported by most bitmap file formats,

such as TIFF, BMP

² RLE works by reducing the physical size of a repeating string of characters.

² This repeating string, called a run, is typically encoded into two bytes.

² The first byte represents the number of characters in the run and is called the run

count

² The second byte is the value of the character in the run, which is in the range of 0 to

255, and is called the run value.

² There are various and different ways in RLE!

² We explain the algorithm that is used in JPEG in future slides


AAAAAAbbbXXXXXt → 6A3b5X1t

Variants on Run-Length Encoding

² There are a number of variants of run-length encoding

²We will discuss about one of them which is used in JPEG

Coding!² It is a bit different from previous slide definition


Arithmetic Coding

² Arithmetic coding represents a input symbol string as a

small interval in [0, 1)

² The size of the interval equals

P(sk1) P(sk2) P(sk3) … P(skn)

²We can then represent the interval with a binary code.


Arithmetic Coding; an example

² A source output symbols {A, B, C, D, E, F, $}. $ is the termination

symbol.

² Their probabilities are as follows:

P(A) = 0.2P(B) = 0.1P(C) = 0.2P(D) = 0.05P(E) = 0.3P(F) = 0.05P($) = 0.1

1

0

0.20.3

0.50.55

0.850.9

18Digital Media Lab - Sharif University of Technology

IMAGE CODING STANDARDSWinter 2011


Image Compression Standards

² GIF (Graphics Interchange Format)

² PNG (Portable Network Graphics)

² JPEG (Join Picture Expert Group)


GIF

² GIF (Graphics Interchange Format)² Indexed image format: up to 256 colors per image, chosen from a variable palette

² One color index can indicate transparency.

² Uses lossless LZW compression of data bytes.

² Optional interlacing capability.

² GIF became very popular in the early days of the Web.

² Supported by NCSA Mosaic.

² Pretty good compression.

² Most displays then were indexed rather than true color.

² Today it’s still good for diagrams, cartoons, and other non-photographic images.

² Lossless encoding good for sharp edges (doesn’t blur)

²


PNG

² PNG (Portable Network Graphics)

² Supports true color, grayscale, and palette-based (8 bit) color maps

² Uses DEFLATE algorithm

² As used in gzip

² LZ77 algorithm with Huffman coding (You know it!)

² Patent free.


JPEG

² JPEG: Joint Photographic Expert Group — an international standard

in 1992.

² Works for both color and grayscale images.

² Applications include satellite, medical imaging, general photography ...

² Good for compressing photographic images

² Gradual changes in color

² Not good for graphics

² Sharp changes in color

² Compression ratio of 10:1 achievable without visible loss


JPEG Modes

² Lossless JPEG² Coding: Predictive, sequential.

² Resolution: From 2 bits/pixels to 16 bits/pixels

² Huffman coding or Arithmetic coding; Four DC tables

² Interleaved and non-Interleaved scans

² Sequential (Baseline) JPEG² Coding: DCT-based, sequential.

² Resolution: 8 bits/pixels

² Huffman coding; Two AC and two DC tables



JPEG Modes (cont.)

² Progressive JPEG² Coding: DCT-based, progressive.

² Resolution: 8 or 12 bits/pixels

² Huffman coding or Arithmetic coding; Four DC tables


² Hierarchical JPEG² Coding: DCT-based or lossless process.

² Multiple frames (non-differential and differential)



Sequential vs. Progressive


The System of JPEG Encoder

8x8 block DCT Quantization

Quantizationtables

DPCM

Run length coding (RLC)

Entropycoding

HeaderTables

Data

DC

AC


YUV

JPEG Blocks in brief

² Color Space Conversion (Not a block indeed) : Convert RGB (24 bit)

data to YUV

² Blocking: Divide each image up into 8x8 tiles

² Transformation: Convert to frequency space using a two-dimensional

DCT

² Quantization (Only in lossy mode): Quantize the frequency space, using

more bits for the lower frequencies

² Coding: Encode the quantized values using Run-length encoding and

Huffman coding in a zig-zag manner.


Color Space Conversion

² JPEG first converts RGB to YUV or YCrCb.

² Advantages:² Reduce correlation between components² Improve coding efficiency² Allow visually relevant quantization

² Two Transforms:² Irreversible colour transform (ICT) ² Reversible colour transform (RCT)


DCT QuantizationDPCM

RLC

Entropy Coding

BlockingRGB to YUV

Color Space Conversion

² JPEG first converts RGB to YUV or YCrCb.

² Advantages:² Reduce correlation between components² Improve coding efficiency² Allow visually relevant quantization

² Two Transforms:² Irreversible colour transform (ICT) ² Reversible colour transform (RCT)



RLC

Entropy Coding

BlockingRGB to YUV

Blocking

² The original image is first divided into a number of smallersub images or blocks which usually are processedindependently of one another.

²Why do we use blocking?² Correlation

² Using Stationary features of Signal

² Simple Implementation (Why?)



RLC

Entropy Coding

BlockingRGB to YUV

Original 8 × 8 Image Block

² The following example represents an 8 × 8 block (Y)of the original image

LENNA.


DCT (Discrete Cosine Transform)

² DCT converts each image 8x8 block into another 8x8 block

² Mean Subtracted Image Block: The value of 128

is subtracted from each pixel prior to the application

of the discrete cosine transform (DCT). This places the

DC coefficient (the top-left corner coefficient that is

proportional to the average brightness of the block) in

the range (–1024, +1016).



RLC

Entropy Coding

BlockingRGB to YUV

DCT of 8 × 8 Image Block

² The JPEG forward DCT is defined as


DCT (Spatial Frequency) (cont.)

u

v


Quantization

² Recall that quantization can be used to collapse the input into

smaller number of values.

² For DCT coefficients, we want to quantize different

coefficients in different details:² DC coefficient should have the most quantization levels.

² Quantization for AC coefficients can be coarser and coarser as the

frequency increases.



RLC

Entropy Coding

BlockingRGB to YUV

Quantization Matrices

Y quantization table

17 18 24 99 99 99 99 9918 21 26 66 99 99 99 9924 26 56 99 99 99 99 9947 66 99 99 99 99 99 9999 99 99 99 99 99 99 9999 99 99 99 99 99 99 9999 99 99 99 99 99 99 99

U, V quantization table

Fq(u,v) = round(F(u,v)/Q(u,v))Quantization:

Fr(u,v) = Fq(u,v) * Q(u,v)De-quantization:

Bet

ter q

uant

izat

ion

at lo

w fr

eque

ncie

s

Coa

rse

quan

tizat

ion

at

high

freq

uenc

ies


² Quantization step size = 20

² DCT unquantized coefficient = 13.56

² Scaled coefficient =13.56/20 = 0.68

² Scaled, quantized (rounded) coefficient = 1

² Dequantized coefficient (Decoder) = 1 × 20 = 20

Quantization ExampleQuantization Example


² After normalization, the coefficients are quantized by rounding off to

the nearest integer.

² The normalization/quantization process typically results in many zero-

valued coefficients which can be coded efficiently.

Normalized/Quantized CoefficientsNormalized/Quantized Coefficients


Quantization (Dequantization) Example

Quantization

Dequantization


DCT

DC & AC Coefficients

² The top-left coefficient in the 2-D DCT array is referred to as the DC coefficient and is proportional to the average brightness of the spatial block.

² DC coefficient is treated separately from the 63 AC coefficients:² DPCM for DC coefficient encoding² Run-Length Coding for AC coefficients encoding


DC Coefficients Encoding

² Differential Pulse Code Modulation (DPCM): Take difference with the

DC coefficient of the previous block.



RLC

Entropy Coding

BlockingRGB to YUV

(These variable length codes are standardized for jpeg)

DC Coefficients Encoding (cont.)

² This difference is first assigned to one of the 12 categories values where

the values in category k are in the range and

² A set of Huffman codes with a maximum codeword length of 16 bits is

used to specify the different categories.

43

)12,2( 1 −± − kk 110 ≤≤ k


The Size Table

7 will be coded as (3, 7)• The first number 3 is Huffman

coded.• The second 7 is encoded as 1’s complement 111.(-7 will be 000)

00-1, 11

-3, -2, 2, 32-7 …-4, 4, …,73

…-2047,…,-

1024,1024,…,204711

AC Coefficients Encoding

² AC coefficients are not differentially encoded, Instead, we first do run-

length coding.

² Order the coefficients in zig-zag order, then, run length coding

44


RLC

Entropy Coding

BlockingRGB to YUV


Zig-

zag

scan

ning

Quantized DCT AC coefficients in zigzag order:

Entropy CodingEntropy Coding

² Entropy coding of the baseline encoder is accomplished in two stages.

² The first stage is the translation of the quantized DCT coefficients into an

intermediate set of symbols.

² In the second stage, variable length codes are assigned to each symbol.

² For the JPEG standard a symbol is structured in two parts:

² a variable length code (VLC) for the first part, normally referred to as

symbol-1

² followed by a binary representation of the amplitude for the second part,

symbol-2

45


RLC

Entropy Coding

BlockingRGB to YUV


Symbol-1

² For each nonzero AC coefficient in zigzag scan order, symbol-1 is described as

a two-dimensional event of (RUN, CAT), sometimes called (RUN, SIZE).

² For the baseline encoder, CAT is the category for the amplitude of a

nonzero coefficient in the zigzag order (it is identical to AC coefficients

table)

² RUN is the number of zeros preceding this nonzero coefficient


18: (Run,CAT)=(0,5)Code word from standard table=11010-21:(Run,CAT)=(0,5)Code word from standard table=11010

Symbol-2

² The symbol-2 is the lower order bits of the value of coefficient


18=000010010-21=111…1101010

Final Code: symbol-1 code+ symbol-2 code18=1101010010-21=1101001010

JPEG DecodingJPEG Decoding

² Decoding is simply the reverse of encoding.² Reverse the Huffman, RLE encodings.² Dequantization² Apply inverse DCT (IDCT):


JPEG Compression ratio

² Compression ratio depends on how large the values in the quantization

matrix are.

² 10:1 achievable without noticeable loss.

² 100:1 achievable, but artifacts are noticeable.


JPEG File FormatJPEG File Format

Start of image Frame End of image

Tables Header Scan Scan Scan…

Tables Header Segment Restart Segment ….

Block Block Block


The JPEG Extended SystemThe JPEG Extended System--JPEGJPEG20002000

² The basic mode of JPEG supports sequential coding (the

order is from top to bottom and left to right).

² The JPEG extended system enhances the baseline system

to satisfy a broader range of applications.


The JPEG Extended System FeaturesThe JPEG Extended System Features

² 12-bit/pixel input image precision

² Sequential progressive build-up

² Hierarchical progressive build-up

² Arithmetic coding

² Adaptive quantization


Why another standard?Why another standard?


² Low Bit-Rate Compression: At low bit-rates (e.g. below 0.25

bpp for highly detailed gray-level images) the distortion in

JPEG becomes unacceptable.

² Lossless and Lossy Compression: Need for standard, which

provide lossless and lossy compression in one code-stream.

² Large Images: JPEG doesn't compress images greater then

64x64K without tiling.

JPEG JPEG 2000 2000 FeaturesFeatures

² Can be both lossless and lossy

² Improves image quality

² Uses Wavelete Coding instead of DCT; a layered file structure

² Progressive transmission

² Progressive rendering

² File structure flexibility

² Could use for a variety of applications

² Many functionalities; e.g. Region-Of-Interest


Wavelet CodingWavelet Coding

² The wavelet transform uses basis functions that are scaled

and translated versions of a single prototype function (the

basic or mother wavelet).

² The wavelet transform can be viewed as a bank of

bandpass filters, where the filters have increasing

bandwidth as the center frequency increases


Wavelet Coding (cont.) Wavelet Coding (cont.)

² Details


Wavelet Coding (cont.) Wavelet Coding (cont.)


(1) Stage1 of DWT (2) Stage2 of DWT (3) Stage3 of DWT

Discrete Wavelet Transform on Baboon image

² In JPEG2000 multiple stages of the DWT are performed.

² JPEG2000 supports from 0 to 32 stages. For natural images,

usually between 4 to 8 stages are used.

58

RegionRegion--ofof--interest (ROI)interest (ROI)

² A ROI is a part of an image that is encoded with higher quality than the

rest of the image (the background). The encoding is done in such a way

that the information associated with ROI precedes the information

associated with the background.


ROI Example

² Region of interest with better quality


JPEGJPEG2000 2000 at at 00..125 125 bppbpp

C. Christopoulos, A. Skodras, T. Ebrahimi, JPEG2000 (online tutorial)60 Digital Media Lab - Sharif University of Technology

JPEG at JPEG at 00..125 125 bppbpp (enlarged)(enlarged)

C. Christopoulos, A. Skodras, T. Ebrahimi, JPEG2000 (online tutorial)61 Digital Media Lab - Sharif University of Technology

References

² Mohammed Ghanbari , “Standard Codecs: Image Compression to

Advanced Video Coding”, Institution of Electrical Engineers, 2003.



Next Session

Video

63

64

Any Question

Thank you!


MMN lec4-Image v2.ppt - ce.sharif.educe.sharif.edu/courses/89-90/2/ce873-1/resources/root/Class...

Documents

Transcript of MMN lec4-Image v2.ppt - ce.sharif.educe.sharif.edu/courses/89-90/2/ce873-1/resources/root/Class...