Compression

COMPRESSION AND DECOMPRESSION

04/22/23 1A.Aruna, Assistant Professor, Faculty of Information Technology

IntroductionVideo and audio have much higher

storage requirements than textData transmission rates (in terms of

bandwidth requirements) for sending continuous media are considerably higher than text

Efficient compression of audio and video data, including some compression standards

COMPRESSIONCompression is a reduction in the number of bits needed to represent data.

save storage capacityspeed file transferdecrease costs for storage hardware and network bandwidth.

04/22/23 A.Aruna, Assistant Professor, Faculty of Information

Technology

3

04/22/23 4

Multimedia CompressionAudio, image and video require vast

amounts of data320x240x8bits grayscale image: 77Kb1100x900x24bits color image: 3MB640x480x24x30frames/sec: 27.6 MB/sec

Low network’s bandwidth doesn't allow for real time video transmission

Slow storage or processing devices don't allow for fast playing back

Compression reduces storage requirementsA.Aruna, Assistant Professor,

Faculty of Information Technology

04/22/23 5

Classification of TechniquesLossless: recover the original

representation.Mechanisms:

Packbits encoding(Run Length Encoding)CCITT Group 3 1DCCITT Group 3 2DCCITT Group 4Lempel – Ziv and Welch Algorithm LZW

A.Aruna, Assistant Professor, Faculty of Information Technology

Classification of TechniquesLossy: recover a representation

similar to the original onegraphics, audio, video and imagesMechanisms:

Joint Photographic Experts Group Moving Picture Experts GroupIntel DVI CCITT H.26l Video Coding AlgorithmFractals


Technology

6

BINARY IMAGE COMPRESSION Used for Documents (Black & White) Continuous Tone Information

Office & Business Document Handwritten TextLine GraphicsEngineering DrawingScanning Documents


Technology

7

BINARY IMAGE COMPRESSION Scanning Process

Scanline – Top to Bottom, Left to rightComposed of Various ObjectsCCD Array Sensor – B/W Dots- Memory

Eg: Faxing – 1 Page – 20 seconds


Technology

8

Packbits Encoding or Runlength Encoding

Simplest and earliest Data Compression Schemes

Binary Image Consecutive Repeated – Two Bytes

First Byte – No.of times Character is Repeated

Second Byte – Character itself


Technology

9

Packbits Encoding or Runlength Encoding


Technology

10

CCITT Group 3 1-D Compression

Based on Runlength Encoding Facsimile & Early document Imaging

System Large Size even after Compression Modified Runlength encoding is

Huffman Encoding


Technology

11


Huffman Encoding Variable Length Encoding Shorter Code – Frequently Longer code – Less Frequently

Probability of Occurrence of white and black Pixel

It is based on a coding Tree, which is constructed based on the probability of occurrences of white pixels and black pixels in the run length or bit streams


Technology

12


probability of occurrences of bit stream of length Rn = P(Rn)


Technology

13


Technology

14


Technology

15

Large Pixel Sequences


Technology

16

Example 16 White Pixel = 101010 - Frequently16 Black Pixel = 0000010111Quicker decoding Tree Structure to be constructed


Technology

17

Makeup code : length in Multiples of 64 pixels

Terminating code: length less than 64 pixels

132 white pixels is 100101011Make up code for 128 = 10010Terminating code for 4 = 1011


Technology

18

Coding Tree 16 white 101010 and black pixel 0000010111


Technology

19

1

0

10

1

0

04/22/23 20

JPEG Modes of OperationSequential DCT: the image is encoded in

one left-to-right, top-to-bottom scanProgressive DCT: the image is encoded

in multiple scans (if the transmission time is long, a rough decoded image can be reproduced)

Hierarchical: encoding at multiple resolutions

Lossless : exact reproductionA.Aruna, Assistant Professor,


04/22/23 21

Furht at.al. 96

JPEG Block Diagrams

A.Aruna, Assistant Professor, Faculty of Information

Technology

04/22/23 22

JPEG EncoderThree main blocks:

Forward Discrete Cosine Transform (FDCT)QuantizerEntropy Encoder

Essentially the sequential JPEG encoderMain component of progressive, lossless

and hierarchical encoders For gray level and color images


Technology

04/22/23 23

Sequential JPEGPixels in [0,2p-1] are shifted in [-2p-1,2p-1-1] The image is divided in 8x8 blocksEach 8x8 block is DCT transformed

0for 1

0for 2

1)(

0for 1

0for 2

1)(

16)12(cos

16)12(cos),(

2)(

2)(),(

7

0

7

0

v

vvC

u

uuC

vyuxyxfvCuCvuFx y


Technology

04/22/23 24

DCT CoefficientsF(0,0) is the DC coefficient: average

value over the 64 samplesThe remaining 63 coefficients are

the AC coefficientsPixels in [-128,127]: DCTs in

[-1024,1023]Most frequencies have 0 or near to 0

values and need not to be encodedThis fact achieves compression


Technology

04/22/23 25

Quantization StepAll 64 DCT coefficients are quantized

Fq(u,v) = Round[F(u,v)/Q(u,v)] Reduces the amplitude of coefficients

which contribute little or nothing to 0Discards information which is not

visually significantQuantization coefficients Q(u,v) are

specified by quantization tablesA set of 4 tables are specified by JPEG


Technology

04/22/23 26

Quantization Tables

for (i=0; i < 64; i++) for (j=0; j < 64; j++) Q[i,j] = 1 + [ (1+i+j) quality];

quality = 1: best quality, lowest compressionquality = 25: poor quality, highest compression

Furht at.al. 96


Technology

04/22/23 27

AC CoefficientsThe 63 AC coefficients

are ordered by a “zig-zag” sequence

Places low frequencies before high frequencies

Low frequencies are likely to be 0

Sequences of such 0 coefficients will be encoded by fewer bits

Furht at.al. 96


Technology

04/22/23 28

DC CoefficientsPredictive coding of DC CoefficientsAdjacent blocks have similar DC intensitiesCoding differences yields high

compression


Technology

04/22/23 29

Entropy EncodingEncodes sequences of quantized DCT

coefficients into binary sequences AC: (runlength, size) (amplitude)DC: (size, amplitude)runlength: number consecutive 0’s, up to 15

takes up to 4 bits for coding(39,4)(12) = (15,0)(15,0)(7,4)(12)

amplitude: first non-zero valuesize: number of bits to encode amplitude 0 0 0 0 0 0 476: (6,9)(476)


Technology

04/22/23 30

Huffman codingConverts each sequence into binaryFirst DC following with ACsHuffman tables are specified in JPEGEach (runlength, size) is encoded

using Huffman codingEach (amplitude) is encoded using a

variable length integer code(1,4)(12) => (11111101101100)


Technology

04/22/23 31

Example of Huffman tableFurht at.al. 96


Technology

04/22/23 32

Furht at.al. 96

JPEG Encoding of a 8x8 block


Technology

04/22/23 33

Compression MeasuresCompression ratio (CR): increases with

higher compressionCR = OriginalSize/CompressedSize

Root Mean Square Error (RMS): better quality with lower RMS

Xi: original pixel valuesxi: restored pixel valuesn: total number of pixels

n

i ii xXn

RMS1

2)(1


Technology

04/22/23 34A.Aruna, Assistant Professor, Faculty of Information

Technology

04/22/23 35

JPEG DecoderThe same steps in reverse order

The binary sequences are converted to symbol sequences using the Huffman tables

F’(u,v) = Fq(u,v)Q(u,v)Inverse DCT

7

0

7

0 16)12(cos

16)12(cos),()()(

41),(

u v

vyuxvuFvCuCyxF


Technology

04/22/23 36

Progressive JPEGWhen image encoding or transmission

takes long there may be a need to produce an approximation of the original image which is improved gradually

Furht at.al. 96


Technology

04/22/23 37

Progressive Spectral Selection

The DCT coefficients are grouped into several bandsLow-frequency bands are firstband1: DC coefficient onlyband2: AC1,AC2 coefficientsband3: AC3, AC4, AC5, AC6 coefficientsband4: AC7, AC8 coefficients


Technology

04/22/23 38

Lossless JPEGSimple predictive encoding

prediction schemesFurht at.al. 96


Technology

04/22/23 39

Hierarchical JPEG Produces a set of images at multiple

resolutions Begins with small images and

continues with larger images (down-sampling)

The reduced image is scaled-up to the next resolution and used as predictor for the higher resolution image


Technology

04/22/23 40

Encoding1. Down-sample the image by 2a in each x, y2. Encode the reduced size image

(sequential, progressive ..)3. Up-sample the reduced image by 24. Interpolate by 2 in x, y5. Use the up-sampled image as predictor 6. Encode differences (predictive coding)7. Go to step 1 until the full resolution is

encoded


Technology

04/22/23 41

Furht at.al. 96


Technology

04/22/23 42

JPEG for Color imagesEncoding of 3 bands (RGB, HSV etc.)

in two ways:Non-interleaved data ordering: encodes

each band separatelyInterleaved data ordering: different

bands are combined into Minimum Coded Units (MCUs)Display, print or transmit images in parallel

with decompression


Technology

04/22/23 43

Interleaved JPEGMinimum Coded Unit (MCU): the smallest

group of interleaved data blocks (8x8)

Furht at.al. 96


Technology

04/22/23 44

Video CompressionVarious video encoding standards:

QuickTime, DVI, H.261, MPEG etcBasic idea: compute motion between

adjacent frames and transmit only differences

Motion is computed between blocks Effective encoding of camera and object

motion


Technology

04/22/23 45

MPEGThe Moving Picture Coding Experts

Group (MPEG) is a working group for the development of standards for compression, decompression, processing, and coded representation of moving pictures and audio

MPEG groups are open and have attracted large participation

http://mpeg.telecomitalialab.comA.Aruna, Assistant Professor,


http://mpeg.telecomitalialab.com/

http://mpeg.telecomitalialab.com/

04/22/23 46

MPEG FeaturesRandom accessFast forward / reverse searchesReverse playbackAudio – visual synchronizationRobustness to errorsAuditabilityCost trade-off


Technology

04/22/23 47

MPEG -1, 2At least 4 MPEG standards finished or

under constructionMPEG-1: storage and retrieval of moving

pictures and audio on storage media352x288 pixels/frame, 25 fps, at 1.5 MbpsReal-time encoding even on an old PC

MPEG-2: higher quality, same principles720x576 pixels/frame, 2-80 Mbps


Technology

04/22/23 48

MPEG-4Encodes video content as objects Based on identifying, tracking and

encoding object layers which are rendered on top of each other

Enables objects to be manipulated individually or collectively on an audiovisual scene (interactive video)

Only a few implementationsHigher compression ratios


Technology

04/22/23 49

MPEG-7Standard for the description of

multimedia content XML Schema for content descriptionDoes not standardize extraction of

descriptionsMPEG1, 2, and 4 make content

availableMPEG7 makes content semantics

availableA.Aruna, Assistant Professor,


04/22/23 50

MPEG-1,2 Compression Compression of full motion video, interframe

compression, stores differences between frames A stream contains I, P and B frames in a given pattern Equivalent blocks are compared and motion vectors

are computed and stored as P and B frames

Furht at.al. 96


Technology

04/22/23 51

Frame StructuresI frames: self contained, JPEG encoded

Random access frames in MPEG streamsLow compression

P frames: predicted coding using with reference to previous I or P frameHigher compression

B frames: bidirectional or interpolated coding using past and future I or P frameHighest compression


Technology

04/22/23 52

Example of MPEG Stream

B frames 2 3 4 are bi-directionally coded using I frame 1 and P frame 5P frame 5 must be decoded before B frames 2 3 4I frame 9 must be decoded before B frames 6 7 8Frame order for transmission: 1 5 2 3 4 9 6 7 8

Furht at.al. 96


Technology

04/22/23 53

MPEG Coding SequencesThe MPEG application determines

a sequence of I, P, B framesFor fast random access code the

whole video as I frames (MJPEG)High compression is achieved by

using large number of B framesGood sequence: (IBBPBBPBB)

(IBBPBBPBB)...A.Aruna, Assistant Professor,


04/22/23 54

Motion EstimationThe motion estimator finds the best

matching block in P, B framesBlock: 8x8 or16x16 pixelsP frames use only forward prediction: a

block in the current frame is predicted from past frame

B frames use forward or backward or prediction by interpolation: average of forward, backward predicted blocks


Technology

04/22/23 55

Motion Vectors

One or two motion vectors per blockOne vector for forward predicted P or B frames

or backward predicted B framesTwo vectors for interpolated B frames

block: 16x16pixles

Furht at.al. 96


Technology

04/22/23 56

MPEG EncodingI frames are JPEG compressedP, B frames are encoded in terms of future

or previous framesMotion vectors are estimated and

differences between predicted and actual blocks are computed These error terms are DCT encoded Entropy encoding produces a compact binary

codeSpecial cases: static and intracoded blocks


Technology

04/22/23 57

MPEG encoder

Furht at.al. 96

JPEG encoding


Technology

04/22/23 58

MPEG Decoder Furht at.al. 96


Technology

04/22/23 59

Motion Estimation Techniques

Not specified by MPEG Block matching techniques Estimate the motion of an nxm block in

present frame in relation to pixels in previous or future framesThe block is compared with a previous or

forward block within a search area of size (m+2p)x(n+2p)

m = n = 16p = 6


Technology

04/22/23 60

Block Matching

Search area in block matching techniquesTypical case: n=m=16, p=6F: block in current frameG: search area in previous (or future) frame

Furht at.al. 96


Technology

04/22/23 61

Cost functions The block has moved to the position that

minimizes a cost functionI. Mean Absolute Difference (MAD)

F(i,j) : a block in current frame G(i,j) : the same block in previous or future

frame (dx,dy) : vector for the search location

dx=(-p,p), dy=(-p,p)

2/

2/

2/

2/

),(),(1),(n

ni

m

mj

dyjdxiGjiFmn

dydxMAD


Technology

04/22/23 62

More Cost FunctionsII. Mean Squared Difference (MSD)

III. Cross-Correlation Difference (CCF)

2/

2/

2/

2/

2),(),(1),(n

ni

m

mj

dyjdxiGjiFmn

dydxMSD

2/1

2

2/1

2 ),(),(

),(),(),(

i ji j

i j

dyjdxiGjiF

dyjdxiGjiFdydxCCF


Technology

04/22/23 63

More cost FunctionsIV. Pixel Difference Classification (PDC)

t: predefined threshold each pixel is classified as a matching

pixel (T=1) or a mismatching pixel (T=0) the matching block maximizes PDC

otherwisetdyjdxiGjiFif

jidydxT

jidydxTdydxPDCi j

0),(),(1

),,,(

),,,(),(


Technology

04/22/23 64

Block Matching TechniquesExhaustive: very slow but accurateApproximation: faster but less

accurateThree-step search2-D logarithmic searchConjugate direction searchParallel hierarchical 1-D search (not

discussed) Pixel difference classification (not discussed here)


Technology

04/22/23 65

Exhaustive SearchEvaluates the cost function at every

location in the search areaRequires (2p+1)2 computations of the

cost functionFor p=6 requires169 computations per

block!!Very simple to implement but very

slow


Technology

04/22/23 66

Three-Step SearchComputes the cost function at the

center and 8 surrounding locations in the search areaThe location with the minimum cost

becomes the center location for the next step

The search range is reduced by half


Technology

04/22/23 67

Three-Step Motion Vector Estimation (p=6)

Furht at.al. 96


Technology

04/22/23 68

Three–Step Search1. Compute cost (MAD) at 9 locations

• Center + 8 locations at distance 3 from center2. Pick min MAD location and recompute MAD

at 9 locations at distance 2 from center3. Pick the min MAD locations and do same at

distance 1 from center• The smallest MAD from all locations indicates

the final estimate• M24 at (dx,dy)=(1,6)• Requires 25 computations of MAD


Technology

04/22/23 69

2-D Logarithic SearchCombines cost function and predefined

threshold TCheck cost at M(0,0), 2 horizontal and 2

vertical locations and take the minimum If cost at any location is less than T then

search is completeIf no then, search again along the

direction of minimum cost - within a smaller region


Technology

04/22/23 70

if cost at M(0,0) < T then search ends! compute min cost at M1,M2,M3,M4; take their min; if min cost < M(0,0)

if (cost less than T) then search ends! else compute cost at direction of minimum cost (M5,M6 in the

example);else compute cost at the neighborhood of min cost within p/2 (M5 in the example)

Furht at.al. 96


Technology

04/22/23 71

Conjugate Direction Search

Repeat find min MAD along dx=0,-1,1 (y fixed): M(1,0) in example find min MAD along dy=0,-1,1 starting from previous min (x

fixed): M(2,2) search similarly along the direction connecting the above mins

Furht at.al. 96


Technology

04/22/23 72

Other Compression Techniques

Digital Video Interactive (DVI)similar to MPEG-2

Fractal Image CompressionFind regions resembling fractalsImage representation at various resolutions

Sub-band image and video coding Split signal into smaller frequency bands

Wavelet-based coding


Technology

04/22/23 73

References B. Furht, S. W. Smoliar, H-J. Zang, “Video and

Image Processing in Multimedia Systems”, Kluwer Academic Pub, 1996


Technology

Compression

Documents

Transcript of Compression