1/88 DCT Transform Decoder. 2/88 Image (512x512) Subsample (128x128) Manipulation Reposition :...

1/88

DCT Transform Decoder

2/88

Image (512x512)

Subsample (128x128)

Manipulation Reposition :(256,256)-(384,384)

Compress (JPEG)

167 123

84 200

2D array of data

3/88

QUANRUN-LEVEL

CODINGFINAL

BITSTR

DCTCOMP &

PART

JPEG

4/88

JPEG 2000 Framework

WAVELETQUAN &

PARTBITPLANECODING

BITSTRASSEMBLY

FINALBITSTR

COMP &TILE

COLOR IMAGE

Y COMP

CR COMP

CB COMP

5/88

Transform

ORIGINAL128, 129, 125, 64, 65, … TRANSFORM COEFFICIENTS

4123, -12.4, -96.7, 4.5, …

6/88

2LOW PASS

ANALYSIS G(z)

HIGH PASSANALYSIS H(z) 2

X(z)

LOW PASSCOEFF γ(z)

HIGH PASSCOEFF λ(z)

2

2

LOW PASSSYNTHESIS G’(z)

HIGH PASSSYNTHESIS H’(z)

Y(z)+

X(z)LOW PASSCOEFF γ(z)

HIGH PASSCOEFF λ(z)

Y(z)SPLIT P(z) P’(z) MERGE

7/88

X(z)

LOW PASSCOEFF γ(z)

HIGH PASSCOEFF λ(z)+

sm(z)

+

SPLIT tm(z)

+

s0(z)

+

t0(z)

K1

K2

8/88

Y(z)

LOW PASSCOEFF γ(z)


sm(z)

+

MERGE tm(z)

+

s0(z)

+

t0(z)

1/K1

1/K2

9/88

X(z)

LOW PASSCOEFF γ(z)


f(a)

+

SPLIT f(b)

+

f(c)

+

f(d)

K1

K2

10/88

Lifting Implementation

a=-1.586

b=-0.052

c= 0.883

d= 0.444

H0

H1

H2

H3

a

aa

a

b

b

b

c

c

cc

d

d

d

a

aa

a

bb

bb

c

c

cc

d

d

d

d

b d

High LowOriginal

x0

x1

x2

x3

x4

x5

x6

x7

x8

L0

L1

L2

L3

L4

a

a1

Just be saved in its own position

Y = (x0+x2)*a + x1

x0

x1

x2

.

.

.

.

.

.

11/88

Inverse Transform

a

aa

a

b

b

b

c

c

cc

d

d

d

a

aa

a

bb

bb

c

c

cc

d

d

d

d

b d

High LowOriginal

x0

x1

x2

x3

x4

x5

x6

x7

x8

L0

H0

L1

H1

L2

H2

L3

H3

L4

cd

d

L

X

R

Y = X + ( L + R ) * d

X = Y + ( L + R ) * ( - d )

TRANSFORM

-d

-d-d

-d

-c

-c

-c

-b

-b

-b-b

-a

-a

-a

-d-d

-d

-c

-c-c -b

-a

-a-c -a

-d -c -a

-a-b

-b

-b

INVERSE TRANSFORM

Original

.

.

.

.

.

.

INVERSE

DATA FLOW

12/88

Lifting Implementation Boundary Handling

a b c d

aa

b

b c

a

a b c

a

aa

bb

b

c

c

c

dd

d

x0

x1

x2

x3

x2

x2

x1

x2

x3

x1

x0

a b

a

a

a2a

2bb

b

c

c

2c

2dd

d

x0

x1

x2

x3

L0

H0

L1

H1

L0

H0

L1

H1

-2d-d

-d

-c

-c-2c -b

-b

-2b

INVERSE TRANSFORMFORWARD TRANSFORM

13/88

Inverse Transform

-2d-d

-d

-c

-c-2c -b

-a

-2a-a-b

-2b

INVERSE TRANSFORM

L0

H0

L1

H1

x0

x1

x2

x3

14/88

Wavelet Transform - 2D

G

H

2

x

a0

a1

2

G

H

2a00

a01

2

G

H

2a10

a11

2

Horizontal filtering Vertical filtering

15/88

Wavelet Packet Structure

mallat spacl packet

HL

LH HH

16/88

Line-based Lifting Scheme

For 9-7 filter,the number of needed line in buffer is 6.

High LowOriginal

x0

x1

x2

x3

x4

x5

x6

x7

x8

L0

H0

L1

H1

L2

H2

L3

H3

L4

1st

Lift2nd

Lift

STEP 2

STEP 3

. . .

STEP 1

17/88

Line-based Lifting Scheme

Use of the partial character of wavelet transformNo need to wait for all data

Buffersize=linenumber ( depend on the filter’s length) * linesize

LL

HL

LH

HH

Horizontal filter

Vertical filter

Lowpass of Vertical

Highpass of Vertical

18/88

Quantizer

TRANSFORM COEFF4123, -12.4, -96.7, 4.5, …

QUANTIZE COEFF(Q=1)4123, -12, -96, 4, …

19/88

0 1

2 3

8 9

10 11

4 5

6 7

20/88

Block Entropy Coding : Finally

45 0 0 0-74 -13 0 0

21 0 4 014 0 23 23

0 0 0 03 0 4 0

0 3 5 00 0 0 0

0 1 -1 0-4 33 0 -1

0 0 1 00 0 0 0

-4 5 0 0-18 0 0 19

4 0 23 0-1 0 0 0

BITSTREAM

D

R

R-D CURVE

21/88

A block of coefficients

45 0 0 0-74 -13 0 0

21 0 4 014 0 23 23

0 0 0 03 0 4 0

0 3 5 00 0 0 0

0 1 -1 0-4 33 0 -1

0 0 1 00 0 0 0

-4 5 0 0-18 0 0 19

4 0 23 0-1 0 0 0

0 1 0 1 1 0 10 1 0 1 1 0 1 +

1 0 0 1 0 1 0 -

0 0 1 0 1 0 1 +

0 0 0 1 1 1 0 +

0 0 0 0 1 0 0 -

0 0 1 0 0 1 0 -

0 0 0 0 1 0 0 +

0 0 0 0 0 0 1 -

SIGNb1 b2 b3 b4 b5 b6 b7

w0

w1

w2

w3

w4

w5

w6

w7

ON

E L

INE

OF

CO

EF

45

-74

21

14

-4

-18

4

-1

22/88

0 1 0 1 1 0 10 1 +1 0 0 1 0 1 0 -0 0 1 0 1 0 1 +0 0 0 1 1 1 0 +0 0 0 0 1 0 0 -0 0 1 0 0 1 0 -0 0 0 0 1 0 0 +0 0 0 0 0 0 1 -


w0

w1

w2

w3

w4

w5

w6

w7

45

-74

21

14-4

-18

4

-1

SIGNIFICANT IDENTIFICATION

REFINEMENT

PREDICTEDINSIGNIFICANCE(PN)

PREDICTEDSIGNIFICANCE(PS)

REFINEMENT (REF)

23/88

Bits of Coefficients

0 1 0 1 1 0 10 1 0 1 1 0 1 +1 0 0 1 0 1 0 -0 0 1 0 1 0 1 +0 0 0 1 1 1 0 +0 0 0 0 1 0 0 -0 0 1 0 0 1 0 -0 0 0 0 1 0 0 +0 0 0 0 0 0 1 -


w0

w1

w2

w3

w4

w5

w6

w7

CO

EF

FIC

IEN

TS

45

-74

21

14-4

-18

4

-1

24/88

Embedded Coding – The First Pass

01 -000000


001 -000000

w0

*w1

w2

w3

w4

w5

w6

w7

VALUE

0RANGE

-63..63

-96 -127..-64

-63..630

-63..630

-63..630

-63..630

-63..630

-63..630

25/88


01 -000000

+SIGNb1 b2 b3 b4 b5 b6 b7

00 1 +1 -0 000000

*w0

*w1

w2

w3

w4

w5

w6

w7

VALUE

48RANGE

32..63

-96 -127..-64

-31..310

-63..630

-63..630

-63..630

-63..630

-63..630

26/88


01 -000000


00 1 +1 0 -0 000000

*w0

*w1

w2

w3

w4

w5

w6

w7

VALUE

48RANGE

32..63

-80 -95..-64

-31..310

-63..630

-63..630

-63..630

-63..630

-63..630

27/88


01 -000000


00 1 +1 0 -0 00 00 00 00 00 0

*w0

*w1

w2

w3

w4

w5

w6

w7

VALUE

48RANGE

32..63

-80 -95..-64

-63..630

-31..310

-31..310

-31..310

-31..310

-31..310

28/88

After Bitplane Scanning & Context Modeling

45 0 0 0-74 -13 0 0

21 0 4 014 0 23 23

0 0 0 03 0 4 0

0 3 5 00 0 0 0

0 1 -1 0-4 33 0 -1

0 0 1 00 0 0 0

-4 5 0 0-18 0 0 19

4 0 23 0-1 0 0 0

Bit: 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 ……Ctx: 0 0 9 0 0 0 0 0 0 7 10 0 0 0 0 0 0 0 0 ……

29/88

h: number of significanthorizontal neighbors (0-2)

v: number of significantvertical neighbors (0-2)

d: number of significantdiagonal neighbors (0-4)

v

h

d

CURRENT

30/88

Zero Coding Context

LH Subband (also used for LL) (vertically high-pass)

HL Subband (horizontally high-pass)

HH Subband (diagonally high-pass)

h v d context h v d context d h+v context

2 x x 8 x 2 x 8 3 x 8

1 1 x 7 1 1 x 7 2 1 7

1 0 1 6 0 1 1 6 2 0 6

1 0 0 5 0 1 0 5 1 2 5

0 2 x 4 2 0 x 4 1 1 4

0 1 x 3 1 0 x 3 1 0 3

0 0 2 2 0 0 2 2 0 2 2

0 0 1 1 0 0 1 1 0 1 1

0 0 0 0 0 0 0 0 0 0 0

31/88

Context in Sign Coding

v

h

h, v: neighborhood sign status-1: one or both negative0: both insignificant or both significant but opposite sign1: one or both sig. & positive

Expected Sign

h v x context

1 1 0 13 1 0 0 12 1 -1 0 11 0 1 0 10 0 0 0 9 0 -1 1 10 -1 1 1 11 -1 0 1 12 -1 -1 1 13

32/88

Magnitude Refinement

],[ nm h+v Context

1 x 160 1 150 0 14

],[ nm : Whether magnitude refinement has been applied

h, v : Defined as in zero coding

33/88

MQ-CoderBITS

CTXBITSTREAM

34/88

Arithmetic Coding - Illustration

What is arithmetic coding

0

1

1-P0

P0

1-P1

P1 1-P2

P2

S0=0 S1=1 S2=0

0.100

Coding result:

(Shortest binarybitstream ensures thatintervalB=0.100 0000000 toD=0.100 1111111 is(B,D) A)

AB

D

35/88

Probability estimation

Assume within each context, the symbol is i.i.d (independently identically distributed)

We can count the # of symbols and # of 1s and estimate the probability1 0 0 1 0 0 0 1 0 0 0 1 0 0 1 0 1 0 0 0 0

21p 3

1p 31p 10

3p

36/88

JPEG 2000 Bitstream Assembler

37/88

Assemble the Bitstream

Bitstream Rate-distortion optimized, for

progressive by quality May be reordered Region with resolution

access, progressive by quality

Res1

Res2

.

.

.Encode each block separately & record a bitstream for each block

. . . . . .

Abstraction Layer

38/88

Rate-Distortion Optimized Assembling (1st layer)

D1

R1

D2

R2

D3

R3

D4

R4

D1

R1

D2

R2

D3

R3

D4

R4

r1 r2

r3 r4

. . .

39/88

Rate-Distortion Optimized Assembling (2nd Layer)

D1

R1

D2

R2

D3

R3

D4

R4

r’1r’2

r’3 r’4

D1

R1

D2

R2

D3

R3

D4

R4

r1 r2

r3 r4

. . .

40/88

JPEG 2000 Bit Stream Syntax

Packet

Head Body Res

yn

c

SO

T m

ark

er

SO

C m

ark

er

Glo

ba

l H

ead

er

SO

S m

ark

er

Til

e H

ead

er

Packet

Head

Body Res

yn

c

SO

T m

ark

er

SO

S m

ark

er

Til

e H

ead

er

EO

I m

ark

er

extra tiles

Packet

Head Body Res

yn

c

Packet

Head Body Res

yn

c

Layer n

Layer 1...

42/88

JPEG 2000 Bitstream

What is in a packet? The compressed bitstream of

One component One resolution At a quality level

Packet header : length of a block bitstream in this packet

Packet body : The compressed bitstream

43/88

Vmedia Interactive Image Browsing

44/88

Vmedia

Network

Network

Server

Vmedia

Media Program

Media

Network

Vmedia

Media Program

. . .

Client1 Clientn

Vmedia

Most work done at client end

JPEG 2000CompressedImages

45/88

Virtual Media Concept

Media Apps

Network Service

VMedia APIs

Client

Network Service

Server

BMP JPEG2000

Connect Request

nid, nfid

vfid

Cache

JPEG 2000

Virtual Media

46/88

Render Apps

Network Service

VMedia APIs

Client

Cache

Network Service

Server

BMP JPEG2000

Data Request

Streaming with prioritization

Data

Cache Management

47/88

Initial Stage

Read filehead & media structure

Packet

Head Body Res

ync

SO

T m

arke

r

SO

C m

arke

r

Glo

bal

Hea

der

SO

S m

arke

r

Til

e H

eade

r

Packet

Head Body Res

ync

SO

T m

arke

r

SO

S m

arke

r

Til

e H

eade

r

EO

I m

arke

r

extra tiles

48/88

Initial Stage – JPEG 2000

JPEG 2000 image header & packet head are

readin

49/88

Entire Image – Low Res

50/88

Zooming In

x

x

51/88

Panning Around

x

x

52/88

Operation Flow of the Vmedia Image Browser

User Request

Identify regionof interest

Identify relatedcoefficient blocks

Locate & accessAssociated bitstream

Vmedia Interface

Assemble bitstream

Entropy decoding

Inverse Quantization

Inverse Lifting

Shown to user

53/88

Operation Flow of the Vmedia Client

Access Request

AvailabilityCheck

Network Queues

Cache

Network

Serve Requests

Mediadatabase

Vmedia client Vmedia server

55/88

Preemptive Loading

Predict what the user wants next Zooming in and out, panning are the most

common browse request Access data which is at the neighborhood of

current view

56/88

Demo

Application Browser

57/88

Summary

What you should learn from the course Difference in file and image compression Key components of a modern image

compression system New application and functionalities facilitated

by the modern image compression system

1/88 DCT Transform Decoder. 2/88 Image (512x512) Subsample (128x128) Manipulation Reposition :...

Documents

Transcript of 1/88 DCT Transform Decoder. 2/88 Image (512x512) Subsample (128x128) Manipulation Reposition :...