ImageTransforms - IIT Kanpurhome.iitk.ac.in/~venkats/teaching/Transforms1.pdf · Indian Institute...

Separable and Unitary TransformsSinusoidal transforms

Non-sinusoidal transformsWavelet Transform

Image TransformsEE 604

Govind Sharma

Department of Electrical EngineeringIndian Institute of Technology, Kanpur

26/2/2019

G. Sharma Transforms



Outline

Separable and Unitary Transforms

Sinusoidal transforms

Non-sinusoidal transforms

Wavelet Transform




Image Transforms

Transforms are used in many image processing applications.(a) Image representation(b) Mathematical convenience (Filtering)(c) Image compression/ Dimensionality reduction(d) Feature extraction(e) Pattern recognition(f) Pre-processing stepThere is no universally good transform.




Discrete Fourier Transform

For an M×N image array f [m,n] its two-dimensional DFT F [u,v ]is defined as

F [u,v ] = 1√MN

M−1∑m=0

N−1∑n=0

f [m,n]e−j2π( umM + vn

N )︸︷︷︸kernel

and the IDFT is given by

f [m,n] = 1√MN

M−1∑u=0

N−1∑v=0

F [u,v ]ej2π( umM + vn

N )︸︷︷︸kernel

This is a Linear, Separable kernel, Unitary transform.




General Linear Transform

For an M×N image array f [m,n] its two-dimensional transformF [u,v ] is defined as

F [u,v ] =M−1∑m=0

N−1∑n=0

f [m,n]A[m,n;u,v ]︸︷︷︸forward kernel

Inverse transform gives mapping from transform domain to imagespace

f [m,n] =M−1∑u=0

N−1∑v=0

F [u,v ]B[m,n;u,v ]︸︷︷︸inverse kernel




Separable Kernel

Transform has separable kernel if

A[m,n;u,v ] = AC [m;u]AR [n;v ]B[m,n;u,v ] = BC [m;u]BR [n;v ]

where subscripts C and R refer to one dimensional column and rowtransform kernels.




Unitary Transform

Transform is unitary if

M−1∑m=0

N−1∑n=0

A[m,n;u,v ]A∗[m,n;u′,v ′] = δ[u−u′,v − v ′]M−1∑m=0

N−1∑n=0

B[m,n;u,v ]B∗[m,n;u′,v ′] = δ[u−u′,v − v ′]M−1∑u=0

N−1∑v=0

A[m,n;u,v ]A∗[m′,n′;u,v ] = δ[m−m′,n−n′]M−1∑u=0

N−1∑v=0

B[m,n;u,v ]B∗[m′,n′;u,v ] = δ[m−m′,n−n′]




Unitary Transform in Vector Space form

Notation

f = matrix with f [m,n] as elementsF = matrix with F [u,v ] as elementsf = columns of f stacked togatherF = columns of F stacked togather

f and F are vectors in MN-dimensional vector space




Unitary Transform in Vector Space form

Forward transform can be written as

F = Af

and inverse transform can be written as

f = BF

It can be seen easily that B = A−1. Transform is unitary if

AHA = I = AAH or BHB = I = BBH

If A and B are real matrices then transform is called orthogonaltransform.




Separable Transform in Matrix notation

If A is an m×n matrix and B is a p×q matrix then LeftKronecker product A⊗B is an mp×nq matrix defined by

A⊗B =

b11A · · · b1qA... . . . ...

bp1A · · · bpqA

Transform kernel is separable if

A = AC ⊗ARB = BC ⊗BR

where AC ,AR ,BC ,BR are matrices for one-dimensional transforms.




Separable Kernel Unitary Transform

When the transform is separable kernel and unitary we can writeimage matrix f and transform-domain matrix F as

F = AC fATR

andf = BCFBT

R

where BC = A−1C and BR = A−1

R .




Outer Product Form

Let aC [m],aR [m] represent mth row of unitary matrices AC and ARrespectively, then

F =M−1∑u=0

N−1∑v=0

α[u,v ]aC [u]aTR [v ]

The unit rank matrices aC [u]aTR [v ] are called basis matrices.




Properties of Unitary Transform

Euclidean norm of the MN dimensional vector is preserved

‖f ‖= ‖F‖

Angle between vectors (for real case) are also preserved.




Cosine transform

Cosine transform is defined as

F [u,v ] = 2N C [u]C [v ]

N−1∑m=0

N−1∑n=0

f [m,n]cos(π

N

(um + 1

2

))cos

(π

N

(vn + 1

2

))

The inverse transform is given as

f [m,n] = 2N

N−1∑m=0

N−1∑n=0

C [u]C [v ]F [u,v ]cos(π

N

(u (m + 1

2

)))cos

(π

N

(v (n + 1

2

)))

where C [0] = 1/√2 and C [m] = 1,m 6= 0.




Properties of Cosine Transform

Linear, separable, unitary transform. It is symmetric,anti-periodicfor 2N but periodic for 4N in one-dimension. Let X [k] denote thecosine transform of x [n] sequence of length N then(a) X [−k] = X [k](b) X [k +2N] =−X [k](c) X [k +4N] = X [k](d) Convolution property does not hold(e) Diagonalizes symmetric tri-diagonal matrix shown in next slide




Properties of Cosine Transform

Symmetric tri-diagonal matrix diagonalized by cosine transform

1−α −α−α 1 −α 0

. . . . . . . . .0 −α 1 −α

−α 1−α




Modified Cosine Transform

Image data is generally divided into blocks and then transform istaken for these blocks. Coding may create artificial effects at blockboundaries. To reduce this modified DCT is used.(a) Instead of N data points take 2N data points and take 2N

point DCT and retain only first N coefficients.(b) For the next block of 2N points take N points from previous

block and N new points and repeat first step.




Sine Transform

Discrete sine transform (DST) for a sequence {x [n],0≤ n≤ N−1}and its inverse are defined as

X [k] =√

2N+1

N−1∑n=0

x [n] sin π(k+1)(n+1)N+1 , 0≤ k ≤ N−1

x [n] =√

2N+1

N−1∑k=0

X [k] sin π(k+1)(n+1)N+1 , 0≤ n ≤ N−1

In 2−D case we take row wise transform and then column wisetransform.




Properties of Sine Transform

Linear, separable, orthogonal transform. It is symmetric. Forwardand inverse transforms are same. It diagonalizes the Toeplitzmatrix shown below:

1 −α−α 1 −α 0

. . . . . . . . .0 −α 1 −α

−α 1




Hartley Transform

Hartely transform pair is defined by

F [u,v ] = 1N

N−1∑m=0

N−1∑n=0

f [m,n]cas(

2πN (um + vn)

)0≤ u,v ≤ N−1

f [m,n] = 1N

N−1∑u=0

N−1∑v=0

F [u,v ]cas(

2πN (um + vn)

)0≤ u,v ≤ N−1

where cas(θ) = cos(θ)+ sin(θ). This is not a separable transform.Separable discrete Hartley transform has been defined by firsttaking row wise transform and then taking column wise transform.This transform is closely related to DFT.




Examples of image transforms




Hadamard TransformThe elements of the basis vectors for Hdamard transform take only±1 so no multiplication is required. This is defined easily forN = 2m, although can be defined for some multiples of 4 also.Hadamard matrices are defined below:

H2 = 1√2

[1 11 −1

]For larger values of N, we define recursively as

H2N = HN ⊗H2 = 1√2

[HN HNHN −HN

]Using the notation defined earlier f for image matrix and F fortransform coefficient, for a N×N image the Hadamard transformis given by

F = HN fHTN




Properties of Hadamard Transform

(a) Matrix H is real, symmetric and orthogonal.(b) 1-D transform can be calculated using N logN addition and

subtraction operations.(c) It has good energy compaction for highly correlated images.




Haar TransformThe Haar transform can be calculated recursively using thefollowing matrices:

H̃N =[VNWN

]where VN is a N/2×N scaling (approximation) matrix and WN isa N/2×N wavelet (detail) matrix given below:

VN = 1√2

1 1 0 0 0 · · · 0 0 0 00 0 1 1 0 · · · 0 0 0 0

...0 0 0 0 0 · · · 1 1 0 00 0 0 0 0 · · · 0 0 1 1




Haar Transform

WN = 1√2

1 −1 0 0 0 · · · 0 0 0 00 0 1 −1 0 · · · 0 0 0 0

...0 0 0 0 0 · · · 1 −1 0 00 0 0 0 0 · · · 0 0 1 −1




Haar Transform

The first level Haar transform of a N×1 vector f is

f 1 = H̃N f = [a1|d1]T

where a1 = VN f represents approximate part and d1 = WN frepresents detail part. The next step in the recursion process is tocompute Haar transform from the approximate part a1. The resultis

f 2 = [a2|d2|d1]T

where a2 = VN/2a1 and d2 = WN/2a1.




Examples of image transformsFor the girl image in earlier example




Haar Transform, Daubechies Transform

The process continues until for N = 2J we obtain

f J = [aJ |dJ |dJ−1| · · · |d1]T

Haar transform does not have good energy compaction propertyfor images. The Haar transform utilizes running average andrunning difference of two elements of the vector. The Daubechiestransform uses weighted sums and differences over longer lengths.For example Daubechies transform of support four, denoted byDaub4 is similar to Haar transform with VN and WN matricesdefined below:




Daubechies Transform

VN =

α1 α2 α3 α4 0 0 · · · 0 0 0 00 0 α1 α2 α3 α4 · · · 0 0 0 0

...0 0 0 0 0 0 · · · α1 α2 α3 α4α1 α2 0 0 0 0 · · · 0 0 α3 α4

WN =

β1 β2 β3 β4 0 0 · · · 0 0 0 00 0 β1 β2 β3 β4 · · · 0 0 0 0

...0 0 0 0 0 0 · · · β1 β2 β3 β4β1 β2 0 0 0 0 · · · 0 0 β3 β4

where α1 =−β4 = 1+

√3

4√

2 ,α2 = β3 = 3+√

34√

2 ,, α3 =−β2 = 3−√

34√

2 ,α4 = β1 = 1−

√3

4√

2G. Sharma Transforms



Sub-band coding

Haar transform and Daubechies transform are special cases ofwavelet transform. Sub-band coding, multi-resolution analysis(MRA) and wavelet transform are closely related.In two band sub-band coding the input signal x [n] is divided in twosignals, a low pass signal xLP [n] and a high pass signal xHP [n] asshown below. the signal x [n] can be reconstructed perfectly fromxLP [n] and xHP [n] as shown below.




Sub-band coding

x [n]

h0[n] ↓ 2 xLP [n]

h1[n] ↓ 2 xHP [n]

Analysis Filter bank

xLP [n] ↑ 2 g0[n]

+

xHP [n] ↑ 2 g1[n]

x̂ [n]Synthesis Filter bank




Sub-band coding

ω

|H0(ejω)|

0 π/2 π

|H1(ejω)|

For perfect reconstruction(x [n] = x̂ [n]) we have{g0[n] = (−1)nh1[n]g1[n] = (−1)n+1h0[n] or

{g0[n] = (−1)n+1h1[n]

g1[n] = (−1)nh0[n]

The filter impulse responses should also satisfy the biorthogonalitycondition

〈hi [2n−k],gj [k]〉= δ[i− j]δ[n], i , j ∈ {0,1}




Sub-band coding

In some cases filters satisfy an additional constraint

〈gi [n],gj [n +2m]〉= δ[i− j]δ[m], i , j ∈ {0,1}

which defines orthonormality for perfect reconstruction filter banks.In this case all filter responses can be computed from a singleprototype response h0[n]. In case of images we apply filters first inone dimension (say row-wise) and then apply them in the otherdimension (column-wise).




Multi Resolution Analysis (MRA)

Consider a set of functions {φk(x),x ∈ R,k ∈ Z}. Suppose

f (x) =∑

kαkφk(x)

If αk are unique then φk ’s are linearly independent and are calledbasis functions. Let

V = spank{φk(x)}

where over-bar represents closure of the set i.e. iffn(x)→ f (x), fn(x) ∈ V then f (x) is also included in V . Forexpansion set {φk(x)} there is a set of dual functions {φ̃k(x)} thatcan be used to calculate coefficients {αk}.





αk = 〈f (x), φ̃k(x)〉=+∞∫−∞

f (x)φ̃k(x)dx

There are three possibilities





(a) {φk(x)} form an orthonormal basis for V . In this caseφ̃k(x) = φk(x)

(b) {φk(x)} are linearly independent. We can use dual functionsφ̃k(x) to calculate αk , provided following condition calledframes condition is satisfied:

A‖f ‖2 ≤∑

k|〈f (x),φk(x)〉|2 ≤ B‖f ‖2, A> 0, B <∞

(c) {φk(x)} satisfy frames condition but are not linearlyindependent. In this case we have a series representation butαk ’s are not unique. In this case the set {φk(x)} and its dualare said to be overcomplete.





Let φ(x) be real square integrable function. We define

φj,k(x) = 2j/2φ(2jx −k), j ,k ∈ Z

For a fixed j defineVj = span

k{φj,k(x)}

It is seen readily thatVj ⊂ Vj+1

We say that the scaling function φ(x) satisfies requirements ofMRA if:





Requirements of MRA(a) Ortogonal Translation: Integer translates of scaling functions

are mutually orthonormal, i.e.〈φ(x −k),φ(x −m)〉= δ[k−m].

(b) Nesting property: f (x) ∈ Vj ⇒ f (2x) ∈ Vj+1 and

V−∞ ⊂ ·· ·V−1 ⊂ V0 ⊂ V1 ⊂ ·· · ⊂ V∞

(c) Only function common to all Vj ’s is f (x) = 0.(d) Any finite energy signal can be represented in terms of

φj,k(x), i.e. V∞ = L2(R).





Letφ(x) =

{1 06 x < 10 otherwise

satisfies all the requirements of MRA. There are many scalefunctions with finite and infinite support that satisfy MRArequirements.





V0V1

V2

Orthogonal complement of V0 in V1 is called W0, i.e. everyfunction f (x) ∈ V1 can be written uniquely as f (x) = s(x)+w(x)where s(x) ∈ V0 and w(x) ∈W0 and they are orthogonal. This wewrite as V1 = V0⊕W0.




Multi Resolution Analysis (MRA)In a similar manner orthogonal complement of V1 in V2 is calledW1 and so on. We can write

VJ = VJ−1⊕WJ−1

= VJ−2⊕WJ−2⊕WJ−1

= V0⊕W0⊕·· ·⊕WJ−2⊕WJ−1

Similarly

L2(R) = V0⊕W0⊕W1⊕·· ·= VJ ⊕WJ ⊕WJ+1⊕·· ·= · · ·W−1⊕W0⊕W1⊕·· ·




Wavelet series

Given a scaling function φ(x) that meets MRA requirements{φj,k(x),k ∈ Z} is an orthonormal basis of Vj . It can be shownthat there exists a corresponding function ψ(x) such that

{ψj,k(x) = 2j/2ψ(2jx −k), j ,k ∈ Z}

forms an orthonormal basis for Wj . Scaling function φ(x) issometimes called as father wavelet and function ψ(x) is calledmother wavelet.




Wavelet seriesSince V0 ⊂ V1, and φ0,0(x) = φ(x) we get

φ0,0(x) =∑

nhφ[n]φ1,n(x)

φ(x) =∑

nhφ[n]

√2φ(2x −n)

Similarly W0 ⊂ V1, and ψ0,0(x) = ψ(x) we get

ψ(x) =∑

nhψ[n]

√2φ(2x −n)

It can be shown that hψ[n] is related to hφ[n] as

hψ[n] = (−1)nhφ[1−n]




Wavelet seriesFor Haar wavelet scaling coefficients and wavelet coefficients arehφ[0] = 1/

√2,hφ[1] = 1/

√2 and hψ[0] = 1/

√2,hψ[1] =−1/

√2.

Given a square integrable function f (x) we can start at any scale Jand express it as

f (x) =∞∑

k=−∞cJ [k]φJ,k(x)+

∞∑j=J

∞∑k=−∞

dj [k]ψj,k(x)

where

cJ [k] = 〈f (x),φJ,k(x)〉dj [k] = 〈f (x),ψj,k(x)〉




Wavelet series

Example: Given

f (x) ={

1 06 x < 10 otherwise

using Haar wavelet we get for j = 0 and j = 1

c0[0] = 1/3d0[0] =−1/4d1[0] =−

√2/32

d1[1] =−3√2/32

We can calculate more coefficients.




Wavelet seriesAn algorithm to calculate wavelet series coefficients

φ(x) =∑

nhφ[n]

√2φ(2x −n)

φ(2jx −k) =∑

nhφ[n]

√2φ(2(2jx −k)−n)

=∑

nhφ[n]

√2φ(2j+1x −2k−n)

=∑m

hφ[m−2k]√2φ(2j+1x −m), (putting) m = 2k +n

Similarly

ψ(2jx −k) =∑m

hψ[m−2k]√2φ(2j+1x −m)




Wavelet seriesNow

cj [k] = 〈f (x),φj,k(x)〉= 〈f (x),2j/2φ(2jx −k)〉= 〈f (x),2j/2∑

mhφ[m−2k]

√2φ(2j+1x −m)〉

=∑m

hφ[m−2k]〈f (x),2j+1

2 φ(2j+1x −m)〉

=∑m

hφ[m−2k]〈f (x),φj+1,m(x)〉

=∑m

hφ[m−2k]cj+1[m]




Wavelet seriesSimilarly

dj [k] =∑m

hψ[m−2k]cj+1[m]

We can also write these two relations as

cj [k] = hφ[−n]∗ cj+1[n] |n=2k

dj [k] = hψ[−n]∗ cj+1[n] |n=2k

cj+1[n]

hφ[−n] ↓ 2 cj [n]

hψ[−n] ↓ 2 dj [n]




Wavelet series

Inverse of these relations can be shown to be

cj+1[k] =∑

nhφ[k−2n]cj [n]+

∑n

hψ[k−2n]dj [n]

cj [n] ↑ 2 hφ[n]

+

dj [n] ↑ 2 hψ[n]

cj+1[n]




Wavelet series

Sequences hφ[n] and hψ[n] satisfy the following conditions knownas quadrature mirror filter (QMF) conditions1. hψ[n] = (−1)nhφ[1−n]2.∑n

hφ[n] =√2

3.∑n

hψ[n] = 0

4.∑n

hφ[n]hφ[n−2k] = δ[k]

5.∑n

hψ[n]hψ[n−2k] = δ[k]

6.∑n

hψ[n]hφ[n−2k] = 0

7.∑n

hφ[m−2n]hφ[k−2n]+∑n

hψ[m−2n]hψ[k−2n] = δ[m−k]




Wavelet series

From the QMF conditions it can be seen that if hφ[n] is a finitelength sequence then so is hψ[n] and both sequences have samelength. Each sequence must have even length.




Discrete time wavelet transformSuppose that we are given a discrete time sequence {x [n]}. Wedefine a function f (x) by

f (x) =∑

nx [n]φ0,n(x)

so that c0[n] = x [n]. Thus we have assumed that given sequencevalues are scaling coefficients for some function f (x). Note thatcoefficients are not the sample values of the function f (x). Usingpreviously developed algorithm we can calculate scale and detailcoefficients cj [n] and dj [n] for j < 0. Given J ,the discrete timewavelet transform of a sequence is the collection

{c−J [k], k ∈ Z}∪{dj [k], −J ≤ j ≤−1, k ∈ Z}




Discrete wavelet transform (DWT)

Consider a finite length sequence c0[n], 0≤ n ≤ 2N −1. We canassume it to be discrete time signal with other values zero. If hφ[n]and hψ[n] have length L, then output of the filters have length2N +L−2. After decimation by 2 length of c−1[n] and d−1[n]sequences will be 2N+L

2 , so total no. of coefficients will be 2N +L.Similarly after each decomposition this length will increase by L.Hence after J decomposition DWT length will become 2N +JL.For small values of J and L this increase may be acceptable.




Discrete wavelet transform (DWT)

Another method of treating a finite sequence is to assume that it isa period of periodic sequence. Since the output of a linear systemwith periodic input is also periodic with same period, lengths of thefilter outputs do not change. Hence after decimation by 2 length ofc−1[n] and d−1[n] sequences will be 2N

2 , so total no. of coefficientswill be 2N . Since the length of sequence does not change afterdecomposing into approximation and detail sequences, after Jdecompositions total no. of coefficients will remain 2N . Thus DWTalgorithms assume periodic extension of the given finite sequence.




Wavelet transform of imagesFor 2-D data we use separable transform. We apply 1-D transformcolumn-wise and then apply transform row-wise.




Wavelet transform of images

Resulting decomposition




Wavelet transform of imagesSynthesis of image




Example of wavelet transform of images

First level decomposition using Haar wavelet




Bi-orthogonal waveletsFor images we would like that hφ[n] and hψ[n] have symmetricresponses. But we have seen earlier that length of these sequencesis even so symmetric response is not possible. To use a symmetricresponse we drop the orthogonality condition and use bi-orthogonalwavelets. As mentioned earlier in discussion of MRA coefficientsare calculated using dual functions. Hence

f (x) =∞∑

k=−∞cJ [k]φJ,k(x)+

∞∑j=J

∞∑k=−∞

dj [k]ψj,k(x)

where

cJ [k] = 〈f (x), φ̃J,k(x)〉dj [k] = 〈f (x), ψ̃j,k(x)〉




Bi-orthogonal wavelets

cj+1[n]

hφ̃[−n] ↓ 2 cj [n]

hψ̃[−n] ↓ 2 dj [n]

cj [n] ↑ 2 hφ[n]

+

dj [n] ↑ 2 hψ[n]

cj+1[n]




Wavelet transform of images

Daubechies 9/7 and LeGall 5/3 are examples of bi-orthogonalwavelets where first number refers to no. of coefficients in analysisfilter and second number refers to number of coefficients insynthesis filter. Energy is not preserved in non-orthogonaltransforms.We decompose only the scale coefficients. If we decompose detailcoefficients also we get wavelet packets.




decomposition lena.jpg




References

1. Introduction to Digital Image Processing, William K. Pratt,CRC Press.

2. Digital Image Processing, Rafael C. Gonzalez and Richard E.Woods, Prentice Hall.

3. Fundamentals of Digital Image Processing, Anil K. Jain,Prentice Hall.


ImageTransforms - IIT Kanpurhome.iitk.ac.in/~venkats/teaching/Transforms1.pdf · Indian Institute...

Documents

Transcript of ImageTransforms - IIT Kanpurhome.iitk.ac.in/~venkats/teaching/Transforms1.pdf · Indian Institute...