CSE 559A: Computer Visionayan/courses/cse559a/PDFs/lec...FOURIER TRANSFORM and are both 2D array of...

CSE 559A: Computer Vision

Fall 2020: T-R: 11:30-12:50pm @ Wrighton 300 / Zoom

Instructor: Ayan Chakrabarti ([email protected]).Course Staff: Adith Boloor, Patrick Williams

October 1, 2020

http://www.cse.wustl.edu/~ayan/courses/cse559a/

1

FOURIER TRANSFORMFOURIER TRANSFORM

and are both 2D array of the same size .

is complex-valued (while is typically real-valued)

These equations are linear. So both the Fourier transform and its inverse are linear operations.

But each depends on values of at ALL locations (not local like a convolution).

Note that the expressions in both are similar—one has a negative sign inside the , indicating acomplex conjugate.

But, in one, we hold fixed and sum over . In the other, vice-versa.

F[u, v] = X[ , ] exp(−j 2π ( + ))1

WH ∑=0nx

W−1

∑=0ny

H−1

nx nyu nx

W

v ny

H

X[ , ] = F[u, v] exp(j 2π ( + ))nx ny ∑u=0

W−1

∑v=0

H−1u nx

W

v ny

H

F[u, v] X[ , ]nx ny W × H

F X

F[u, v] X[ , ]nx ny

exp(⋅) exp

(u, v) ( , )nx ny

2

FOURIER TRANSFORMFOURIER TRANSFORMDFT as a Co-ordinate Transform

where each can be thought of as a different (complex-valued) image:

and denotes the complex-conjugate of .

F[u, v] = [ , ] X[ , ]1

WH‾ ‾‾‾√ ∑=0nx

W−1

∑=0ny

H−1

S̄uv nx ny nx ny

Suv

[ , ] = exp(j 2π ( + ))Suv nx ny1

WH‾ ‾‾‾√

u nx

W

v ny

H

S̄uv Suv

3



is the inner-product between and . (scaled by )

For ,

For real vectors, .

For complex vectors,

is the Hermitian of : transpose AND conjugate

F[u, v] = ⟨ , X ⟩1

WH‾ ‾‾‾√Suv

Suv

[ , ] = exp(j 2π ( + ))Suv nx ny1

WH‾ ‾‾‾√

u nx

W

v ny

H

F[u, v] X Suv WH‾ ‾‾‾√

x, y ∈ ℂn ⟨x, y⟩ = ∑i x̄iyi

⟨x, y⟩ = yxT

⟨x, y⟩ = yx∗

x∗ x

4




F[u, v] = ⟨ , X ⟩1


Suv

[ , ] = exp(j 2π ( + ))Suv nx ny1

WH‾ ‾‾‾√

u nx

W

v ny

H


5




Property:

Inverse-DFT:

F[u, v] = ⟨ , X ⟩1


Suv

[ , ] = exp(j 2π ( + ))Suv nx ny1

WH‾ ‾‾‾√

u nx

W

v ny

H


⟨ , ⟩ = 1 if = u & = v, and 0 otherwise.Suv Su′v′ u′ v′

X[ , ] = F[u, v] exp(j 2π ( + ))nx ny ∑u=0

W−1

∑v=0

H−1u nx

W

v ny

H

6




Property:

Inverse-DFT:

is a weighted sum of the images, weights are given by .

F[u, v] = ⟨ , X ⟩1


Suv

[ , ] = exp(j 2π ( + ))Suv nx ny1

WH‾ ‾‾‾√

u nx

W

v ny

H



X = F[u, v] WH‾ ‾‾‾√ ∑u=0

W−1

∑v=0

H−1

Suv

X Suv F[u, v]WH‾ ‾‾‾√

7


F[u, v] = ⟨ , X ⟩, X = F[u, v] 1

WH‾ ‾‾‾√Suv WH‾ ‾‾‾√ ∑

u=0

W−1

∑v=0

H−1

Suv

8


F[u, v] = ⟨ , X ⟩, X = F[u, v] 1

WH‾ ‾‾‾√Suv WH‾ ‾‾‾√ ∑

u=0

W−1

∑v=0

H−1

Suv

9


is a matrix with each column a different .

So, .

This means is a unitary matrix.

Multiplication by is a co-ordinate transform:

are the co-ordinates of a point in a dimensional space.

Multiplication by changes the ‘co-ordinate system’.

In the new co-ordinate system, each ‘dimension’ now corresponds to frequency rather than location.

is a length-preserving matrix ( ).

It does rotations or reflections (in dimensional space).

F = X, X = S F1

WH‾ ‾‾‾√S∗ WH‾ ‾‾‾√

S WH × WH Suv


S = S = I ⇒ =S∗ S∗ S−1 S∗

S

S

X WH

S∗

S ‖ X = ‖XS∗ ‖2 ‖2

WH

10


11


12


13


14


15


16


17


18


19


20


21


22


23

FOURIER TRANSFORMFOURIER TRANSFORMThe FT gave us a different representation for images.Decomposing image into different frequency ‘components’.

What else ?

24

CONVOLUTION THEOREMCONVOLUTION THEOREM

25


26


27


28


29


is not square for valid / long convolution.

Question:

Let correspond to . Now, let . How is related to by convolution ? What operation does represent ?

A: Full convolution with (flipped version of )

Y = X ∗ k ⇒ Y = XAk

Ak

Y = XAk Y = X k∗valid = YX′ ATk X

′ Y

ATk

k[− ,− ]nx ny k

30


Now if we consider the square matrix corresponding to ‘same’ convolution with circular padding:

Then, is diagonalized by the Fourier Transform !

Here, is a diagonal matrix.

The above equation holds for every

You get different diagonal matrices .

But is the diagonalizing basis for all kernels.

In the Fourier co-ordinate system, convolution is a ‘point-wise’ operation !

Y = X ∗ k ⇒ Y = XAk

Ak

X[W + , ] = X[ , ]nx ny nx nyX[ ,− ] = X[ ,H − ]nx ny nx ny

Ak

= S Ak Dk S∗

Dk

Ak

Dk

S

Y = X = S X ⇒ ( Y) = ( X)Ak Dk S∗ S∗ Dk S

∗

31

CONVOLUTION THEOREMCONVOLUTION THEOREMWhy does this happen ?

(by linearity / distributivity)

, assuming circular padding, is also a sinusoid with the same frequency and magnitude,but different phase.

Multiplying by changes the magnitude, but frequency still the same.

Adding different sinusoids of the same frequency gives you another sinusoid of the same frequency.

, where is some complex scalar.

Sinusoids are eigen-functions of convolution

X = F[u, v]WH ‾ ‾‾‾‾√ ∑u,v Suv

Y = X ∗ k = F[u, v] ∗ kWH ‾ ‾‾‾‾√ ∑u,v Suv

( ∗ k)[n] = k[ ] [n − ]Suv ∑n′ n′ Suv n

′

[n − ]Suv n′ (u, v)

k[ ]n′

( ∗ k)[ , ] = [ , ]Suv nx ny duv:k Suv nx ny duv:k

Y = X ∗ k = F[u, v] ∗ k = (F[u, v] ) WH ‾ ‾‾‾‾√ ∑u,v

Suv WH ‾ ‾‾‾‾√ ∑u,v

duv:k Suv

32


What’s more, the diagonal elements of are the Fourier transform of (assuming it’s of size ).

This is the convolution theorem.Computational advantage for performing (and inverting!) convolution, albeit under circular padding.Good way of analyzing what a kernel is doing by looking at its Fourier transform.

= S Ak Dk S∗

Dk k W × H

= diag( k)Dk 1WH‾ ‾‾‾√ S∗

33


34


35


36


37


38


39


40


41


42


43


44

SCALE & ALIASINGSCALE & ALIASING

45


46


47


48


49


50


51


52


53


54


55


56


57


58


59


60


61


62


63


64


65


66


67


68


69


70


71

EFFICIENT COMPUTATIONEFFICIENT COMPUTATIONConvolution, in the most general case, takes time.

, .

Convolution in the frequency domain:FFT, point-wise multiply, Inverse FFT

FFT/IFFT complexity is (Most efficient for power of 2 image size)

May be worth it for large kernelsOr same image convolved with many different kernels

O( )nxnk

=nx WxHx =nk WkHk

O( log )nx nx

72

EFFICIENT COMPUTATIONEFFICIENT COMPUTATIONSeparable Kernels

and derivatives of Gaussian also separable.

Realize that .

This is by interpreting and as having size and .

So . This takes operations instead of .

O�en if a kernel itself isn’t separable, it can be sometimes expressed as a sum of separable kernels.

E.g., Unsharp Mask: (don’t combine!)

G[ , ] ∝ exp(− ) = [ ] [ ]nx ny+n2x n

2y

2σ2Gx nx Gy ny

x− y−

k[ , ] = [ k[ ] =nx ny kx nx]y ny kx ∗full ky

kx ky × 1Wk 1 × Hk

X ∗ k = X ∗ ( ∗ ) = (X ∗ ) ∗kx ky kx ky +Wk Hk WkHk

(1 + α)δ − αGσ

73

CSE 559A: Computer Visionayan/courses/cse559a/PDFs/lec...FOURIER TRANSFORM and are both 2D array of...

Documents

Transcript of CSE 559A: Computer Visionayan/courses/cse559a/PDFs/lec...FOURIER TRANSFORM and are both 2D array of...