Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier...
Transcript of Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier...
![Page 1: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/1.jpg)
Spectral Domain Image Processing
● Continuous and Discrete Fourier Transform● DFT issues
– Aliasing
– Smoothing/Leakage
– Edge Effects
– …
● Solutions– Zero-padding
– Centering
– …
![Page 2: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/2.jpg)
Book a bit slow start, but good if you are not familiar with the Fourier Transform.
I will use 1D examples, easy to visualize, principle generally applicable to 2D.
![Page 3: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/3.jpg)
Wikipedia/Wikimedia commons
![Page 4: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/4.jpg)
Fourier transform
NON-PERIODIC functions, with finite area under curve can be expressed as the integral of waves multiplied by a coefficient.
Images have finite extent.
Images have two dimensions, so not a 1D wave.
More like a ...
![Page 5: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/5.jpg)
Fourier series+++
+
=
Joseph Fourier: The Fourier series decompose a PERIODIC continuous signal into waves with frequency, amplitude and phase.
c.f. e.g. Taylor series/expansion
Fig. 4.1 in book
![Page 6: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/6.jpg)
Fourier transform
NON-PERIODIC functions, with finite area under curve can be expressed as the integral of waves multiplied by a coefficient.
Images have finite extent.
Images have two dimensions, so not a 1D wave.
More like a ...
![Page 7: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/7.jpg)
Ruffled potato chips!
Wikimedia Commons
![Page 8: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/8.jpg)
Defined for a sampled image f(x, y) of MxN pixels:
where x = 0, 1, 2…M-1, y = 0,1,2…N-1 and u = 0, 1, 2…M-1, v = 0, 1, 2…N-1.
F (u , v )=∑x=0
M−1
∑y=0
N −1
f ( x , y )e− j 2 π (ux /M +vy /N )
The 2D Discrete Fourier Transform
(Book: eq. 4.5-15)
![Page 9: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/9.jpg)
Defined for a sampled image f(x, y) of MxN pixels:
where x = 0, 1, 2…M-1, y = 0,1,2…N-1 and u = 0, 1, 2…M-1, v = 0, 1, 2…N-1.
F (u , v )=∑x=0
M−1
∑y=0
N −1
f ( x , y )e− j 2 π (ux /M +vy /N )
The 2D Discrete Fourier Transform
(Book: eq. 4.5-15)
DFT (complex) IMAGE Wave: freq. u,v in x,y directioni.e. the ruffled potato chips
F(u,v) is also periodic!
![Page 10: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/10.jpg)
F(0,0) and basis functions
![Page 11: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/11.jpg)
A sum of corrugated surfaces with different frequency, amplitude and phase. Also remember
Euler’s formula.
e j θ=cosθ+ j sinθ
![Page 12: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/12.jpg)
Amplitude and PhaseEach Fourier component (i.e. u,v coordinate) is a
complex number
C = Re + jIm , C* = Re - jIm
Amplitude : |F(u,v)| = (Re**2 + Im**2)**0.5
Phase : φ(u,v) = arctan(Im/Re)
Amplitude=Spectrum=Magnitude
atan2 in
Matlab!
![Page 13: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/13.jpg)
= +
DFT ( )
Amplitude Phase
Reversible process!
The center F(u,v) is the average intensity of the image
![Page 14: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/14.jpg)
Defined for a sampled image f(x, y) of MxN pixels:
How do you get back? Inverse transform! i.e. sum those sine and cosines at each x,y coordinate.
F (u , v )=∑x=0
M−1
∑y=0
N −1
f ( x , y )e− j 2 π (ux /M +vy /N )
Inverse DFT
f ( x , y )=1
MN∑u=0
M−1
∑v=0
N−1
F (u , v )e j2 π (ux / M+ vy/ N )
(Book: eq. 4.5-15)
(Book: eq. 4.5-16)
DFT (complex) IMAGE Wave: freq. u,v in x,y direction
+
![Page 15: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/15.jpg)
A signal/function/image that can be represented by a finite number of Fourier components/terms.
Band limited function?
A Fourier transformed band limited function can be fully restored to its original. (μ=u)
Figure 4.6 in book
![Page 16: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/16.jpg)
● DFT works on finite images with MxN pixels→ Frequency smoothing/leakage
● DFT uses discrete sampled images i.e. pixels→ Aliasing
● DFT assumes periodic boundary conditions→ Centering, Edge effects, Convolution
DFT effects
![Page 17: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/17.jpg)
Images have borders, they are truncated (finite).This causes freq. smoothing and freq. leakage.
|F(u)|f(x)
![Page 18: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/18.jpg)
Aliasing - short version
Images consists of pixels, they are sampled.
Too few pixels → fake signals (aliasing)!
![Page 19: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/19.jpg)
Aliasing example (1D)
Figure 4.10 in book
![Page 20: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/20.jpg)
Aliasing examples (2D)
![Page 21: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/21.jpg)
Aliasing explained with DFT (1D)
Figure 4.6 in book
![Page 22: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/22.jpg)
Aliasing explained with DFT (1D)
Can then be inversed to get spatial function back. Or do analysis.
Figure 4.8 in book
![Page 23: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/23.jpg)
Aliasing explained with DFT (1D)
Figure 4.9 in book
![Page 24: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/24.jpg)
Aliasing explained with DFT (1D)
Remember!
ΔT <1
2μmax
Figure 4.7 in book
Figure 4.10 in book
![Page 25: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/25.jpg)
Aliasing explained with DFT - in 2D!
Figure 4.15 in book
![Page 26: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/26.jpg)
Aliasing explained with DFT - in 2D!
![Page 27: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/27.jpg)
and
Aliasing: Take home message
How do you avoid aliasing?1. Make sure signal is band limited. How?2. Sample with enough pixels!
What is enough pixels? Nyquist Theorem: The signal must be measured at least twice per period, i.e.:
![Page 28: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/28.jpg)
Sampling theorem
“A continuous, band-limited function can be recovered with no error from a set of its samples if the sampling intervals exceed twice the highest frequency present.”
and
![Page 29: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/29.jpg)
Quick pause to discuss
● What is frequency leakage? Why is it important?
● What is aliasing? Why is it important?How do you avoid aliasing?
● What do you think you can you do if your signal/image is not band-limited?
![Page 30: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/30.jpg)
Centering: Looking at DFTs
DFT
![Page 31: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/31.jpg)
Centering: Looking at DFTs
DFT
![Page 32: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/32.jpg)
Centering: Looking at DFTs
![Page 33: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/33.jpg)
Edge effects: Example
DFT
What do you see?
![Page 34: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/34.jpg)
Edge effects: Example
![Page 35: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/35.jpg)
Edge effects
● Spikes or lines in FT because of sharp-edged objects in image, spike is perpendicular to direction of the edge.
● Vertical spikes when there is a large difference in brightness between top and bottom of picture.
● Horizontal spikes when there is a large difference in brightness between left and right.
![Page 36: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/36.jpg)
Edge effects
● Spikes or lines in FT because of sharp-edged objects in image, spike is perpendicular to direction of the edge.
● Vertical spikes when there is a large difference in brightness between top and bottom of picture.
● Horizontal spikes when there is a large difference in brightness between left and right.
![Page 37: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/37.jpg)
Edge effects
● Spikes or lines in FT because of sharp-edged objects in image, spike is perpendicular to direction of the edge.
● Vertical spikes when there is a large difference in brightness between top and bottom of picture.
● Horizontal spikes when there is a large difference in brightness between left and right.
![Page 38: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/38.jpg)
Quick pause to discuss
● What does centering mean?Why is it useful?
● Give an example of edge effects. Explain why this happens.
● What would the main Fourier strikes be if we Fourier transform an image of a “T”?
![Page 39: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/39.jpg)
Fourier transform of characters
![Page 40: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/40.jpg)
The convolution theorem is your friend!
Convolution in spatial domain is equivalent to multiplication in frequency domain!
Much more convenient!
Convolution (c.f. last week)
![Page 41: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/41.jpg)
Convolution: Image vs DFT
Convolution is a multiplication in the spectral domain. However, we still do filtering in the spatial domain sometimes, why?
![Page 42: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/42.jpg)
Convolution: Image vs DFT
A general linear convolution of N1xN1 image with N2xN2 convolving function (e.g. smoothing filter) requires in the image domain of order
N12N2
2 operations.
Instead using DFT, multiplication, inverse DFT one needs of order
4N2Log2N operations.
Here N is the smallest 2n number greater or equal to N1+N2-1.
Rule of thumb: Use Image convolution for small convolving functions, and DFT for large convolving functions.
![Page 43: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/43.jpg)
Convolution example
What artifacts do you see in the convolved image?
![Page 44: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/44.jpg)
Convolution: Wrap-around errors
● Why? DFT assumes periodic functions/data.● Avoid by using zero padding! How much needed?● Consider two NxM images. If image 1 is nonzero
over region N1xM1 and image 2 is nonzero over region N2xM2 then we will not get any wrap-around errors if
If the above is not true we need to zero-pad the images to make the condition true!
![Page 45: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/45.jpg)
Convolution: Wrap-around errors
N should be at least 256+256-1=511
![Page 46: Spectral Domain Image Processing - Chalmersfy.chalmers.se/~romeo/RRY025/notes/M1.pdfFourier transform NON-PERIODIC functions, with finite area under curve can be expressed as the integral](https://reader030.fdocuments.in/reader030/viewer/2022040120/5e9558c890b7880e5764b780/html5/thumbnails/46.jpg)
DFT lecture 1
● What is 1D continuous FT?● The 2D Discrete Fourier Transform● Important things in a discrete world:
– Freq. Smoothing and leakage
– Aliasing
– Centering
– Edge effects
– Convolution
● Coming up: Matlab exercises