Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image...

63
Introduction to image processing and filtering Florence Tupin Athens Week

Transcript of Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image...

Page 1: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Introduction to image processing and

filtering

Florence Tupin

Athens Week

Page 2: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Aim of the course

Understand the content of the Fourier transform

of an image

Be able to associate an image and its FT

Understand the aliasing phenomenon

Be able to predict the effect of sub-sampling

Understand what is 2D convolution

Page 3: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

09/11/2015

Introduction

Image transforms

Image sampling

2D filtering

Page 4: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

09/11/2015

Introduction

Image transforms

Image sampling

2D filtering

Page 5: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

5

What is an image ?

Support of a message

2D continuous signal given by a physical measure

• Passive imagery

• Active Imagery (X-rays, emission of electro-magnetic waves, …)

Dimensions

• 2D (images)

• 3D (volumes –medical imagery-)

• Video (2D+t)

• 3D+t (sequences of volume images)

Page 6: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

6

What is a numerical image ?

A numerical image (2D or 3D) is a matrix defined by :

Its resolution (size of a pixel “picture element”)

Its depth (8 bits, 16 bits, float, vector,…)

Its color Look Up Table (CLUT)

Page 7: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

7

Numerical image

Page 8: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

8

Many different images Image Application Size # channels

Video Visiophony

TV

HD TV

256 x 256

720 x 625

1920 x 1150

1

3

3

Bio-medical Computerized Tomography

IRM

Radiography

512x512x512

256x256x256

1024 x 1024

1-3

Remote Sensing 1970-1980

1985-1990

1990-2010

2010-

2000 x 3000

6000 x 6000

15000 x 15000

40000 x 40000

3-7

3-20

3-256

3-8461 (!)

Robot vision Quality control

Automatic driving

256 x 256

512 x 512

1

2-3

Defense Survey, tracking 256 x 256

512 x 512

1

2-3

Page 9: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

9

Digitization of an image

Sampling ⇨ number of pixels

Quantification ⇨ number of bits per pixel

Page 10: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

10

sampling ⇨ number of pixels

Quantification ⇨ number of bits per pixel

Digitization of an image

Page 11: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

09/11/2015

Introduction

Image transforms

Image sampling

2D filtering

Page 12: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Image transforms

Image :

),(),(

:

yxgyx

RRRg

x

y

Page 13: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150
Page 14: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

2D Fourier Transform

),(),(),(),)((),( yxyxyx ffHffGffRyxhgyxr

dxdyeyxgffG

RLRLg

yfxfj

Ryx

yx )(2

2122

),(),(

)()(

Spatial representation

Space domain Frequential representation

Frequency domain

Properties

y

x

y

x

yxyx

f

fM

f

f

y

xM

y

x

ffGffHyxgyxh

'

'

'

'

)','(),()','(),(

Page 15: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

2D Discrete Time Fourier Transform

Numerical image : set of samples g(n,m) taken

on a regular square grid

Normalized spatial frequencies

Sampling 2D periodicity of the spectrum

in time in frequencies

]2

1,

2

1[]

2

1,

2

1[),(

),(),()(2

yx

mfnfj

n myx

ff

emngffG yx

Page 16: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Example of DTFT

fx

fy

Page 17: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

2D discrete Fourier Transform (DFT2D)

Numerical Image : set of samples g(n,m) taken on a rectangular

square grid and with a borned support : N columns, M lines and

sampling in the frequential space

Discrete spatial frequencies

Spatial sampling 2D periodicity of the spectrum

Frequential sampling 2D periodicity of the image

]2

,12

[]2

,12

[),(

),(),()(21

0

1

0

MMNNlk

emnglkGM

lm

N

knjN

n

M

m

Page 18: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

2D discrete Fourier Transform (DFT-2D)

Page 19: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Properties of the DFT-2D

Value of the DFT in (0,0):

Mean of the image xNM

Visualization of the logarithm of the modulus of

the DFT

If the image is real:

Hermitian symmetry of the DFT

Symmetry vs origin for the modulus of DFT

Complex DFT : the quantity of information

remains inchanged

]2

,12

[]2

,12

[),(

),(),()(21

0

1

0

MMNNlk

emnglkGM

lm

N

knjN

n

M

m

Page 20: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Properties of the DFT-2D

),(),()),(),((

),(),()),(),((

),()),((

),()),((

),(),(),(),(

),(),(),(

2

2

2

00

00

2

00

00

lkHlkFmnhmnf

lkHlkFmnhmnf

elkFmmnnf

lkFemnf

lkFlkFmnfmnf

qplkFqlpkF

D

D

lmknj

mnj

DFT

DFT

DFT

DFT

DFT

ZxZ

Page 21: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Inverse DFT-2D

Going from frequencies to spatial samples:

]1,0[]1,0[),(

),(1

),()(2

12

2

12

2

MNmn

elkGNM

mngM

lm

N

knj

N

Nk

M

Ml

Page 22: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Example of DFT-2D : sinusoïd

else 0

for )(

)()(

else 0

0 if )(

)()(),()()(),(

),(

0)

)((21

0

)(21

0

2)(21

0

)(21

0

2

00

0

0

0

qkNekG

eeengkG

lMelG

lGkGlkGmgngmng

emng

N

qf

N

nkqjN

n

x

N

knjN

n

N

nqj

N

knjN

n

xx

M

lmjM

m

y

yxyx

N

nqj

Page 23: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Case of a sinusoïd

fy

fx

Point (q0,0) : vector F0=(q0,0)’

• norm of the vector increases when the variations increase

• F0 is orthogonal to the invariance direction of the pattern

Page 24: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Case of a sinusoïd

fy

fx

Point (q0,0) : vector F0=(q0,0)’

• norm of the vector increases when the variations increase

• F0 is orthogonal to the invariance direction of the pattern

Page 25: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Variation of frequency

Page 26: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Variation of frequency

Page 27: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Variation of orientation

Page 28: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Example of DFT-2D : « door » signal

)sin(

)'sin(

'

1)(

1

1

'

1

'

1)(

else 0

0 if )(

)(Re'

1),(

))1'(

(

)(2

)'

(2)(21'

0

)(21

0

'

N

kN

kN

eN

kG

e

e

Ne

NkG

lMelG

nctN

mng

N

Nkj

x

N

kj

N

kNj

N

knjN

n

x

M

lmjM

m

y

N

Page 29: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150
Page 30: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Phase and modulus of the DFT-2D

Modulus of the spectrum :

Frequential contributions, orientation of the

structures

Phase of the spectrum

Linked to the localization in space

(translation: variation of the phase)

lmknjelkFmmnnf 002

00 ),()),((

DFT

Page 31: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Examples of DFT-2D

Page 32: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Examples of DFT-2D

Page 33: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Information of the phase

Phase image Re-transform using only magnitude

Page 34: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

TF(I1)= G1 (|G1|, arg(G1)) TF(I2)= G2 (|G2|, arg(G2))

Page 35: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Reconstruction with inversion of modulus

Inverse FFT (|G1|, arg(G2)) Inverse FFT (|G2|, arg(G1))

Page 36: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Case of textures

Reconstruction with random phase

Page 37: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

09/11/2015

Introduction

Image transforms

Image sampling

2D filtering

Page 38: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

1D Sampling

Preserving the signal content

1D Aliasing: wrong frequency

Page 39: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

2D sampling

Preserve the frequential content of an image

2D aliasing: wrong orientation and periodicity for periodical

pattern !

Page 40: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Sampling

Shannon-Nyquist Theorem:

Reconstruction formula

Generalization : anisotropic sampling

)2,2max(

],[],[support frequency limited with ),(

yxe

yyxx

BBF

BBBByxg

)(sin)(sin),(),(

]2

,2

[]2

,2

[),( 1

),(

),(),(),(

2

2

e

c

e

c

p q ee

rec

eeeeyx

e

yx

p q

eyexeyxyxrec

F

qy

F

px

F

q

F

pgyxg

FFFFff

FffH

qFfpFfGFffHffG

Page 41: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150
Page 42: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150
Page 43: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Sampling

Must be done at a sampling frequency at least two times

superior to the highest frequency of the 2D signal (called

Nyquist frequency)

Else, the original image must be low-pass filtered at a

frequency which is the sampling frequency divided by 2

Page 44: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150
Page 45: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Sub-sampled image (factor 2)

Page 46: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Effect of sub-sampling

Page 47: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Original spectrum Zoom on the sub-sampled spectrum

Page 48: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Sub-sampling

Original spectrum defined on [-0.5,0.5]

The original 2D spectrum on [-0.5,0.5] x [-0.5,0.5] is periodic

with replication on the points (0,1), (0,2),…(p,q),…

When sub-sampling with a factor 2:

The original 2D spectrum on [-0.5,0.5] x [-0.5,0.5] is now

periodic with replication on the points (0,0.5),

(0,1),…(p/2,q/2),…

The different patterns overlap in the Fourier domain

The new spectrum corresponds to the area of the old spectrum

in [-0.25,0.25] x [-0.25,0.25]

Page 49: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150
Page 50: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

(0,0) (1,0) (-1,0)

Periodization on (p,q)

Page 51: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

(0,0) (1,0) (-1,0)

Periodization on (p/2,q/2)

Page 52: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

(0,0) (1,0) (-1,0)

Selection of the area [-0.25,0.25]x [-0.25,0.25]

Page 53: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Sub-sampling

To respect Shannon conditions:

Multiplication of the spectrum by an indicative

function

Convolution of the image by a bidimensional

cardinal sinus

Apparition of oscillating patterns around the

image discontinuities (edges)

Page 54: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Gibbs Phenomenon (ringing)

Page 55: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

09/11/2015

Introduction

Image transforms

Image sampling

2D linear filtering

Page 56: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Linear filtering

),(),(),( lmknxlkhmnylk

),(),(),( yxyxyx ffXffHffY

Spatial domain

(h convolution kernel)

Frequency domain

Page 57: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Linear filtering

example

Page 58: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Linear filtering

Low-pass filtering

denoising using a mean filter:

suppression of the high frequencies (mean of N samples : divides

the noise variance by N)

Introduction of blur on the edges

Page 59: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Linear filtering

High-pass or band-pass filters:

Selection of frequencies of interest:

• High frequencies (edges)

• Particular frequencies (texture analysis)

Page 60: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

High-pass filter

Page 61: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Band-pass filter

y

x

v

u

Page 62: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Non-linear filtering

Median filter

The output of the filter is the median value of the

pixel values inside the analysis window

Rank filters

Min / Max : mathematical morphology

Page 63: Introduction to image processing and filtering · 2017-12-12 · 8 Many different images Image Application Size # channels Video Visiophony TV HD TV 256 x 256 720 x 625 1920 x 1150

Example of median filter