EDGE DETECTION · Image Segmentation: Edge Detection 10 A continuous approach for edge detection:...
Transcript of EDGE DETECTION · Image Segmentation: Edge Detection 10 A continuous approach for edge detection:...
Image Segmentation: Edge Detection 1
EDGE DETECTION
1. Edge model
2. Classical approaches
3. Analytical approaches
4. Snakes
5. Edge representation
Image Segmentation: Edge Detection 2
des formes
contours
zones
textures
mesuresreconnaissance
analyse de scènes
segmentation
Image Segmentation: Edge Detection 3
Détection de contours
Fermeture de contours
Détection de zones homogènes
Obtentation de contours
Formes bi-dimensionnelles
Paramétrisation
Reconnaissance des formes Classification Analyse de scène
Image Segmentation: Edge Detection 4
What is an edge ?
Image Segmentation: Edge Detection 5
An image as a surface
Image Segmentation: Edge Detection 6
What is an edge ?
rampemarche d’escalier toit
Image Segmentation: Edge Detection 7
A continuous approach for edge detection: the gradient
The image is supposed to be a continuous surface: i(x, y)
The gradient of i(x, y):−→G =
−→∇i =
∂i∂x
∂i∂y
G = |
−→∇i| =
[(∂i∂x
)2+
(∂i∂y
)2]1/2
and −→g =−→∇i
|−→∇i|
egde = maxima of the gradient in the direction of the gradient
∂G∂g = 0 and ∂2G
∂g2 < 0 with ∂∂g = −→g .
−→∇
∂i∂x .
∂∂x
√(∂i∂x
)2+
(∂i∂y
)2
+ ∂i∂y .
∂∂y
√(∂i∂x
)2+
(∂i∂y
)2
= 0
Image Segmentation: Edge Detection 8
A continuous approach for edge detection: the gradient
Derivative approaches: discrete approximation
1 1
-1
-1
-1
-1
1
2
1
-1
-2
-1
gradient Roberts Prewitt Sobel
1
1
1-1
Norm approximation: G = | ∂i∂x | + | ∂i
∂y |.
Pre-processing: filtering (median), ...
Post-processing: thresholding, local maxima of the gradient, hysteresis, following, closing
Image Segmentation: Edge Detection 9
A continuous approach for edge detection: the laplaciany
g
t
x
Φ
Edge: ∂2i∂g2 = 0 ⇔ ∂2i
∂x2 cos2 Φ + 2. ∂2i∂x∂y cos Φ sinΦ + ∂2i
∂y2 sin2 Φ = 0
Laplacian: ∆i = ∂2i∂g2 + ∂2i
∂t2
if ∂2i∂t2 ≈ 0 then ∂2i
∂g2 = 0 ⇔ ∆i = 0
∂2i∂t2 ≈ 0 ⇒ low curvature
Isotropy: ∆i = ∂2i∂x2 + ∂2i
∂y2
Image Segmentation: Edge Detection 10
A continuous approach for edge detection: the laplacian
Interest: closed edges (separation positive / negative values)
Very sensitive to noise: need for a strong filtering
edge =(∂2
∂x2 +∂2
∂y2
)[low pass filter ∗ image ]
edge =[ (
∂2
∂x2+
∂2
∂y2
)low pass filter
]∗ image
g(x, y) =1√2πσ
exp(− (x2 + y2)2σ2
)
∆g =1√
2πσ3((x2 + y2)
2σ2− 1) exp(− (x2 + y2)
2σ2)
Image Segmentation: Edge Detection 11
gaussian laplacian of gaussian
Image Segmentation: Edge Detection 12
Image Segmentation: Edge Detection 13
A continuous approach for edge detection: the laplacian
• laplacian of gaussian (LOG) : human visual system (Marr 80)
• difference of gaussians (DOG) σi
σe=1.6
• DOG approximated with DOB (difference of boxes)
filtre Log
filtre Dog
Image Segmentation: Edge Detection 14
Ameliorations
• DOG: separable filter
DOG(x, y) = DOG(x)DOG(y)
complexity: N2 ⇒ 2N
• Huertas and Medioni 86
DOG(x, y) = H1(x)H2(y) +H1(y)H2(x)
with H1(x) = (1− x2
σ2 )exp(− x2
2σ2 ) and H2(x) = exp(− x2
2σ2 )
RIF :
H1 = [−1− 6− 17− 17 18 46 18− 17− 17− 6− 1]
H2 = [0 1 5 17 36 46 36 17 5 1 0]
Image Segmentation: Edge Detection 15
Analytical approach: “optimal” edge
Canny 83: 1D step edges, linear filtering, gaussian white noise
I(x) = AU(x) + n(x)
O(xo) =∫ +∞−∞ I(x)f(xo − x)dx
f(x) ? (with f(x) = −f(−x) -derivative operator-)
• a good detection: Σ =∫∞0 f(x)dx√∫∞−∞ f2(x)dx
• a good localisation: Λ = |f ′(0)|√∫∞−∞ f ′2(x)dx
• a unique response:|f ′(0)|√∫∞
−∞ f ′′2(x)dx= k
∫ 0−∞ f(x)dx√∫∞−∞ f2(x)dx
Image Segmentation: Edge Detection 16
Resolution
Maximise ΣΛ
• Euler-Lagrage equation:∫ b
aψ(x, f, f ′, f ′′)dx =
∫ b
af2 + λ1f
′2 + λ2f′′2 + λ3fdx
⇒ 2f(x)− 2λ1f′′(x) + 2λ2f
′′′′(x) + λ3 = 0
• general solution:
f(x) = a1ex/σ sinωx+ a2e
x/σ cosωx+ a3e−x/σ sinωx+ a4e
−x/σ cosωx+ .
λ2 − λ214 > 0; 1
σ2 − ω2 = λ12λ2
; 4ω2
σ2 = λ21−4λ2
4λ22
• FIR filter on [-M,M]: f(0) = 0 f(M) = 0 f ′(0) = S f ′(M) = 0
• performances: ΣΛ = 1.12
Image Segmentation: Edge Detection 17
Analytical approaches around Canny
• Approximation: derivative of a gaussian
f(x) ≈ −x. exp(− x2
2σ2
)performance: ΣΛ = 0.92 (IIR)
max response to the filter = zeros of the second derivative (Marr)
• Shen et Castan (86)
maximisation of Σ
f(x) = csign(x) exp(−α|x|)
discontinuity at zero
better localisation / more sensitive to noire
Image Segmentation: Edge Detection 18
Analytical approaches around Canny
Deriche 1987
• IIR with conditions at the limits
f(0) = 0; f(+∞) = 0; f ′(0) = S; f ′(+∞) = 0;
• optimal solution
f(x) = −cx exp(−α|x|)cos(ωx)
• performance: Λ =√
2α ; Σ =√
2αα2+ω2
• optimal parameters:
ω → 0 ⇒ ΣΛ = 2f(x) = −cx exp(−α|x|)
c = − [1−exp(−α)]2
exp(−α)
• link with the gaussian: α =√
πσ
Image Segmentation: Edge Detection 19
Deriche operator: implementation
• Z transform
h(x) = cx exp(−α|x|) H(z) =∑+∞
n=∞ h[n]z−n
h[n] = h−[n] + h+[n]
H−(z) = ce−αz−1
1−2e−αz−1+e−2αz−2 and H+(z) = −ce−αz1−2e−αz+e−2αz2
second order recursive filters (stable)
• condition at the origine: B[0] = 1 ⇒ c = − (1−e−α)2
e−α
• finite difference equations
B1[n] = ce−αA[n− 1] + 2e−αB1[n− 1]− e−2αB1[n− 2]
B2[n] = −ce−αA[n+ 1] + 2e−αB2[n+ 1]− e−2αB2[n+ 2]
B[n] = B1[n] +B2[n]
NB : independant of α
Image Segmentation: Edge Detection 20
−10 −8 −6 −4 −2 0 2 4 6 8 10−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
Gaussian / Deriche
−10 −8 −6 −4 −2 0 2 4 6 8 10−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
Deriche / Shen & Castan
Image Segmentation: Edge Detection 21
2D implementation
calcul of the gradient⇒ appplication of a derivative operator
• f(x) : “optimal” derivative operator f
• g(x) : integral of f (smoothing)
g(x) = b(α|x|+ 1)e−alpha|x|
• 2D expression
∂i
∂x→ [f(x)g(y)] ∗ i(x, y) = f(x) ∗ g(y) ∗ i(x, y)
∂i
∂y→ [f(y)g(x)] ∗ i(x, y) = f(y) ∗ g(x) ∗ i(x, y)
Image Segmentation: Edge Detection 22
−10 −8 −6 −4 −2 0 2 4 6 8 10−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
f(x)
−10 −8 −6 −4 −2 0 2 4 6 8 100
0.05
0.1
0.15
0.2
0.25
g(x)
Image Segmentation: Edge Detection 23
2D Deriche’s filter
Image Segmentation: Edge Detection 24
Snakes or active contours
Kass, Witkins and Terzopoulos 1988
Evolution of a curve under internal and external constraints: v(s) = [x(s) , y(s)]t
where s is the curviligne abscisse: s ∈ [0, 1]
The global energy takes the form:
Etotal =∫ 1
0[Einternal(v(s)) + Eimage(v(s)) + Eext(v(s))] ds
with
Einterne = α(s)(
dvds
)2+ β(s)
(d2vds2
)2
where α(s) is the tension and β(s) represents the curvature or the elasticity of the curve.
Image Segmentation: Edge Detection 25
Resolution
• Euler-Lagrange Equation
−(αv′)′(s) + (βv′′)′′(s) +∇P (v) = 0
P (v) = Eimage(v) + Eext(v)
F (v) = −∇P (v)
( + conditions at the limits)
• Discretisation with the finite differences
V t = [vt0, v
t1, v
t2, ......, v
tn−1]
t
(step h, α, β cst)βh2 vi+2 − (α
h + 4 βh2 )vi+1 + ( 2α
h + 4 6βh2 )vi − (α
h + 4 βh2 )vi−1 + β
h2 vi−2 = F (vi)
Image Segmentation: Edge Detection 26
Resolution
Evolution of the curve with the time
γ∂v
∂t− αv′′ + βv′′′′ = F (v)
v(t+ 1) = (τA+ I)−1(τF (v(t)) + v(t))
• initialisation
• choice of τ
• matrix inversion (circular, Toplitz or not)
• discretisation step constant
• stop criterium
Image Segmentation: Edge Detection 27
Resolution
2α+6β β 0−α−4β
−α−4β 2α+6β β−α−4β
0 β −α−4β 2α+6β
0 β −α−4β 2α+6β
0
0
β
−α−4β
... ... ...
... ...
...
...
...
0
β −α−4β
0 β −α−4β
2α+6β −α−4β
2α+6β
−α−4β β 0
β −α−4β
closed snake: NxN
Image Segmentation: Edge Detection 28
Resolution
-2
-2
β −α−4β 2α+6β
0
β
−α−4β
0
... ...
... ...
...
...
0
...
β −α−4β
0 β
...
2α+6β −α−4β
2α+6β
0
0 0
0 0
β β 0β
0
β
−α−4β
β β
−α−4β β2α+5β−α−2β
−α−2β2α+5β−α−4ββ
snake with free extremities: NxN, v′′(0) = v′′′(0) = 0
Image Segmentation: Edge Detection 29
Resolution
-2
2α+6β β−α−4β
0 β −α−4β 2α+6β
−α−4β
0
0
β
−α−4β
... ... ...
...
...
...
...
0
β −α−4β
0 β −α−4β
2α+6β −α−4β
2α+6β
0
0 0
0 0
β 0
β...
β β
2α+6β −α−4β
snake with fixed extremities: (N-2)x(N-2)
Image Segmentation: Edge Detection 30
Choice of the constraintes
• linked to the features we are looking at: F (v) = ∇P (v)
– intensity: P (v) = I(v)
– edge: P (v) = −|∇I(v)|2 (or Canny-Deriche)
– edge: P (v) = (∆(I ∗Gσ))2 (0 of the laplacian)
• minimisation with dynamique programming (Amini)
positions à l’instant t snake à t
snake à t+1positions possibles à t+1
M
N
P
1
2
3
4
1
2
3
4
1
2
3
4
1
3
4
M N P Q
0
1
2
3
4Q
Image Segmentation: Edge Detection 31
Ameliorations
Balloon strength, Cohen 1991
Problems with classical snakes:
• bad initialisation: no attraction
• no constraints: retraction of the curve
F (v(s)) = k1−→n (s)− k ∇P
|∇P |−→n (s) normal vector at s
(k > k1 edge > pression strength, τk = pixel size)
initialisation : interior ou exterior of the object
(no need to be close to the object)
Image Segmentation: Edge Detection 32
Ameliorations Gradient Vector Flow, Xu and Prince 1997
• Goals:
– independant to the initialisation
– convergence to concave regions
• Diffusion of the gradient all over the image
minimisation of:
E =∫ ∫
µ(u2x + u2
y + v2x + v2
y) + |F (x, y)|2|−→v − F (x, y|2dxdy
• possibility to initialise through the object
Image Segmentation: Edge Detection 33
Classical snakes
Final result, iter = 500
0 10 20 30 40 50 60 70
10
20
30
40
50
60
Image Segmentation: Edge Detection 34
Classical snakes
Image Segmentation: Edge Detection 35
Gradient Vector Flow
Image Segmentation: Edge Detection 36
Gradient Vector Flow
Image Segmentation: Edge Detection 37
Edge representation
• list of the cartesian coordinates: (xi, yi), i = 1...n
• Freeman’s code
– cartesian coordinates of the first point
– list of the displacements
04
2 13
5 6 7
A B
6 6 7 0 1 7 6 0 1 0 2 4 3 1
Image Segmentation: Edge Detection 38
• Relative Freeman’s code
– first point + first displacement
– list of the direction changes
54 6 7
0 1 2 3 0 1 1 1 6 7 2 1 7 2 2 7 6
Image Segmentation: Edge Detection 39
Freeman’s code
0
1
2
3
0
123
4
5 6 7
3
91011
12
13
6 75
14
4
12
80
15
4 directions : 2 bits 8 directions : 3 bits 16 directions : 4 bits
1 1 1 0 1 0 0 0 0 0 0 3 0 3 0 3 0 3 3 2 3 2 3 0 3 0 0 0 0 0 1 0 1 0
2 2 1 1 0 0 0 0 7 7 7 7 6 5 4 6 7 0 0 0 0 1 0 1
10 9 8 8 15 15 6 5 4 6 7 8 8 1 0 1 16 x 4 = 64 bits
24 x 3 = 72 bits
34 x 2 = 68 bits
Image Segmentation: Edge Detection 40
Edge representation
Signature
• centre of gravity:origin of polar coordinates
• list of the distances to the origin according to the angle: ρ(θ)
• invariant to translations
• detection of small defects
• rotation = translation along θ
Image Segmentation: Edge Detection 41
Signature - examples
R