CS-F441: Selected Topics from Computer Science (Deep ... › dl › lecture-06-kt-cv.pdfCS-F441:...

CS-F441: SELECTED TOPICS FROM COMPUTER

SCIENCE (DEEP LEARNING FOR NLP & CV)

Lecture-KT-08: Canny Edge Detector, Feature Engineering

Dr. Kamlesh Tiwari,Assistant Professor,

Department of Computer Science and Information Systems,BITS Pilani, Rajasthan-333031 INDIA

Oct 30, 2019 (Campus @ BITS-Pilani July-Dec 2019)

Recap: Sharpening Filter

STCS-DL4NLP&CV (CS-F441) Campus @ BITS-Pilani Lecture-KT-08 (Oct 30, 2019) 2 / 31

Recap: Sharpening


Recap: Image gradient

The gradient of an image: Of = [ ∂f∂x ,

∂f∂y ]

Points in the direction of most rapid increase in intensity

where strength is ||Of || =√( ∂f∂x )

2 + ( ∂f∂y )

2

and direction is θ = tan−1( ∂f∂y /

∂f∂x )

Associative property of convolutionddx

(f ∗ h) = f ∗ ddx

h


Recap: Edges

An edge is a place of rapid change in the image intensity function

How to differentiate a digital image (filter ?)

∂f∂x

[x , y ] = F [x + 1, y ]− F [x , y ]


Recap: Roberts

Simple approximation to the first derivative.

It makes the edge point only, not the information about the edgeorientation.It works well in binary images.The Roberts method finds edges using the Roberts approximationto the derivative.√

(I(i , j)− I(i − 1, j − 1))2 + (I(i , j − 1)− I(i − 1, j))2[0 1−1 0

] [1 00 −1

]


Recap: Effects of Noise


Recap: Prewit Operator

Performs smoothening (by averaging) and then finds the magnitude ofderivatives and applies threshold on them.

The mask used is as below

hx =

1 0 −11 0 −11 0 −1

hy =

1 1 10 0 0−1 −1 −1

Bigger size mask is

hx =

−3 −1 1 3−3 −1 1 3−3 −1 1 3−3 −1 1 3

hy =

3 3 3 31 1 1 1−1 −1 −1 −1−3 −3 −3 −3


Recap: Sobel OperatorCommon approximation of derivative of Gaussian

Edge magnitude is√

s2x + s2

y and the direction is tan−1(sy/sx)

Edge is detected by using threshold


Recap: Non-maximum supression

Check if pixel is local maximum along gradient direction


Recap: Laplacian of Gaussian (Marr-Hildreth)It is high-pass, and non-directional filter. Uses second derivatives 0 −1 0

−1 4 −10 −1 0

−1 −1 −1−1 8 −1−1 −1 −1

Apply LoG to remove noise.

0 0 1 0 00 1 2 1 01 2 −16 2 10 1 2 1 00 0 1 0 0

1 Detect all zero crossings{+,−}, {−,+}, {+,0,−},{−,0,+}

2 Use threshold to keep onlystrong ones


Canny Edge Detector [1/3]

Standard edge detector: Maximizes probability of detecting real edgeswhile minimizing the probability of false detection of non-edge points.

Five separate steps:

1. Smoothing: Blurring of the image to remove noise. (by applying aGaussian filter). The kernel of a Gaussian filter with a standarddeviation of σ = 1.4 is given by

1159

2 4 5 4 24 9 12 9 45 12 15 12 54 9 12 9 42 4 5 4 2



2. Finding gradients: The edges should be marked where thegradients of the image has large magnitudes (by applyingSobel-operator).

3. Non-maximum suppression: Only local maxima should bemarked as edges.

1 Round the gradient direction to nearest 45◦, corresponding to theuse of an 8-connected neighborhood.

2 Compare the edge strength of the current pixel with the edgestrength of the pixel in the positive and negative gradient direction.i .e. if the gradient direction is north (theta = 90◦), compare with thepixels to the north and south.

3 If the edge strength of the current pixel is largest; preserve thevalue of the edge strength. If not, suppress (i .e. remove) the value.



4. Double thresholding: Edge pixels stronger than the highthreshold are marked as strong; edge pixels weaker than the lowthreshold are suppressed and edge pixels between the twothresholds are marked as weak. Strong edges are white, whileweak edges are grey. Edges with a strength below boththresholds are suppressed

5. Edge tracking by hysteresis: Final edges are determined bysuppressing all edges that are not connected to a very certain(strong) edge. Strong edges are interpreted as certain edges, andare included in the final edge image. Weak edges are included ifand only if they are connected to strong edges.


Canny Edge Detector


Humans are the best


General Object Detection Pipeline

1 Image acquisition2 Enhancement3 Feature Extraction4 Feature Matching5 Decision


Desired Properties of Feature

1 Repeatability: The same feature can be found in several imagesdespite geometric and photometric transformations

2 Saliency: Each feature has a distinctive description3 Compactness and efficiency: Many fewer features than image

pixels4 Locality: A feature occupies a relatively small area of the image;

robust to clutter and occlusion


What feature you would you choose?


Local measures of uniquenessSuppose we only consider a small window of pixelsWhat defines whether a feature is a good or bad candidate?

Measure feature uniqueness using How does the window changewhen you shift it?

flat region: no change in all directionsedge: no change along the edge directioncorner: significant change in all directions 1

1Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institute


What is the change?

If we shift the window W by (u, v) then SSD is

E(u, v) =∑

(x ,y)∈W

[I(x + u, y + v)− I(x , y)]2

Recall Taylor series expansion

I(x + u, y + v) = I(x , y) +∂I∂x

u +∂I∂y

v + ...

For small u and v we may neglect higher order terms so

E(u, v) =∑

(x ,y)∈W

[Ixu + Iyv ]2 =∑

(x ,y)∈W

[[

Ix Iy] [ u

v

]]2

where Ix = ∂I∂x and Iy = ∂I

∂y


Change continued...

E(u, v) =∑

(x ,y)∈W

[[

Ix Iy] [ u

v

]]2

E(u, v) =∑

(x ,y)∈W

[[

u v] [ I2

x Ix IyIy Ix I2

y

] [uv

]]2

E(u, v) =∑

(x ,y)∈W

[[

u v]

H[

uv

]]2

Move the window on blue circle.Which directions will have maximalE? Does eigenvectors of H helpsuggesting?


Change continued...

E(u, v) =∑

(x ,y)∈W

[[

Ix Iy] [ u

v

]]2

E(u, v) =∑

(x ,y)∈W

[[

u v] [ I2

x Ix IyIy Ix I2

y

] [uv

]]2

E(u, v) =∑

(x ,y)∈W

[[

u v]

H[

uv

]]2

Move the window on blue circle.Which directions will have maximalE?

Does eigenvectors of H helpsuggesting?


Change continued...

E(u, v) =∑

(x ,y)∈W

[[

Ix Iy] [ u

v

]]2

E(u, v) =∑

(x ,y)∈W

[[

u v] [ I2

x Ix IyIy Ix I2

y

] [uv

]]2

E(u, v) =∑

(x ,y)∈W

[[

u v]

H[

uv

]]2

Move the window on blue circle.Which directions will have maximalE? Does eigenvectors of H helpsuggesting?


Eigenvalue/eigenvector review


Feature Detection


Feature PointsDo the following

Compute gradient of each point in the imageCreate the H matrix using gradientsCompute EigenvaluesFind points with large response (λ > threshold)Choose those points where λ is a local maximum as feature point


Corner Response Function

f =λ1λ2

λ1 + λ2


Harris Operator

Do the following:1 Compute E for each image window to get their cornerness scores.2 Find points whose surrounding window gave large corner response

(f > threshold)3 Take the points of local maxima, i.e., perform non-maximum

suppression


Example of Harris Feature


Properties of the Harris corner detector

Rotation Invarient? YESScale Invarient? NO


Thank You!

Thank you very much for your attention2!

Queries ?

2https://www.cs.cornell.edu/courses/cs6670/2011sp/lectures/lec02 filter.pdf


CS-F441: Selected Topics from Computer Science (Deep ... › dl › lecture-06-kt-cv.pdfCS-F441:...

Documents

Transcript of CS-F441: Selected Topics from Computer Science (Deep ... › dl › lecture-06-kt-cv.pdfCS-F441:...