1

Today’s TopicsWritten assignment due today

Today: More on edge detection

• Edge detection using zero-crossings

• Scale-space techniques

• Case study: Intelligent Scissors

Advanced Edge Detection TechniquesMotivating example #1

• How do we extract edges corresponding to differentlevels of image detail?

Input image

Detected edges(high image detail)

Detected edges(low image detail)

2

Advanced Edge Detection TechniquesMotivating example #2

• How do we extract edges in the presence of noise in theinput image?

Input images

Edge-enhanced images

Low-noise High-noise

Edge-enhanced imagew/ noise comphensation

• Examples

• Gradient magnitude: gives the “strength” of the edge

• Gradient orientation: gives the orientation of the edge

Last Time: The Image Gradient

22

c

I

r

II

∂∂+

∂∂=∇

∂∂∂∂

=φ −

cIrI

tan 1

∂∂=∇c

I,0I

∂∂=∇ 0,r

II

∂∂

∂∂=∇

c

I,

r

II

φ

3

• Algorithm:

1. Compute the gradient of an image

2. Compute the gradient magnitude at every pixel

3. Label as “edge pixels” all pixels whose gradientmagnitude is above a pre-determined threshold

• Difficulties:

How do we determine the right threshold to use??

How do we control sensitivity to noise?

Detecting Edges of Any Orientation

• We will now consider 3 ideas:

• Using zero-crossings of 2nd-derivative to detect edges

• “Smoothing” images using Gaussian masks

• Defining the notion of “scale” for processing images atvarious levels of detail

Advanced Edge Detection Techniques

4

Today’s TopicsMore on edge detection

• Edge detection using zero-crossings

• Scale-space techniques

• Case study: Intelligent Scissors

Edge Enhancement as Image Differentiation

• 1st derivative:– positive or negative in the neighborhood of an edge– zero in image regions of constant intensity

• 2nd derivative:– crosses zero in the neighborhood of an edge– advantage: can be used to detect edges without

relying on a threshold

zero crossings

5

Edge Enhancement as Image Differentiation• Discrete approximation to a 1D 1st derivative:

• Discrete approximation to a 1D 2nd derivative:

• 2nd derivative mask:

)x(I)1x(I)x(Idx

d −+≅

)1x(Idx

d)x(I

dx

d)x(I

dx

d2

2−−≅

( ) ( ))1x(I)x(I)x(I)1x(I −−−−+≅

)1x(I)x(I2)1x(I −+−+≅

1 -2 1

Edge Detection in 1D Using 2nd Derivatives1. Apply the 2nd derivative mask to the input 1D image

2. Find all zero-crossings in the resulting edge-enhanced image

1 1 1 50 50 50 60 60 1 1 1

0 49 -49 0 10 -10 -59 59 0

49 -49 10 -10 -59 59

6

• How do we generalize the notion of the zero-crossing of a 2nd derivative to 2D images?

The Image Laplacian

∂∂

∂∂=∇

c

I,

r

II

1st order

Option #1: Compute all 2nd-order partial derivatives

rc

I,

cr

I,

c

I,

r

I 22

2

2

2

2

∂∂∂

∂∂∂

∂∂

∂∂

2nd order

Option #2: Compute Laplacian

2

2

2

22

c

I

r

II

∂∂+

∂∂=∇

2nd orderProperties of Laplacian:• Single scalar value for every pixel• Zero when both 2nd derivatives are zero

• What is the 2D mask corresponding to the image Laplacian?

The Image Laplacian

1 -2 1

1D case

0 0 0

2nd derivative along a row

1 -2 1

0 0 0

0 1 0

2nd derivative along a column

0 -2 0

0 1 0

+ =

0 1 0

Laplacian mask(sum of the 2 derivatives)

1 -4 1

0 1 0

2

2

2

22

c

I

r

II

∂∂+

∂∂=∇

Laplacian

7

Edge Detection in 2D Using Laplacian1. Apply the Laplacian mask to the input 2D image

2. Find all zero-crossings in the resulting edge-enhanced image

5 10 0

-2 -2 1

5 5 10

• In general, pixels in the Laplacian image may not be exactly zero• A pixel is labelled as zero crossings if one of its 8 neighbors has opposite sign

Zero-crossing Detection ExampleInput image Zero crossings

Q: How can we eliminate most of the noise-induced clutter in the image?

“Strong” zero crossings

8

Dealing with Noisy ImagesIdeally, we would like to remove noise from the input imagebefore doing edge enhancement/detection

Edge-enhanced imagesLow noise

High noise

General ApproachSteps:

• Given input image I, compute a new, “smoothed”image I’

• Apply edge enhancement/edge detection to thesmoothed image

Compute zero crossings of the Laplacian of I’

9

Noise Removal (a.k.a Image Smoothing)How can we smooth noisy images?

• Simplest idea: Replace pixel with the average of itsneighbors

• Averaging mask:1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

Degree of smoothing can be controlled by choosing masksof different sizes

Noise Removal Using GaussiansThe averaging mask gives the same weight to every point ina pixel’s neighborhood

An entirely different smoothing operation can be defined byassigning larger weights to pixels that are closer to the centerpixel

In 1D, one such smoothing operation is given by theGaussian function (usually m assumed to be 0):

( )2

2

2

mx

2e

2

1)x(G σ

−−

πσ=

10

Gaussians in 1D and 2D

( )2

2

2

mx

2e

2

1)x(G σ

−−

πσ=1D Gaussian

2D Gaussian( ) ( )

2

2c

2r

2

mcmr

2e

2

1)c(G)r(G)c,r(G σ

−+−−

πσ==

Noise Removal Using GaussiansDegree of smoothing controlled by parameter σ2 (variance)

How can we create a mask for Gaussian smoothing?

1. Sample the Gaussian at regular intervals on a square grid

2. Choose a mask size that captures most of the “mass” of the Gaussian (usually sufficient to choose masks whose elements span the range [-3σ,3σ])

G(0) G(1) G(2) G(3)G(-3) G(-2) G(-1)

(mask for σ=1)

11

Normalized vs. Unnormalized MasksWhat happens if we apply to an image a smoothing maskwhose elements sum to a value α ≠ 1?

Ans: The resulting image will appear brighter (α > 1) or darker (α < 1) than the original

Normalized Gaussian mask (σ = 1):

G(0) G(1) G(2) G(3)G(-3) G(-2) G(-1)

1 1 1 1 1 1 1 111 11

α αα ......

G(0) G(1) G(2) G(3)G(-3) G(-2) G(-1) G(0) *1α

Laplacian of the Gaussian

Idea: Rather than smooth image first using a Gaussian & thencomputing the Laplacian of the result, we can derive a maskthat does both operations simultaneously

)c,r(G

Laplacian of Gaussian(L.O.G.)

2

2

2

22

c

G

r

G)c,r(G

∂∂+

∂∂=∇

0 1 0

1 -4 1

0 1 0

Note similarity to 3x3 Laplacian mask

L.O.G. mask computed by sampling the L.O.G. function on a discrete grid(mask size chosen to be identical to that of the corresponding Gaussianmask)

12

Marr-Hildreth Edge Detection Algorithm

Given: An input image I and a value for the variance σ2

Steps:

1. Compute the L.O.G. mask corresponding to the desired value of σ2

2. Apply the L.O.G. mask to image I

3. Compute zero crossings of the result

4. Label as edges all zero crossing pixels

5. Optional step: label as edges only those zero-crossings whosetransition from + to - or from - to + is greater than a threshold

Input imagezero crossings

σ=1.0zero crossings

σ=5.0

One Remaining Question...

How do we choose the right value for σ ?

σ is generally referred to as the scale parameter

13

Today’s TopicsMore on edge detection

• Edge detection using zero-crossings

• Scale-space techniques

• Case study: Intelligent Scissors

Choosing the “Right” Scale??

Rather than trying to decide which scale to use, we canprocess images at all scales & examine how the resultingimage changes with increasing values of σ

Basic principles of scale-space approaches• No single scale is “special”• Treat s as a continuous parameter• As s increases, we should obtain “simpler” images, i.e., no “new structures” should be created• “Structures” that remain present for a broad range of scales signify “important” features in the image

14

Scale Space in 1D

input signal (σ=0)

smoothed signal (low σ)

smoothed signal (high σ)

Scale Space in 1D

Resulting Scale-Space Surface

x

σ

15

Zero Crossings in 1D

input image (σ=0)

smoothed signal (low σ)

smoothed signal (high σ)

Idea: As the 1D image gets increasingly smoothed, the position of zero crossings moves along the smoothed curve, creating trajectory curves

Zero crossings of 2nd image derivative = inflections of the intensity function

Scale Space in 1DIdea: As the 1D image gets increasingly smoothed, the position of zero crossings moves along the smoothed curve, creating trajectory curves

Trajectory of a single zero-crossing

• For the Gaussian function, trajectories form nested, upside-down U’s• The Gaussian smoothing function is the only function that ensures no new zero-crossings are created as σ increases• Therefore, it is the only function for which zero crossing trajectories have the nesting property

16

Scale Space in 2D• Zero crossing contours form surfaces in (r,c)-σ space• No new zero-crossings created as σ increases

Today’s TopicsMore on edge detection

• Edge detection using zero-crossings

• Scale-space techniques

• Case study: Intelligent Scissors

17

MotivationQuestion: How easy would it be for an image analysissystem to automatically find the person’s outline (or that ofa vase in the picture) from the output of an edge detector?

Motivation: Image ScissoringBy scissoring portions of one or more images & pastingthem together we can create new, composite images

sour

ce im

ages

composite image

18

Image Scissoring: Requirements• Interactive operation

• “User is always right”

Scissoring system must allow a user to select arbitraryregions

• Scissoring operations must be performed efficiently

• Scissoring interface must be simple & easy to use

Image Scissoring• Brute-force approach

User chooses every single image pixel that defines the

region boundary

• Here: Intelligent Scissors approachdeveloped by Eric Mortensen & presented at SIGGRAPH’95

19

Intelligent Scissors Basic operation

• User loads an image & specifies a “seed” point on theboundary that must be outlined

• User then positionsthe mouse close tothe object boundary

• System automaticallycreates a “live-wire”that connects seed &current mouseposition & that followsthe region boundary as much as possible

• Live wire is updated whenever the mouse moves

Approach answers one basic question:

• How should we define a path from seed to mouse thatfollows an object boundary as closely as possible?

• Answer: Define a path that is as close as possible toimage edges

Intelligent Scissors: Basic Idea

20

Approach taken in intelligent scissors attempts to exploituser interaction while avoiding the need to detect edgescorresponding to object boundaries very accurately

Intelligent Scissors: Basic Idea

Solution:

• Assign an “edge-ness” weight to every pixel in the image(computed using an edge-enhancement operation)

• Weights defined so that edge pixels have very lowweights

• To connect seed & mouse positions, choose the paththat minimizes the total weight of pixels along the path

Intelligent Scissors: Basic Idea

21

Two questions must be answered to fully-specify thealgorithm:

• How do we assign weights to pixels?

• How do we find the lowest-cost path between any twoimage points?

Intelligent Scissors: Basic Idea

Path: a sequence of adjacent pixels in the imageLink: a pair of pixels along the path

In the scissoring algorithm, weights are actually assigned tolinks, not individual pixels

Given a link defined by pixels p & q, the link weight is definedto be

Intelligent Scissors: Weight Assignment

)q(f14.0)q,p(f43.0)q(f43.0)q,p(l GDz ++=

if Laplacian=0 at potherwise

0)q(fz =1)q(fz = )Imax(

|I|1)q(fG ∇

∇−=

largest possiblevalue in image

term that penalizes links where the gradient orientation

at p and q is very different

22

Intelligent Scissors: Path OptimizerPath optimization formulated as a graph search algorithmthat computes the minimum-cost, n-link path from seed toall other image pixels

Weights

Step 1 Step 2 Step 3

Step n

Intelligent Scissors: Results

23

Intelligent Scissors: Results

Intelligent Scissors: ResultsBy scissoring portions of one or more images & pastingthem together we can create new, composite images

sour

ce im

ages

composite image

24

Today’s TopicsMore on edge detection

• Edge detection using zero-crossings

• Scale-space techniques

• Case study: Intelligent Scissors

Next time: Another image primitive--Regions

• Drawing polygonal regions (polygon filling, etc)

• Segmenting real images into regions