Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486,...

36
CSE486, Penn State Robert Collins Lecture 2: Intensity Surfaces and Gradients

Transcript of Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486,...

Page 1: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Lecture 2:Intensity Surfaces

and Gradients

Page 2: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Intensity pattern

Visualizing ImagesRecall two ways of visualizing an image

2d array of numbers

We “see it” at this level Computer works at this level

Page 3: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Bridging the Gap

Motivation: we want to visualize images at a level highenough to retain human insight, but low enough to allowus to readily translate our insights into mathematicalnotation and, ultimately, computer algorithms that operate on arrays of numbers.

Page 4: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Surface heightproportional topixel grey value(dark=low, light=high)

Images as Surfaces

Page 5: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Examples

Mean = 164 Std = 1.8

Note: see demoImSurf.m in matlab examples directory oncourse web site if you wantto generate plots like these.

Page 6: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Examples

Page 7: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Examples

Page 8: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Examples

Page 9: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Examples

Mean = 111 Std = 15.4

Page 10: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Examples

Page 11: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

How does this visualization help us?

Page 12: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Terrain Concepts

Page 13: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Terrain Concepts

www.pianoladynancy.com/ wallpapers/1003/wp86.jpg

Page 14: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Terrain ConceptsBasic notions:

Uphill / downhillContour lines (curves of constant elevation)Steepness of slopePeaks/Valleys (local extrema)

Gradient vectors (vectors of partial derivatives) will help us define/compute all of these.

More mathematical notions:Tangent PlaneNormal vectorsCurvature

Page 15: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Math Example : 1D GradientConsider function f(x) = 100 - 0.5 * x^2

Page 16: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Math Example : 1D GradientConsider function f(x) = 100 - 0.5 * x^2

Gradient is df(x)/dx = - 2 * 0.5 * x = - x

Geometric interpretation: gradient at x0 is slope of tangent line to curve at point x0

x0

f(x)

tangent line

Δy

Δx

slope = Δy / Δx

= df(x)/dx

x0

Page 17: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Math Example : 1D Gradientf(x) = 100 - 0.5 * x^2 df(x)/dx = - x

x0 = -2

grad = slope = 2

Page 18: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Math Example : 1D Gradientf(x) = 100 - 0.5 * x^2 df(x)/dx = - x

gradients 3

2

10

-3

-2

-1

Page 19: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Math Example : 1D Gradientf(x) = 100 - 0.5 * x^2 df(x)/dx = - x�

gradients 3

2

10

-3

-2

-1

Gradientson this sideof peak arepositive

Gradientson this sideof peak arenegative

Note: Sign of gradient at point tells you what direction to go to travel “uphill”

Page 20: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Math Example : 2D Gradientf(x,y) = 100 - 0.5 * x^2 - 0.5 * y^2df(x,y)/dx = - x df(x,y)/dy = - y

Gradient = [df(x,y)/dx , df(x,y)/dy] = [- x , - y]

Gradient is vector of partial derivs wrt x and y axes

Page 21: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Math Example : 2D Gradientf(x,y) = 100 - 0.5 * x^2 - 0.5 * y^2

Gradient = [df(x,y)/dx , df(x,y)/dy] = [- x , - y]

Plotted as a vector field,the gradient vector at each pixel points “uphill”

The gradient indicates thedirection of steepest ascent.

The gradient is 0 at the peak(also at any flat spots, and local minima,…butthere are none of those for this function)

Page 22: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Math Example : 2D Gradientf(x,y) = 100 - 0.5 * x^2 - 0.5 * y^2

Gradient = [df(x,y)/dx , df(x,y)/dy] = [- x , - y]

Let g=[gx,gy] be the gradient vector at point/pixel (x0,y0)

Vector g points uphill (direction of steepest ascent)

Vector - g points downhill (direction of steepest descent)

Vector [gy, -gx] is perpendicular,and denotes direction of constantelevation. i.e. normal to contourline passing through point (x0,y0)

Page 23: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Math Example : 2D Gradientf(x,y) = 100 - 0.5 * x^2 - 0.5 * y^2

Gradient = [df(x,y)/dx , df(x,y)/dy] = [- x , - y]

And so on for all points

Page 24: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Image Gradient

The same is true of 2D image gradients.

The underlying function is numerical(tabulated) rather than algebraic. So need numerical derivatives.

Page 25: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Numerical DerivativesSee also T&V, Appendix A.2

Taylor Series expansion

Finite forward difference

Manipulate:

Page 26: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Numerical DerivativesSee also T&V, Appendix A.2

Taylor Series expansion

Finite backward difference

Manipulate:

Page 27: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Numerical Derivatives

Finite central difference

See also T&V, Appendix A.2

Taylor Series expansion

subtract

Page 28: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Numerical DerivativesSee also T&V, Appendix A.2

Finite backward difference

Finite forward difference

Finite central differenceMoreaccurate

Page 29: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Example: Temporal Gradient

derivative

df(x)/dx

A video is a sequence of image frames I(x,y,t).

Each frame has two spatial indices x, y andone temporal (time) index t.

Page 30: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Example: Temporal Gradient

Consider the sequence ofintensity values observedat a single pixel over time.

derivative

df(x)/dx

1D function f(x) over time 1D gradient (deriv) of f(x) over time

Page 31: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Temporal Gradient (cont)What does the temporal intensity gradient at each pixel look like over time?

Page 32: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Example: Spatial Image Gradients

I(x+1,y) - I(x-1,y)

2I(x,y)

I(x,y+1) - I(x,y-1)

2

Ix=dI(x,y)/dx

Iy=dI(x,y)/dy

Partial derivative wrt x

Partial derivative wrt y

Page 33: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Functions of GradientsI(x,y) Ix Iy

Magnitude of gradient sqrt(Ix.^2 + Iy.^2)

Measures steepness ofslope at each pixel

Page 34: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Functions of GradientsI(x,y) Ix Iy

Angle of gradient atan2(Iy, Ix)

Denotes similarityof orientation of slope

Page 35: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Functions of Gradients

What else dowe observe inthis image?

atan2(Iy, Ix)

Enhanced detailin low contrastareas (e.g. foldsin coat; imagingartifacts in sky)

I(x,y)

Page 36: Robert Collins CSE486, Penn State - Penn State College of ...rtc12/CSE486/lecture02.pdf · CSE486, Penn State Robert Collins Terrain Concepts Basic notions: Uphill / downhill Contour

CSE486, Penn StateRobert Collins

Next Time: Linear Operators

Gradients are an example of linear operators,i.e. value at a pixel is computed as a linearcombination of values of neighboring pixels.