Agenda

• Seam-carving: finish up• “Mid-term review” (a look back)• Main topic: Feature detection

Seam carving: construct an energy image

Define (vertical) seam

All the pixels in the seam are removed (shift row/column) Visual artifacts are visible only along the seam

k=1 k=2 k=M

1 1{ } { ( ), } , such that: , ( ) ( 1)x x N Ni i is s x i i i x i x i k

Slide credit: Andrea Tagliasacchi

4

Define cost function for seam• Energy function:

(gradient)

• let I be an n×m image and define a vertical seam to be:

• Optimal seam s* that minimizes this seam cost :

How to efficiently compute s* ?

Find best seam with dynamic progamming

, 1, 1 1, 1, 1( , ) min , ,i j i j i j i jM e i j M M M

For M row by N column image:

Scan every row in the image from i=2 to i=M updating with local best choice

In the final bottom row we pick the smallest entry Mij (for i=M) and we backtrack a path choosing always the local minima

Slide credit: Andrea Tagliasacchi

i-1Mi,j = cost of best seam from top row (i=1) down to pixel i,j

i

j j+1j-1

(c) ariel shamir

Alternative formulation: mincut on a graph

pi+1,j

pi,j

pi+1,j+1

pi,j+1y

x

x

y

pi,j pi,j+1

pi+1,j pi+1,j+1

y y

x

x

S T

-Each pixel is a node that’s connected to its 4 neighbors and a “source” and “terminal”-Weight edges appropriately (eg, gradient magnitude)-Find minimum cost cut that separates S and T – this is a “graphcut” problem

-Graphcuts is a very common tool in pixel labeling problems-Can solve mincut/maxflow problem (cf, Algorithms textbook)-With a particular choice of of weights on edges, the min cut is equivalent to DP soln

(c) ariel shamir

Extension to video: 3D graph of pixels

Frame t

Frame t+1

Frame t+2 Time

Video Cube

(c) ariel shamir

3D Graph Cut

Video Cube

Video

Halfway done with course; take a breather

• Midway evaluation form is posted on EEE

– First-time course, so please provide feedback!

A look back

• Part I: Introduction– Camera optics– Color– Fourier/filtering

• Part II: (Photography) Image enhancement– Texture synthesis

• Texture models: histograms of textons/filter responses• Markov models: sampling from conditional probability tables

– Image blending • Gradient-domain editing• Constrained optimization (lagrangian techniques)

– Image matting• Compositing• Bayesian modeling• Gaussian color models

– Image retargeting (resizing)• Dynamic programming• Combinatorial optimization (graphcuts)

• Part III: (Vision) Visual analysis– Feature matching– Mosaicing/stitching images– Recognition (finding and recognizing faces)