5/20/13

1

Global Alignment and Structure from Motion

Today’s Readings •  Photo Tourism (Snavely et al., SIGGRAPH 2006)

–  http://phototour.cs.washington.edu/Photo_Tourism.pdf

(Adapted from slides by Noah Snavely)

Problem: Drift

Solution •  add another copy of first image at the end •  this gives a constraint: yn = y1 •  there are a bunch of ways to solve this problem

–  add displacement of (y1 – yn)/(n -1) to each image after the first –  compute a global warp: y’ = y + ax –  run a big optimization problem, incorporating this constraint

»  best solution, but more complicated »  known as “bundle adjustment”

(x1,y1)

copy of first image

(xn,yn)

I1 I2 I3 I4

Global optimization

t1

(0,0)

t2 t3 t4

Input

Output

p1 p2 p3

We want to estimate ti . We know pi •  how do ti relate to pi?

I1 I2 I3 I4

p1 p2 p3

Global optimization

Recipe 1.  Identify the variables you want to estimate

•  in our case: ti •  Identify a set of objectives you want to satisfy

1. in our case: pi - pj = ti - tj and similar for q, r, s •  Define an objective function F over these variables, whose

minimum occurs at the “answer” for these variables •  Find the minimum of F

q2 q3

q4

r3 r4 s4 s1

5/20/13

2

I1 I2 I3 I4

p1 p2 p3

Objective function

Objective function

+ similar terms for q, r, s

q2 q3

q4

r3 r4 s4 s1 p1

p2 p3

Objective function

Objective function

Matrix form pi=  (ui,  vi)    

ti=  (xi,  yi)    

Objective Function Adding in q, r, s give a larger matrix equation

A x b Defines a least squares problem: minimize

•  Solution: •  Problem: there are multiple solutions for ! (det = 0) •  We can add a global offset to a solution and get the same error

Ambiguity in the solution

•  Each of these solutions has the same error •  Called the gauge  ambiguity  •  Solution: fix the translation of one image (t1 = (0,0))

(0,0)

(-100,-100)

(200,-200)

5/20/13

3

Structure from motion

Computed 3D structure Images on the Internet

Structure from motion

Camera  1  

Camera  2  

Camera  3  

R1,t1 R2,t2

R3,t3

p1

p4

p3

p2

p5

p6

p7

minimize

g (R, T, P)

•  aka “bundle adjustment” (texts: Zisserman; Faugeras)

Given point x  and rotation and translation R, t  

Minimize sum of squared reprojection errors:

predicted image location

observed image location

SfM objective function Solving structure from motion Minimizing g  is difficult:

–  g is non-linear due to rotations, perspective division –  lots of parameters: 3 for each 3D point, 6 for each camera –  difficult to initialize –  gauge ambiguity: error is invariant to a similarity transform

(translation, rotation, uniform scale)

Many techniques use non-linear least-squares optimization (bundle adjustment) –  Levenberg-Marquardt is a popular algorithm –  http://en.wikipedia.org/wiki/Levenberg-Marquardt_algorithm

Good code online –  Bundler: http://phototour.cs.washington.edu/bundler/ –  Multicore: http://grail.cs.washington.edu/projects/mcba/

5/20/13

4

Photo Tourism

Photo tourism video: http://www.youtube.com/watch?v=5Ji84zb2r8s Microsoft Photosynth: http://photosynth.net/ Google Photo Tours: http://maps.google.com/phototours

5/20/13

5

?

5/20/13

6

Reconstructing Rome • In a day...

• From ~1M images • Using ~1000 cores

• Sameer Agarwal, Noah Snavely, Rick Szeliski, Steve Seitz • http://grail.cs.washington.edu/rome

St. Peters (inside)

Trevi Fountain

St. Peters (outside)

Il Vittoriano

Coliseum (inside)

Coliseum (outside)

Forum

Rome 150K: Colosseum Rome: St. Peters

5/20/13

7

Venice (250K images) Venice: Canal

Dubrovnik More info •  Rome-in-a-day page

–  http://grail.cs.washington.edu/rome