www.cs.technion.ac.il/~ron

Numerical Geometry of Images:

Shape ReconstructionRon Kimmel

Geometric Image Processing Lab

Computer Science Department Technion-Israel Institute of Technology

Shape reconstruction techniques

Shape from: Inputs

Shading

Stereo

Auto-stereograms

Photometric stereo

Structured/coded light

IR-pulses (3DV-Systems)

Stereo

Shape from Stereo

Shape from autostereograms

for i = (stripesize+1):xsize+stripesize, for j = 1:ysize, stereo(j,i) = stereo(j,i-floor(stripesize*(1-z(j,i)))); end %for jend %for i

Image Formation

F. Guichard 93Mondrian world:Lambertian surface patches

lyxNyxyxI

),,(),(),(

Image formationLambetian

model

V

lN

)cos(

,),(

lNyxI

Lambertian surfaces

Images formation

.1

1),( ˆfor

,),,(),(

2z

yxIzl

lyxNyxI

Shape from photometric stereo

Given Lambertian shading model, multiple images each with different light source.

At each point we have 3 unknowns that can be computed from at least 3 images.

Next, the problem is surface integration from its gradient vector field

.1

}1,,{

,),,(),(),(

2z

zzN

lyxNyxyxI

yx

ii

,,, yx zz

},{ yx zzz

Shape from photometric stereo

Next, the problem is surface reconstruction from

Define the integral measure

For which the Euler-Lagrange is a Poisson equation

or in more compact notations

dxdyqzpzzE yx 22 )()()(

},{ qpz

yxyyxx qpzz

),( yxfz

Shape from photometric stereo

Efficient numerical algorithm for solving a Poisson equation depends on the boundary conditions.

For periodic b.c. Fourier transform is an option

),( yxfz

22

1

2222

),(ˆ),(

)()()(

)()()()),((

)),((),(ˆ

)),((),(ˆ

vu

vufyxz

zvuzvzu

zzzzyxz

yxfvuf

yxzvuz

yyxxyyxx

F

FFF

FFFF

F

F

Shape from photometric stereo

Efficient numerical algorithm for solving a Poisson equation depends on the boundary conditions.

For point constraints, see Kimmel-Yavneh Algebraic Multi-Grid approach.

),( yxfz

Multigrid Poisson solvers

Solve a Poisson equation

),( yxfz

2 2 2

solve ( , )

search for in a coarse scale 2

ˆsolving for ,

we have that our solution is given by

ˆ.

h h h

h h h h

h

h h h

z f x y

r z f

r h

z r

z r

z z z

Shape from shading

Horn: characteristic strip expansion Bruckstein: equal height contours Rouy-Turin: Minimal cost approach Kimmel-Sethian: Fast Marching

Summary: Shape reconstruction

Shape from shading Explicit methods (Horn/Bruckstein)

Open Questions

A working automatic shape from stereo algorithm does not exist (but we are getting there).

www.cs.technion.ac.il/~ron