General Imaging ModelMichael Grossberg and Shree Nayar

CAVE Lab, Columbia University

ICCV ConferenceVancouver, July 2001

Partially funded by NSF ITR Award, DARPA/ONR MURI

Imaging

• What is a general imaging model ?• How do we Compute its Parameters ?

Scene Imaging System Images

?

Perspective Imaging Model

Camera Obscura

rays selectedrays become image points

Systems that are not perspective

multiple camera system

catadioptric system

fisheye lens

compound eyes

General Imaging Model• Essential components:

– Photosensitive elements– optics

i

Pi

• Maps incoming pixels to rays

Raxel = Ray + Pixel

• Small perspective camera– Simple lens– One pixel photo-detector

Raxel symbol

Index Geometry Radiometry

Position Direction Point Spread Fall-off Response

• Most general model is a list of raxels

Ray Surfaces

(pX, pY, pZ) (q, q)

imaging optics

virtual detectors(raxels)

physical detectors

(pixels)

ray surface

Position: (pX, pY, pZ)Direction: (q, q)

perspective

Rays in 2D• Singularity of rays called a caustic

position-directionspace

positionspace

XY

non-perspective

caustic

Computing Caustics• Change coordinates

– (x,y,d) (X,Y,Z)

ddx

y(X,Y,Z)

• Solve for d

ZZZZZ

YYYYY

XXXXX

qyqd

yp

xqd

xp

qyqd

yp

xqd

xp

qyqd

yp

xqd

xp

J

)det(

Caustic Ray Surface

• Caustic is a singularity or envelope of incoming rays• Caustic represents loci of view-points

raxels

Caustic curve

imaging optics

Simple Examples

perspective single viewpoint multi-viewpoint

Raxel Radiometry

• Non-linear response of photosensitive element

• Linear fall-off of optical elements

Raxel index

Normalized Fall-off

h(x)

Normalized Exposure (e)

Normalized Response

g(e)

Point Spread

• Elliptical gaussian model of point spread.– Major and minor deviation lengths, a (d), b (d)– Angle of axis (when a (d), b (d) are different)

Impulse at Scene point

d, Scene depth

Chief ray

a

b

Image plane

Finding the Parameters

• Known optical components: Compute

• Unknown optical components: Calibration Environment

?

Calibration Apparatus• Structured light at two planes

– Geometry from binary patterns– Radiometry from uniform patterns

z

pfpnqf

i

Finding the parameters: Perspective System

laptop LCDvideo camera with perspective lens

translating stage sample image

Computed Raxel Model: Geometry

180

160

360

140

120

100

80

60180

160140

120100

80340320300280260

X in mm

Y in mm

Z in mm

Computed Raxel Model: Radiometry

• Radiometric response g(e)

normalized exposure

normalizedresponse

• Pointwise fall-off h(x,y)

radius in pixels

normalizedfall-off

1.0

0.8

0.6

0.4

0.2

0.0

1.0

0.8

0.6

0.4

0.2

0.0

1.00.90.80.70.60.50.40.30.20.10.0 1.00.90.80.70.60.50.40.30.20.10.0 0 50 100 150 200 250 300

0.10.0

1.00.90.80.70.60.50.40.30.2

Finding the parameters: Non-single Viewpoint System

laptop LCDvideo camera with perspective lens

translating stageparabolic Mirror sample image

Computed Raxel Model: Geometry

• Rotationally symmetric

10

5

-35

0

-5

-10

-15

-20

-25

-30

-60-40

-200

6040

20

-60-40

-200

6040

20

mm from caustic max

mm from axis of symmetrymm from axis of symmetry

Computed Raxel Model: Radiometry

• Fall-off toward edge as resolution increases:– less light collected

radius in pixels

normalizedfall-off

1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.227025023021019017015013011090 290

Summary

• Most general model simply list of raxels

• Caustics summarize geometry• Simple procedure for obtaining

parameters from a black box system

Index Geometry Radiometry

Position Direction Point Spread Fall-off Response

x, y pX, pY, pZ q, q a, b, h g(e)