Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection...
-
Upload
edgar-moody -
Category
Documents
-
view
235 -
download
1
Transcript of Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection...
![Page 1: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/1.jpg)
Shape from Shading #1
![Page 2: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/2.jpg)
Topics
irradiance and radiance basic concepts of reflection reflection map photometric stereo
Reflectance map and Photometric stereo
![Page 3: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/3.jpg)
Brightness
irradiance amount of light falling on a surfacefalling energy measured by a unit surface area [watt/m2]
amount of light radiated from a surfaceemitting energy measured from a unit forshorted light source surface area to a unit solid angle [watt/ m2 ・ Sr]
solid angle --- steradian
radiance
2
cos
R
A R Θ
A
![Page 4: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/4.jpg)
Reflection geometry
•irradiance at a pixel depends on•illumination•materials•geometry
•under the same illuminate condition, we observe irradiance difference on the same material surfacethere is a relationship between pixel irradiance and geometry•Reflectance geometry
L=illuminationN=normalV=viewer
VLg
VNe
LNi
cos
cos
cos
L
i=incidence anglee=emitting angleg=phase angle
NVi e
g
![Page 5: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/5.jpg)
Gradient space
reflection functions are defined in the local coordinate system(e,i,g)
For our development, we will redefine the reflectance geometry in the gradient space viewers is always on the Z axis
1
)1,,(
,
22
qp
qpn
Y
Zq
X
Zp
q
p
![Page 6: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/6.jpg)
Surface and body reflection Surface reflection and body reflection
surface reflection=gloss,highlights very directional(specular)body reflection =object color all direction(diffuse) plastic, paint have both metal has only surface reflection
body
air
incident light
surfacereflection
bodyreflection
internalpigment
![Page 7: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/7.jpg)
Model for body reflection
Diffuse---scatters in all directionscommon approximation:
equal in all directions“lambertian”Lambertian’s cosine law“perfectly diffuse reflector”
reflectance=constant * geometric factorf(i,e,g) = Kb * cos i
why cos i ? angle of incidence affects “density” of illumination.(irradiance)
irradiance=light/arealight=1area=1/cos iirradiance = cos i
![Page 8: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/8.jpg)
![Page 9: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/9.jpg)
Calculating a reflection map (Lambertian)
for each(p,q), N=(p,q,1) light source direction, S= )1,,( ss qp
11
1
cos),,,(
2222
ss
ss
ss
qpqp
qqpp
LNiqpqpR
)0,0(),( ss qp
1
1),(
22
qpqpR
iso-brightness contour
p
q
0.5
0.80.9
![Page 10: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/10.jpg)
![Page 11: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/11.jpg)
Reflectance map(continue)
Lambertian
Self-shadow line
p
q
![Page 12: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/12.jpg)
Surface reflection
metals have the only surface reflectiondielectrics(plastics,paint)have the surface reflection as well as the body reflectionsimplest approximation: perfect mirror
reflection is specular direction, S’S’ is coplanar with S,NSN = i = NS’: opposite sides
SN
S’
i i
![Page 13: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/13.jpg)
Phong’s model
calculate angle between S’ and V ---α
f-surface(i,e,g)=Ks * cos α
typical : n = 10 to 500heuristic model
tells amount of light at each angle
n=1
Cos α
3
5
Real surfaces are rough : light scatters
n n
![Page 14: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/14.jpg)
![Page 15: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/15.jpg)
Reflectance map
brightdark
q
q
p
p
![Page 16: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/16.jpg)
Better model of surface reflection
Phong’s model R=Ks cosnα not based on physics
just looks OK for graphics, not really accurate
off-specular effect
Torrance and Sparrow --- geometrical opticsBeckmann and Spizzichino --- physical optics
composite surface reflectionPhong’s model real surface
Phong’s model real surface
![Page 17: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/17.jpg)
Torrance and Sparrow model
Geometrical optics– a collection of planar mirror-like facets– surface reflection caused only by these microfacets– their sizes are much larger than wave length
average normal direction
microfacet
facet slopes to be normally distributed
V-shaped valleys
facet normal
0
)/exp()( 22
cp
α
![Page 18: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/18.jpg)
surface reflectance = constant for material* effect of one ray
* % not blocked by others (geometrical attenuation)
* % of all facets involved
Surface reflectance
![Page 19: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/19.jpg)
Effect of one ray
incoming energy = A cos i
outgoing energy =(A cos i) / (A cos e) =cos i / cos e
i
e
![Page 20: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/20.jpg)
Geometric attenuation
1) masking 2) shadowing
g(i,e)
![Page 21: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/21.jpg)
% of all facets involved
idd cos/4'
eeigK s cos/),()/exp( 22
'd α
'dd
i
reflection distribution
facet normal distributionα
N
)/exp( 22
![Page 22: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/22.jpg)
Beckmann and Spizzichino modelphysical optics
surface is continuous h(x,y)light is wave reflection off of surfaceroughness is amplitude and spatial frequency of variations in
h(x,y)E(x,y,z) “field” of light energysurface is assumed to be a perfect conductor(metal)--- > Maxwell’s equation exact solution is vicious integral
,!
cos'
1
4/22
222
222
22
22
m
mTm
gi xyemm
g
A
DTe
R
AEEE
where
22
y
2
)cos(coscos
cossinsincoscos1
)X)sinc(sinc(
)cos(cos2
yxxy
rii
iriri
x
rih
D
Y
g
![Page 23: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/23.jpg)
Our model (Nayer,Ikeuchi,Kanade89)
Torrance and Sparrow + Beckmann and Spizzichino•diffuse lobe --- cosine function•specular spike --- delta function•specular lobe --- gaussian function
![Page 24: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/24.jpg)
![Page 25: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/25.jpg)
recall
VNe
VSa
em
aK
qpqp
qqppK
qpqpR
qqp
qpp
qp
qpS
V
qpN
qpL
s
ss
ssb
ss
ss
sss
ss
ss
ss
cos
cos
cos/exp11
1
),,,(
1,11
,11
then
)1,0,0(
)1,,(
)1,,(
2
2222
22
22
22
22
Calculating reflectance mapspecular lobe + diffuse lobe
Lambertian (diffuse lobe) contours Specular peak
p
q
![Page 26: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/26.jpg)
Shape-from-shading
recover object shape (orientation) from image irradiance (brightness)
brightness surface orientation
E(x,y)=R(p,q) -- image irradiance equation
gives one constraint on the gradient space at each pixel--- > ill-posed problem (cannot solve !!!!!)
(p,q,1)0.8
p
q0.8
![Page 27: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/27.jpg)
Photomotric stereo•one image irradiant equation gives only one constraint
--- > use multiple equations at each pixel.•take multiple images from the same points under different light source directions
•recall different light source directions give different reflectance map•at each pixel, multiple irradiance values
p
q
p
q
![Page 28: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/28.jpg)
Photometric Stereo
![Page 29: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/29.jpg)
Analytical solution
•real world gives complicated light source direction--- > look-up table method
),,(
),,(
1
1,
1,
1),,(
1
1,
1,
1),,(
3333
2222
21
21
21
21
1
21
21
11111
222222
zyx
zyx
ssss
s
ss
szyx
zyx
SSSS
SSSS
qpqp
q
qp
pSSSS
qpqp
q
qp
pnnnn
nAE
nSE
nSE
nSE
33
22
11
zyx
zyx
zyx
SSS
SSS
SSS
A
333
222
111
EAn
nAAEA
1
11
![Page 30: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/30.jpg)
Look-up table method
![Page 31: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/31.jpg)
![Page 32: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/32.jpg)
Summary
Basic concepts of reflectionradiance and irradiancereflection geometrysurface reflection and body reflection
Shape-from-shading problemreflectance mapimage irradiance equationphotometric stereo
![Page 33: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/33.jpg)
Shape-from-shading #2
![Page 34: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/34.jpg)
Shape-from-shading #2
get a depth map from a needle map
get a needle map from a single image
![Page 35: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/35.jpg)
Depth from surface orientation
1 dimensional caserecall
2 dimensional case
z1
)(1)( pdxdzz
dxdzP
x
0z
1zdyq
dxp
dy
dxqdypdxzz 01
![Page 36: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/36.jpg)
Recovering depth map from a needle map(direct integration method)
Photometric stereo gives a needle map
assume a depth at the originget depth along the x of the needle map
get the depth map
pdxzz '
qdypdxzz '
![Page 37: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/37.jpg)
Direct integration
rapid
accumulates errors
![Page 38: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/38.jpg)
Relaxation method
observed orientation (p,q) should be same as those of the depth map (zx,zy)
reduce the total error within a boundary (the calculus of variations See Horn pp.469-474)
an iterative formula
22 )),(),(()),(),(( yxy
zyxqyx
x
zyxp
dxdyyxy
zyxqyx
x
zyxp
A
22 )),(),(()),(),((
),(),(4
1
)1,()1,(),1(),1(4
1),(1
yxy
qyx
x
p
yxzyxzyxzyxzyxz nnnnn
![Page 39: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/39.jpg)
),(),(4
1
...),1(4
1),(1
yxy
qyx
x
p
yxzyxz nn
iterative method
needle map
brightness
depth map
Relaxation method (Example)
![Page 40: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/40.jpg)
Jacobi’s method
yxnnnnn qpyxHyxHyxHyxHyxH
4
1)1,()1,(),1(),1(
4
1),(1
Iteratively computing the equation itself convergence is very slow
The number of iteration needed to converge is
when the error decreased to 10-p times, for the pixels whose size is J×J
2
2
1pJr
Eg. r=1200000 when J=400, p=15
![Page 41: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/41.jpg)
Gauss-Seidel method
yxnnnnn qpyxHyxHyxHyxHyxH
4
1)1,()1,(),1(),1(
4
1),( 111
2
4
1pJr Eg. r=600000 when J=400, p=15
yxnnnnn qpyxHyxHyxHyxHyxH
4
1)1,()1,(),1(),1(
4
1),(1
Jacobi:
Gauss-Seidel:
scan-line order checkerboard pattern
Iteration number:
![Page 42: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/42.jpg)
SOR (successive overrelaxation) method ),(1
4
1)1,()1,(),1(),1(
4
1),( 111 yxHqpyxHyxHyxHyxHyxH n
yxnnnnn
ω: overrelaxation parameter
Converge if 0<ω<2
0<ω<1 (underrelaxation): slower than Gauss-Seidel1<ω<2 (overrelaxation): faster than Gauss-Seidel
Optimal ω:J
1
2Eg. ω=1.984 when J=400
pJr3
1 Eg. r=2000 when J=400, p=15Iteration number:
Chebyshev acceleration (faster convergence): change ω properly for each iteration
same as Gauss-Seidel
![Page 43: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/43.jpg)
ADI (alternating-direction implicit) method
yxnnnnnn qpyxHyxHyxHyxHyxHyxH )1,(),(2)1,(),1(),(2),1( 212121
yxnnnnnn qpyxHyxHyxHyxHyxHyxH ),1(),(2),1()1,(),(2)1,( 212121111
Jpr 10log3
8 Eg. r=104 when J=400, p=15Iteration number:
: changed properly for each iterationif =0 Gauss-Seidel
-1 +2 -1
……
H(1,y)
H(J,y)
……
*
…
…
*-1 +2 -1
-1 +2 -1
……
H(x,1)
H(x,J)
……
*
…
…
*-1 +2 -1
A: tridiagonal matrix (given)b: vector (given)x: solve Ax=b by using linear system solver (forward substitution) O(N)
![Page 44: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/44.jpg)
Natural boundary condition
ds
dxds
dy
q
p
ds
dxds
dy
H
H
y
x
T
ds
dx
ds
dy
s… arc length of boundary
Natural boundary condition of "gradient-to-height problem" is
·… dot product … the normal of boundary
Algorithm:
[Truth] [Wrong result][Correct result]
Same height boundaryNatural boundary condition
Fast convergenceSlow convergence
Boundary:Height known just use it
Height unknown natural boundary condition
yxyxyx HHp ,1,1, 2
1 1,1,, 2
1 yxyxyx HHq
Calculate the height "H" at the boundary by
or for each iteration
![Page 45: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/45.jpg)
Shape-from-shading with a single view
Photometric stereo uses multiple images.
Is there a way to recover shape from a single image?
Yes, there is a way.1. characteristics strip expansion method:obtain surface orientati
on along characteristics strips of image irradiance equation (Horn 75)
2. relaxation method:obtain surface orientation using image irradiance equation and smoothness constraint (Ikeuchi and Horn 81)
3. global method: assume a surface is a part of sphere (Pentland 83)
![Page 46: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/46.jpg)
Characteristic strip expansion method
the steep descent direction of the reflectance map(gradient space)
the steep descent direction of the image brightness(image brightness)
p
q
x
y
![Page 47: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/47.jpg)
x
y
SDD of RM
SDD of IB
SDD of RM
SDD of IB
p
q
move towards the SDD of the reflectance map on the image plane
move towards the SDD of the image brightness on the gradient space
Characteristic Strip
![Page 48: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/48.jpg)
Proof1. Taylor expansion of p(x,y) and q(x,y)
2. derivative of the image irradiance equation
dyy
qdx
x
qyxqdyydxdq
dyy
pdx
x
pyxpdyydxxp
),(),(
),(),(
dyy
qdx
x
qdq
dyy
pdx
x
pdp
),( yxp
),( ydxxp
x dxx
)),(),,((),( yxqyxpRyxE
x
q
q
R
x
p
p
R
x
E
y
q
q
R
y
p
p
R
y
E
![Page 49: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/49.jpg)
Move towards the SDD of the reflectance map on the image plane
then, what happen to (p(x,y),q(x,y)) ?
move towards the SDD of the image brightness on the gradient
space
Sp
Rx
Sq
Ry
p
R
q
R
S
),(q
R
p
R
on the image
Sx
E
Sx
q
q
R
x
p
p
RS
y
p
q
R
x
p
p
R
Sq
R
y
pS
p
R
x
py
y
px
x
pp
y
E
x
E, ),( qpon
![Page 50: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/50.jpg)
),( qp RR
),( qp EE p
q
0.10.20.30.40.5
xEdp yEdq
x
y
0.1
0.2
0.30.4
from a known point, (you know (p,q) and E)you can determine (p,q) along a characteristic strip
![Page 51: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/51.jpg)
x
y
Sp
Rx
Sq
Ry
character strip
reconstructed contour
q
p
Horn 75
![Page 52: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/52.jpg)
Problem of characteristic stip method
1. Error accumulation.
2. The method starts from a singular point the start point is unreliable.
3. Determine surface orientation only along characteristic stripes.
4. Occluding contours are big evidences. We cannot use that information. (p,q) become infinite.
relaxation method with occluding contours
???
![Page 53: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/53.jpg)
Occluding boundary
Surface orientations on occluding boundaries are known from the shape of silhouette.
These surface orientations cannot be represented by the gradient space. (p,q) becomes infinite.
We will use the stereographic plane, (f,g). On (f,g) plane, occluding boundaries lie on the unit circle.
boundary condition
![Page 54: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/54.jpg)
Stereographic Projection
2222 11
2,
11
2
qp
qg
qp
pf
occluding boundaries lie on the unit circle
gradientspace
p
q
gaussian sphere
![Page 55: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/55.jpg)
Relaxation method
1. Image irradiance equation on (f,g) space on the (f,g) space, we can also define a reflectance map.
2. Smoothness constraint. Neighboring points have roughly the same surface orientation.
)),(),,((),( yxgyxfRyxE
02222
y
g
x
g
y
f
x
f
![Page 56: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/56.jpg)
Relaxation method3. Set up a minimization problem.
4. Using the calculus of variations get iterative formula.
dxdyy
g
x
g
y
f
x
f
yxgyxfRyxEE
2222
2))),(),,((),((
)),(),,((),(
)1,()1,(),1(),1(41),(1
yxgyxfRyxE
yxfyxfyxfyxfyxf
nn
nnnnn
→min
)),(),,((),(
)1,()1,(),1(),1(41),(1
yxgyxfRyxE
yxgyxgyxgyxgyxg
nn
nnnnn
![Page 57: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/57.jpg)
...)...,1(41),(1 yxfyxf nn
...)...,1(41),(1 yxgyxg nn
n+hsolution
n+1+hsolution
brightness image
occluding boundary
needle map depth map
Ikeuchi & Horn 81
![Page 58: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/58.jpg)
Summary
1. Gaussian sphere and reflectance map2. Get a depth map from a needle map
1. direct integration2. relaxation method
3. Get a needle map from a single image1. characteristic strip expansion method2. relaxation method regularization
4. Read Horn pp. 244-269, B&B pp. 93-101
![Page 59: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/59.jpg)
Shape-from-shading #3
![Page 60: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/60.jpg)
Shape-from-shading #3– Shape from shading
More advanced researches– 4-light photometric stereo– Extended light source– Photometric sampling– Shape from interreflection– Inverse polarization raytracingspecular spike interreflection
+ transmission
diffuse interreflection
diffuse
diffuse + specular
specular spike
diffuse + specular spike
![Page 61: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/61.jpg)
4-light photometric stereo
[diffuse pixel]
4 correctanswer
smalldeviation
[specular pixel]
1 correctanswer
largedeviation
![Page 62: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/62.jpg)
4-light photometric stereo [Coleman&Jain]
[diffuse pixel]
Averageof four
[specular pixel]
Use 3dark pixels
Deviation
Smaller than threshold Larger than threshold
![Page 63: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/63.jpg)
4-light photometric stereo [Barsky&Petrou]
specular
diffuse
length
threshold[specular]dark 3
[non-specular]bright 3
[specular pixel] [diffuse pixel] [shadow pixel]
light
specular normal surface normal
difference
![Page 64: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/64.jpg)
4-light photometric stereo [Solomon&Ikeuchi]
4 3
3
3
3
2 2
22
Gaussian sphere
[4 light region]
[Coleman&Jain]
[3 light region]
Use two lights
p
q
Gradient space
Two possible normals
Choose negative shading of shadow light
shadowspecular
[2 light region]Use two lights Two possible normals
diffuse
diffuse
p
q
Gradient spaceChoose negative shading of shadow light
shadowshadow
diffuse
diffuse
![Page 65: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/65.jpg)
Planar extended light source
L
l0
X
Brightness: f
Line lamp
Specular object
Lambertian plane(Extended light source)Camera
r
Irradiance: E(f,r,X,l0,L)
![Page 66: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/66.jpg)
Reflectance map3 linear lamps 3 reflectance maps Solve by photometric stereo
![Page 67: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/67.jpg)
Photometric sampler
![Page 68: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/68.jpg)
Spherical extended light sourceL()=L(;s,R,H,I,C)
C: constantI: radiant intensity of light S
: termination angle
![Page 69: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/69.jpg)
Photometric function for Lambertian surface
)cos( nsAI Lambertian
Lambertian of extended light)cos( nkk AI
Estimate reflectance A'and surface normal n
by fitting cosine function to images
![Page 70: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/70.jpg)
Photometric function for specular surface
)2( nsBI Specular spike
)2( nkk LBI Specular spike of extended light
Point sources are separatedby source termination angle
Only two specular peaks appear
![Page 71: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/71.jpg)
Photometric function for specular surface)2( nkk LBI
Point sources are separatedby source termination angle
)()(
)()()(
1
1
kk
kkk LL
LLD
1-to-1 correspondence
n is calculated
)2( nk
k
L
IB
is calculated
![Page 72: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/72.jpg)
Photometric function for hybrid surface)2()cos( nsns BAI
)2()cos( nknkk LBAI
1. Remove 2 intensities2. Calculate nl by fitting
Lambertian model fromremaining intensities
3. Calculate ns from 2 intensities
Calculate (nl,ns)for all possibilities
Choose where
Finally,
nsnl
BA
BA nsnln
![Page 73: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/73.jpg)
Example
![Page 74: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/74.jpg)
Interreflection
![Page 75: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/75.jpg)
Form factor
dxdx'r
'
Interreflection kernel: 2
coscos),(),(
r
xxVxxK
Visibility function: V=1 if visible, V=0 if invisible
xdxLxxKx
xxdLm )(),()(
),(
Radiance of x Irradiance of x
Albedo Radiance of x'
![Page 76: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/76.jpg)
Interreflection equationPKLLL s
Facet radiance vector: TmLLL ,,, 21 L Source contribution vector: Tsmsss LLL ,,, 21 L
Albedo matrix:
m
00
00
00
1 2
1
P Kernel matrix:
0
0
0
21
221
112
mm
m
m
KK
KK
KK
K
![Page 77: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/77.jpg)
Inverse radiosity
PKLLL s sLPKIL 1
sFL s (Shading) Facet matrix: TmNNNF ,,, 21 i
ii nN
Albedo
Surface normal
Source direction vector: Tzyx sss ,,s
sFPKIL 1
sFL p
FPKIF 1pPseudo facet matrix:
(Pseudo shading) Estimate Fp by conventional photometric stereo(L: input image, s: known light source)
Estimate F iteratively from initial value Fp
pFPKIF
kP kKand is straighforwardly calculated from kF
pkkk FKPIF 1Recovery algorithm:
![Page 78: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/78.jpg)
Example
![Page 79: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/79.jpg)
Application to book scanning
Image scanner
Scanning a book
Observed image
Estimated shape
Restored image
![Page 80: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/80.jpg)
Polarization
Light = wave oscillates Oscillates in certain direction polarization DOP = degree of polarization
Unpolarized(DOP 0)
Light
Perfectly polarized(DOP 1)
Polarizer
Partially polarized(DOP 0~1)
Incident Reflected
AirObject
Transmitted
![Page 81: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/81.jpg)
Reflection and transmissionNormal
Unpolarized
Air
Object
Partially polarized
Partially polarized
Light
Dependsupon
![Page 82: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/82.jpg)
Polarization raytracing
Calculate polarization(Mueller calculus)
Calculate reflection & transmission(Ray-tracing)
Light(4D vector)
Reflection&transmission(4x4 matrix)
![Page 83: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/83.jpg)
Minimization
Solve by Shape-From-Shading technique
dxdyqHpHII yxRE222min
RE II Polarization raytracing equation
Input Calculated by polarization raytracing
![Page 84: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/84.jpg)
Example
Result(10 iteration)
Glass(refractive index 1.5)
![Page 85: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/85.jpg)
Summary
4-light photometric stereo– Photometric stereo from proper 3 lights
Extended light source– Photometric stereo from Lambertian planar light source
Photometric sampling– Shape and reflectance from spherical extended light source
Shape from interreflection– Photometric stereo for concave diffuse object
Inverse polarization raytracing– Shape of transparent object by using SFS technique
![Page 86: Shape from Shading #1. Topics u irradiance and radiance u basic concepts of reflection u reflection map u photometric stereo Reflectance map and Photometric.](https://reader035.fdocuments.in/reader035/viewer/2022062308/56649e035503460f94aee256/html5/thumbnails/86.jpg)
Reference Horn, B.K.P. "Robot Vision," MIT Press, 1986 Zhang, R., Tsai, P.S., Cryer, J.D., Shah, M., "Shape from Shading: A
Survey," IEEE Trans. PAMI, Vol. 21, No. 8, pp. 690-706, 1999. Horn, B.K.P “Obtaining shape from shading information,” in The Psy
chology of Computer Vision, P.H. Winston (ed.), McGraw-Hill, 1995 Ikeuchi, K. & B.K.P. Horn, “Numerical shape from shading and occlu
ding boundaries,” Artificial Intelligence, Vol. 17, 1981. Pentland, A.P., “Local shading analysis,” IEEE Trans. PAMI, Vol.6, 1
984. Klinker, G.J., S.A. Shafer & T. Kanade, “The measurement of highlig
ht in color image,” Int. J. Computer Vision, Vol.2, 1988. Nayar S.K., K. Ikeuchi, and T. Kanade "Surface reflection: physical an
d geometrical perspectives", IEEE Trans. PAMI, Vol. 13, pp.661-634, 1991.