http://www.cg.info.hiroshima-cu.ac.jp/

Computer Graphics Laboratory, Hiroshima City University

All images are compressed.

The images in this file are not the original ones.

SVD Photometric stereo (Miyazaki, Ikeuchi)

http://www.cg.info.hiroshima-cu.ac.jp/

Computer Graphics Laboratory, Hiroshima City University

Photometric stereo under unknown light sources using robust SVD with

missing data

Daisuke Miyazaki The University of Tokyo PreviouslyHiroshima City University Currently

Katsushi Ikeuchi The University of Tokyo

SVD Photometric stereo (Miyazaki, Ikeuchi)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Geometric modeling

SVD Photometric stereo (Miyazaki, Ikeuchi)

[Laser range sensor] [Uncalibrated photometric stereo]

Algorithm(10) Experiment(5) Conclusion(1)Introduction(1/2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Proposed method

• Photometric stereo• SVD: Our hole-filling SVD

– Related work: [Mukaigawa 07] [Tomasi 92] [Shum 95] [Brand 02]

• Robust to outliers: shadow, specular reflection• Point light source in far distance with small size:

directional light• Typically around 30 to 100 number of images• The direction (and the power) of light sources are

unknown– Related work: [Hayakawa 94] [Woodham 91] [Yuille 99]

[Belhumeur 99]

SVD Photometric stereo (Miyazaki, Ikeuchi)

Algorithm(10) Experiment(5) Conclusion(1)Introduction(2/2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Photometric stereo

SVD Photometric stereo (Miyazaki, Ikeuchi)

Lambertian model for diffuse reflection

ln ,0maxdkI

cos,0maxdkI

Algorithm(1/10) Experiment(5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Linearization

SVD Photometric stereo (Miyazaki, Ikeuchi)

Specular reflection

Shadow

pos

neg

Linearization ssd kkI BRDF,0max ln

ln dkI

Linear algebra

Cosine function

Algorithm(2/10) Experiment(5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Three orthonormal basis images using SVD

SVD Photometric stereo (Miyazaki, Ikeuchi)

PCA/SVD

3 bases

[Shashua 97]

-192 4 8

-192 -128 -4

-192 -4 128

Linearized image

Representableby 3 bases

pos: greenneg: red

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Outlier

SVD Photometric stereo (Miyazaki, Ikeuchi)

Shadow

Specular reflection

Outlier

"hole"

Algorithm(4/10) Experiment(5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Hole filling

SVD Photometric stereo (Miyazaki, Ikeuchi)

Without hole

Bases

With hole

???

cba

cba

Algorithm(5/10) Experiment(5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Gradually filling holes

SVD Photometric stereo (Miyazaki, Ikeuchi)

SVD

Bases

Hole filling

SVD

Bases

Hole filling

Algorithm(6/10) Experiment(5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Matrix representation

SVD Photometric stereo (Miyazaki, Ikeuchi)

Pixelnumber

Imagenumber

Matrix-vector representation

Image matrix

SVD

...

Image 1 Image 2

Algorithm(7/10) Experiment(5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Row/column scraping

SVD Photometric stereo (Miyazaki, Ikeuchi)

Input imagewith outliers

Algorithm(8/10) Experiment(5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Row/column sticking

SVD Photometric stereo (Miyazaki, Ikeuchi)

Holefilling

Linearizedinput imageHole

filling

SVD

SVD

Algorithm(9/10) Experiment(5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Algorithm flow

SVD Photometric stereo (Miyazaki, Ikeuchi)

Thresholding & graph cut

Hole filling SVD

Region segmentation

Occluding boundary

Outlier detection

Linearization

Light source direction estimation

[Hayakawa 94] [Shi 10] [Belhumeur 99]

[Sato 07]

[Our method]

Light source direction estimation

1

2

3

4

Algorithm(10/10) Experiment(5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Linearizaiton result

SVD Photometric stereo (Miyazaki, Ikeuchi)

Input image

Linearized image

Algorithm(10) Experiment(1/5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Result

SVD Photometric stereo (Miyazaki, Ikeuchi)

Algorithm(10) Experiment(2/5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Result

SVD Photometric stereo (Miyazaki, Ikeuchi)

Algorithm(10) Experiment(3/5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Comparison

SVD Photometric stereo (Miyazaki, Ikeuchi)

With outlier detection[Our result]

Without outlier detection

Algorithm(10) Experiment(4/5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Comparison

SVD Photometric stereo (Miyazaki, Ikeuchi)

With outlier detection[Our result]

Without outlier detection

Algorithm(10) Experiment(5/5) Conclusion(1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

CG Lab, Hiroshima City University

Conclusion

SVD Photometric stereo (Miyazaki, Ikeuchi)

SVD Photometric Stereo

Con: Computation time (1h~1d)

Light source unknown Shadow Specular reflection

Hole filling SVD

Algorithm(10) Experiment(5) Conclusion(1/1)Introduction(2)

http://www.cg.info.hiroshima-cu.ac.jp/

Computer Graphics Laboratory, Hiroshima City University

(c) Crypton Future Media, Inc.

Crypton Future Media, Inc. owns the copyright of this character, Miku Hatsune. This picture is

drawn by Daisuke Miyazaki under Piapro Character License.

http://piapro.jp/license/pcl/summary

SVD Photometric stereo (Miyazaki, Ikeuchi)

http://www.cg.info.hiroshima-cu.ac.jp/

Computer Graphics Laboratory, Hiroshima City University

(c) Daisuke Miyazaki 2010All rights reserved.

http://www.cg.info.hiroshima-cu.ac.jp/~miyazaki/Daisuke Miyazaki, Katsushi Ikeuchi, "Photometric stereo

under unknown light sources using robust SVD with missing data," Proceedings of International Conference on Image

Processing, pp. 4057-4060, Hong Kong, China, Sep. 2010

SVD Photometric stereo (Miyazaki, Ikeuchi)