Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport
-
Upload
matthew-otoole -
Category
Education
-
view
1.773 -
download
0
description
Transcript of Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport
![Page 1: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/1.jpg)
Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport
http://www.dgp.toronto.edu/~motoole/temporalprobing.html
Matt O’Toole1 Felix Heide2 Lei Xiao2 Matthias Hullin3 Wolfgang Heidrich2,4 Kyros Kutulakos1
1University of Toronto 2University of British Columbia 3University of Bonn 4KAUST
![Page 2: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/2.jpg)
the time-of-flight (ToF) sensing revolution
estimating poses[Schwarz et al. 11]
Kinect for Xbox Onefemto-photography[Velten et al. 2013]
looking-around-corners[Velten et al. 2012]
gesture recognition[Droeschel et al. 11]
Google’s self-driving car
![Page 3: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/3.jpg)
temporal probing for scene analysis
capture depthrobust to indirect light
reconstruct “light-in-flight” videowith high temporal resolution
key insight: complex-valued image formation model
![Page 4: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/4.jpg)
contributions
the transient-frequency transport matrix
unified theory for light transport analysis analyze ToF images using classical techniques
ToF imaging using projectors & maskscapture depth robust to indirect light
reconstruct “light-in-flight” video with high temporal resolution
![Page 5: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/5.jpg)
what is a ToF photo?
ToF camera
![Page 6: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/6.jpg)
what is a ToF photo?
point lightToF camera
![Page 7: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/7.jpg)
what is a ToF photo?
ToF camera ToF projector
![Page 8: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/8.jpg)
what is a ToF photo?
ToF camera ToF projector
![Page 9: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/9.jpg)
what is a ToF photo?
ToF camera ToF projector
![Page 10: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/10.jpg)
what is a ToF photo?
ToF camera ToF projector
![Page 11: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/11.jpg)
what is a ToF photo?
ToF camera ToF projector
![Page 12: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/12.jpg)
what is a ToF photo?
ToF camera ToF projector
![Page 13: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/13.jpg)
what is a ToF photo?
ToF projectorToF camera
![Page 14: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/14.jpg)
what is a ToF photo?
ToF projectorToF camera
![Page 15: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/15.jpg)
what is a ToF photo?
ToF projectorToF camera
intensity
![Page 16: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/16.jpg)
what is a ToF photo?
phase delay
ToF projectorToF camera
![Page 17: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/17.jpg)
what is a ToF photo?
transport coefficient
ToF camera ToF projector
![Page 18: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/18.jpg)
what is a ToF photo?
transport coefficient
ToF camera ToF projector
![Page 19: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/19.jpg)
what is a ToF photo?
transport coefficient
ToF camera ToF projector
![Page 20: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/20.jpg)
mirror
what is a ToF photo?
transport coefficient
ToF camera ToF projector
![Page 21: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/21.jpg)
mirror
camera projector
it is a vector of complex-valued pixels…
camera
photo
![Page 22: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/22.jpg)
camera
it is a vector of complex-valued pixels…
camera
photo pattern
![Page 23: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/23.jpg)
camera
… produced by a complex-valued matrix
camera
photo patterntransport matrix for
![Page 24: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/24.jpg)
conventional image under ambient lighting
![Page 25: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/25.jpg)
brightness
hue
visualizing a complex ToF image
ph
ase
![Page 26: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/26.jpg)
camera
transient-frequency light transport equation
camera
photo patterntransport matrix for
![Page 27: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/27.jpg)
transient-frequency light transport equation
conventional transport matrix [Ng et al. 03; …]
photo transport matrix for pattern
![Page 28: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/28.jpg)
Techniques Reference(s)
transport equation [Ng et al. 03]
dual photography [Sen et al. 05; Sen and Darabi 09]
radiometric compensation [Wetzstein and Bimber 07]
radiosity equation [Goral et al. 84]
radiosity solution [Goral et al. 84]
inverse light transport [Seitz et al. 05; Bai et al. 10]
transport eigenvectors [O’Toole and Kutulakos 10]
structured light transport [O’Toole et al. CVPR 2014]
fast direct/global separation
[Nayar et al. SIGGRAPH 2006]
![Page 29: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/29.jpg)
Techniques Reference(s) Conventional Analysis
transport equation [Ng et al. 03]
dual photography [Sen et al. 05; Sen and Darabi 09]
radiometric compensation [Wetzstein and Bimber 07]
radiosity equation [Goral et al. 84]
radiosity solution [Goral et al. 84]
inverse light transport [Seitz et al. 05; Bai et al. 10]
transport eigenvectors [O’Toole and Kutulakos 10]
structured light transport [O’Toole et al. CVPR 2014]
fast direct/global separation
[Nayar et al. SIGGRAPH 2006]
![Page 30: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/30.jpg)
Techniques Reference(s) Time-of-Flight Analysis
transport equation [Ng et al. 03]
dual photography [Sen et al. 05; Sen and Darabi 09]
radiometric compensation [Wetzstein and Bimber 07]
radiosity equation [Goral et al. 84]
radiosity solution [Goral et al. 84]
inverse light transport [Seitz et al. 05; Bai et al. 10]
transport eigenvectors [O’Toole and Kutulakos 10]
structured light transport [O’Toole et al. CVPR 2014]
fast direct/global separation
[Nayar et al. SIGGRAPH 2006]
![Page 31: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/31.jpg)
Techniques Reference(s) Time-of-Flight Analysis
transport equation [Ng et al. 03]
dual photography [Sen et al. 05; Sen and Darabi 09]
radiometric compensation [Wetzstein and Bimber 07]
radiosity equation [Goral et al. 84]
radiosity solution [Goral et al. 84]
inverse light transport [Seitz et al. 05; Bai et al. 10]
transport eigenvectors [O’Toole and Kutulakos 10]
structured light transport [O’Toole et al. CVPR 2014]
fast direct/global separation
[Nayar et al. SIGGRAPH 2006]
![Page 32: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/32.jpg)
Techniques Reference(s) Time-of-Flight Analysis
transport equation [Ng et al. 03]
dual photography [Sen et al. 05; Sen and Darabi 09]
radiometric compensation [Wetzstein and Bimber 07]
radiosity equation [Goral et al. 84]
radiosity solution [Goral et al. 84]
inverse light transport [Seitz et al. 05; Bai et al. 10]
transport eigenvectors [O’Toole and Kutulakos 10]
structured light transport [O’Toole et al. CVPR 2014]
fast direct/global separation
[Nayar et al. SIGGRAPH 2006]
![Page 33: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/33.jpg)
Techniques Reference(s) Time-of-Flight Analysis
transport equation [Ng et al. 03]
dual photography [Sen et al. 05; Sen and Darabi 09]
radiometric compensation [Wetzstein and Bimber 07]
radiosity equation [Goral et al. 84]
radiosity solution [Goral et al. 84]
inverse light transport [Seitz et al. 05; Bai et al. 10]
transport eigenvectors [O’Toole and Kutulakos 10]
structured light transport [O’Toole et al. CVPR 2014]
fast direct/global separation
[Nayar et al. SIGGRAPH 2006]
![Page 34: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/34.jpg)
structured light transport [O’Toole et al. CVPR 2014]
acquire direct & indirect ToF images using epipolar constraints
fast direct/global separation [Nayar et al. SIGGRAPH 2006]
acquire caustic indirect & diffuse indirect ToF images using radiometric constraints
![Page 35: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/35.jpg)
structured light transport [O’Toole et al. CVPR 2014]
acquire direct & indirect ToF images using epipolar constraints
fast direct/global separation [Nayar et al. SIGGRAPH 2006]
acquire caustic indirect & diffuse indirect ToF images using radiometric constraints
![Page 36: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/36.jpg)
mirror
ToF camera ToF projector
structured light transport for freq.
![Page 37: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/37.jpg)
mirror
ToF camera ToF projector
structured light transport for freq.
![Page 38: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/38.jpg)
mirror
direct paths obey epipolar geometry
ToF camera ToF projector
structured light transport for freq.
![Page 39: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/39.jpg)
mirror
indirect paths DONTobey epipolar geometry
ToF camera ToF projector
structured light transport for freq.
![Page 40: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/40.jpg)
mirror
projection pattern
mask pattern
ToF camera ToF projector
structured light transport for freq.
![Page 41: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/41.jpg)
mirror
mask pattern
ToF camera ToF projector
structured light transport for freq.
projection pattern
![Page 42: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/42.jpg)
mirror
projection pattern
mask pattern
1. open electronic shutter2. for i = 1 to N
use random epipolar mask &project complementary pattern
3. close electronic shutter
ToF camera ToF projector
structured light transport for freq.
![Page 43: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/43.jpg)
brightness
hue
conventional ToF imaging
![Page 44: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/44.jpg)
brightness
hue
indirect ToF
structured light transport conventional ToF imaging
![Page 45: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/45.jpg)
direct ToF
brightness
hue
indirect ToF
structured light transport conventional ToF imaging
![Page 46: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/46.jpg)
application: 3D from ToF
![Page 47: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/47.jpg)
ToF depth maps & indirect illumination
![Page 48: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/48.jpg)
direct-only
conventional vs direct-only ToF
conventional
![Page 49: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/49.jpg)
conventional direct-only
3D from conventional vs direct-only ToF
![Page 50: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/50.jpg)
conventional direct-only
conventional vs direct-only ToF
![Page 51: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/51.jpg)
conventional direct-only
conventional vs direct-only ToF
![Page 52: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/52.jpg)
conventional direct-only
3D from conventional vs direct-only ToF
![Page 53: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/53.jpg)
structured light transport [O’Toole et al. CVPR 2014]
acquire direct & indirect ToF images using epipolar constraints
fast direct/global separation [Nayar et al. SIGGRAPH 2006]
acquire caustic indirect & diffuse indirect ToF images using radiometric constraints
![Page 54: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/54.jpg)
mirror
effect of indirect transport on spatial frequencies
ToF camera ToF projector
![Page 55: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/55.jpg)
mirror
effect of indirect transport on spatial frequencies
caustic high frequencynon-caustic low frequencyToF camera ToF projector
![Page 56: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/56.jpg)
mirror
effect of indirect transport on spatial frequencies
ToF camera ToF projector
caustic high frequencynon-caustic low frequency
![Page 57: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/57.jpg)
mirror
effect of indirect transport on spatial frequencies
1. for i = 1 to Nproject high spatial-freq. patterncapture image
2. non-caustic = min of images
ToF camera ToF projector
caustic high frequencynon-caustic low frequency
![Page 58: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/58.jpg)
direct ToF
indirect ToF
brightness
hue
structured light transport conventional ToF imaging
![Page 59: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/59.jpg)
direct ToF non-caustic ToF
indirect ToF
structured light transport fast caustic/non-caustic separation
brightness
hue
conventional ToF imaging
![Page 60: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/60.jpg)
direct ToF non-caustic ToF
caustic ToFindirect ToF
structured light transport fast caustic/non-caustic separation
brightness
hue
conventional ToF imaging
![Page 61: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/61.jpg)
direct ToF non-caustic ToF
caustic ToF
![Page 62: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/62.jpg)
application: light-in-flight imaging
![Page 63: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/63.jpg)
mirror
“light-in-flight” imaging for time instant t
goal: reconstruct image for time t
ToF camera ToF projector
![Page 64: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/64.jpg)
“light-in-flight” imaging: prior work
femto-photography[Velten et al. 2013]
very high costwell-resolved wavefronts
PMD-based system[Heide et al. 2013]
many modulation frequenciesstrong scene priors
optical wavefronts not resolvable
our work
many modulation frequenciesweaker, transport-specific priors
resolvable optical wavefronts
![Page 65: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/65.jpg)
conventional ToF for = 100 MHz conventional image
piecewise “light-in-flight” video construction
![Page 66: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/66.jpg)
direct ToF for = 100 MHz light-in-flight video
piecewise “light-in-flight” video construction
direct (no regularization)
![Page 67: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/67.jpg)
caustic ToF for = 100 MHz light-in-flight video
piecewise “light-in-flight” video construction
direct (no regularization)
caustic indirect (no regularization)
![Page 68: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/68.jpg)
light-in-flight videonon-caustic ToF for = 12 to 140 MHz
piecewise “light-in-flight” video construction
direct (no regularization)
non-caustic indirect (regularizedusing [Heide et al. 2013])
caustic indirect (no regularization)
![Page 69: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/69.jpg)
light-in-flight video (ours)light-in-flight video [Heide et al. 13]
comparison to [Heide et al. 2013]
![Page 70: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/70.jpg)
light-in-flight video (ours)light-in-flight video [Heide et al. 13]
comparison to [Heide et al. 2013]
![Page 71: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/71.jpg)
conclusion
• a new paradigm for ToF sensing
• combines ToF sensors with projectors & masks
• can now leverage from decade of light transport research
• widespread implications for spatio-temporal transport analysis
![Page 72: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/72.jpg)
come visit us at our E-Tech booth
visualizing light transport phenomena
![Page 73: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/73.jpg)
Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport
http://www.dgp.toronto.edu/~motoole/temporalprobing.html
Matt O’Toole1 Felix Heide2 Lei Xiao2 Matthias Hullin3 Wolfgang Heidrich2,4 Kyros Kutulakos1
1University of Toronto 2University of British Columbia 3University of Bonn 4KAUST
![Page 74: Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport](https://reader034.fdocuments.in/reader034/viewer/2022052601/559839ae1a28ab2d628b4708/html5/thumbnails/74.jpg)
temporal probing setup
max. exposure time of 8 ms, max. modulation frequency of 150 MHz