Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Camera CultureAssociate Professor, MIT Media Lab
Computational Photography
http://raskar.info
Invertible Motion Blur in Video
Photo 1 Photo 2 Photo 3 Deblurring
Agrawal, Xu, Raskar, Siggraph 2009
Traditional Exposure Video
DFTMotion PSF (Box Filter)
Information is lost
Exposure Time
Coded Exposure (Flutter Shutter) Raskar et al. 2006
Single Photo Deblurred Image
Varying Exposure Video
DFTExposure Time
Varying Exposure Video
DFTExposure Time
Exposure Time
Varying Exposure Video
DFT
Exposure Time
Exposure Time
Exposure Time
Varying Exposure Video == PSF Null-Filling
DFT
Joint Frequency Spectrum
Preserves High Frequencies
Varying Exposure Video: Exploit auto-exposure mode
Completely automatic: (i) Segmentation, (ii) PSF estimation, (iii) deblurring
Blurred Photo
Deblurred Result Ground Truth
Input Photos
SynthesisLow Level Mid Level High
LevelHyper
realism
Raw
Angle, spectrum
aware
Non-visual Data, GPS
Metadata
Priors
Comprehensive
8D reflectance field
Digital
Epsilon
Coded
Essence
Computational Photography aims to make progress on
both axis
Camera ArrayHDR, FoV Focal stack
Decomposition problems
Depth
Spectrum
LightFields
Human Stereo Vision
Transient Imaging
Virtual Object Insertion
Relighting
Augmented Human
Experience
Material editing from single photo
Scene completion from photos
Motion Magnification
Phototourism
Resolution
Computational Photography (vs Imaging)
MIT Media Lab Camera Culture
Lots of glass; Heavy; Bulky; Expensive
Enhanced Defocus Blur
MIT Media Lab Camera Culture
Image Destabilization
Lens Sensor
Camera
Static Scene
MIT Media Lab Camera Culture
Image Destabilization
Static Scene
Lens Motion Sensor Motion
Camera
Mohan, Lanman,Hiura, Raskar ICCP 2009
MIT Media Lab Camera Culture
“Time Lens”
Lens Equation:
Virtual Focal Length:
Virtual F-Number:
Ratio of speeds
MIT Media Lab Camera Culture
Adjusting the Focus Plane
all-in-focus image
MIT Media Lab Camera Culture
Adjusting the Focus Plane
focused in the front using destabilization
MIT Media Lab Camera Culture
Adjusting the Focus Plane
focused in the middle using destabilization
MIT Media Lab Camera Culture
Adjusting the Focus Plane
focused in the back using destabilization
MIT Media Lab Camera Culture
Bokode
MIT Media Lab Camera Culture
• Smart Barcode size : 3mm x 3mm• Ordinary Camera: Distance 3 meter
Long Distance Barcodes
MIT media lab camera culture bokode.com mohan, woo, hiura, smithwick, raskar
MIT Media Lab Camera Culture
Defocus blur of Bokode
MIT Media Lab Camera Culture
Coding in Angle
Mohan, Woo, Smithwick, Hiura, Raskar [Siggraph 2009]
MIT media lab camera culture bokode.com mohan, woo, hiura, smithwick, raskar
camera
sensorBokode(angle)
Encoding in Angle, not space, time or wavelength
MIT media lab camera culture bokode.com mohan, woo, hiura, smithwick, raskar
camera
sensor
circle of confusion circle of information
Bokode(angle)
Quote suggested by Kurt Akeley
Encoding in Angle, not space, time or wavelength
MIT media lab camera culture bokode.com mohan, woo, hiura, smithwick, raskar
Bokode
camera
fb
fc
magnification = fc /fb (microscope);
focus always at infinity
‘long-distance microscope’
MIT media lab camera culture
Product labels
Street-view Tagging
MIT media lab camera culture bokode.com mohan, woo, hiura, smithwick, raskar
cell-phone cameraclose to the Bokode
(10,000+ bytes of data)
capturing Bokodes
Augmenting Plenoptic Function
38
Wigner Distribution Function
TraditionaTraditional Light l Light FieldField
WDF
TraditionaTraditional Light l Light FieldField
Augmented LF
Interference & DiffractionInteraction w/ optical elements
ray optics basedsimple and powerful
wave optics basedrigorous but cumbersome
Light Fields
Referenceplane
position angle
Goal: Representing propagation, interaction and image formation of light using purely position and angle parameters
LF propagation
(diffractive)optical
element
LF LF LF LF
LF propagation
light field transformer
Free-space propagation
Light field transformer
Virtual light projectorPossibly negative radiance
40
41
LF Transformer: input to output LF Thin Elements: 6D
General Case: 8D
Angle Shift Invariant: 4D
Augmented LF framework
42
1. LF propagation
(diffractive)
optical element
LF LF LF LF
LF propagation
2. light field transformer
3. negative radiance
Tech report, S. B. Oh et al. http://web.media.mit.edu/~raskar/RayWavefront/
4. interference
Cubic phase plate
43
Pure ray bendingPositive only
radiance
DiffractionInterference of +/-
rays
Rotating PSF
Interference received on Complex Geometry
44
Can you ‘see’ around a corner ?
Femto-Photography: Higher Dimensional LF
FemtoFlash
UltraFast Detector
Computational OpticsSerious Sync
Important DatesSubmission: November 2, 2009Notification : February 2, 2010
Topics
Computational CamerasMultiple Images and Camera ArraysComputational IlluminationAdvanced Image and Video ProcessingScientific Photography and VideographyOrganizing and Exploiting Photo & Video Collections
Program ChairsKyros Kutulakos, U. TorontoRafael Piestun, U. ColoradoRamesh Raskar, MIT
International Conference on Computational Photography (ICCP)
March 29-30, 2010 MIT, Cambridge MA
http://cameraculture.media.mit.edu/ iccp10 /
Beyond Multi-touch: Mobile
Laptops
Mobile
Converting LCD Screen = large Camera for 3D Interactive HCI and Video Conferencing
Matthew Hirsch, Henry HoltzmanDoug Lanman, Ramesh Raskar
Siggraph Asia 2009
BiDi Screen
Overview: Sensing Depth from Array of Virtual Cameras in
LCD
Camera Culture Group, MIT Media Lab Ramesh Raskar http://raskar.info
• Beyond Traditional Imaging• Invertible motion blur in video• Looking around a corner• LCDs as virtual cameras• Computational probes (bokode)• Image destabilization
• Augmented Light Field• Rays for diffraction+interference
Computational PhotographyDigital
Epsilon
Coded
Essence
Computational Photography aims to make progress on both axis
Camera Array
HDR, FoV Focal stack
Decomposition problems
Depth
Spectrum
LightFields
Human Stereo Vision
Transient Imaging
Virtual Object Insertion
Relighting
Augmented Human
Experience
Material editing from single
photo
Scene completion from
photos
Motion Magnification
Phototourism
WDF
Light Light FieldField
Augmented LF
LF propagati
on
(diffractive)
optical element
LF
LF
LF
LF
LF propagati
on
light field transformer
Top Related