Marc Levoy Light Field = Array of (virtual) Cameras Sub-aperture Virtual Camera = Sub-aperture...
-
Upload
arthur-hall -
Category
Documents
-
view
229 -
download
4
Transcript of Marc Levoy Light Field = Array of (virtual) Cameras Sub-aperture Virtual Camera = Sub-aperture...
Marc Levoy
Light Field = Array of (virtual) Cameras
Sub-aperture
Virtual Camera = Sub-aperture View
MERL Mask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture Veeraraghavan, Raskar, Agrawal, Mohan & Tumblin
• Samples individual rays
• Predefined spectrum for lenses• Chromatic abberration
• High alignment precision
• Peripheral pixels wasted pixels
• Negligible Light Loss
• Samples coded combination of rays
• Supports any wavelength
• Reconfigurable f/#, Easier alignment
• No wastage• High resolution image for parts
of scene in focus
• 50 % Light Loss due to mask
Mask
Sensor
Microlens array
Sensor
Plenoptic Camera Heterodyne Camera
x
θ l(x,θ)
x
θ
l(x,θ)
x1
x2
θi θj
θj
x1x2
x1’ = x1 + θi*z
Shear of Light Field
θi
θj
x'1x1
Light Propagation (Defocus Blur)Light Propagation (Defocus Blur)
1-D FFT1-D FFT
2-D FFT
fx
fθL(fx,fθ)
Central Slice
x
θ l(x,θ)
x
θ l(x,θ)
Line Integral
Captured Photo
FFT of Captured
Photo
Space of LF representationsTime-frequency representationsPhase space representationsQuasi light field
WDF
TraditionTraditional light al light
fieldfield
TraditionTraditional light al light
fieldfield
Augmented LFObservable LF
RihaczekDistribution Function
incoherent
coherent
Other LF representatio
ns
Other LF representatio
ns
Other LF representatio
ns
Other LF representatio
ns
Quasi light fieldsthe utility of light fields, the versatility of Maxwell
We form coherent images by
formulating,
capturing,
and integrating
quasi light fields.
WDF
TraditionTraditional light al light
fieldfield
TraditionTraditional light al light
fieldfield
Augmented LFObservable LF
RihaczekDistributi
on Function
incoherent
coherent
Other LF representati
ons
Other LF representati
ons
Other LF representati
ons
Other LF representati
ons
(i) Observable Light Field
• move aperture across plane
• look at directional spread
• continuous form of plenoptic camera
scene
apertureposition s
direction u
(ii) Augmented Light Field with LF Transformer
8
WDFWDF
Light Light FieldFieldLight Light FieldField
Augmented LF
Augmented LF
Interaction at the optical elementsInteraction at the optical elements
LF propagatio
n
(diffractive)
optical element
LF LF LF LF
LF propagatio
n
light field transformer
negative
radiance
Virtual light projector with real valued (possibly negative radiance) along a ray
9
real projector
real projector
first null (OPD = λ/2)
virtual light projector
(ii) ALF with LF Transformer
10
(iii) Rihaczek Distribution FunctionTradeoff between cross-interference terms and localization
y
u
0 m
3 m
0 m 3 m y0 m 3 m y0 m 3 m
u
y
(i) Spectrogram non-negative localization
(ii) Wigner localization cross terms
(iii) Rihaczek localization complex
Property of the Representation
Constant along rays
Non-negativity
Coherence WavelengthInterference Cross term
Traditional LF
always constant
always positive
only incoherent
zero no
Observable LF
nearly constant
always positive
any coherence
stateany yes
Augmented LF
only in the paraxial region
positive and negative
any any yes
WDFonly in the paraxial region
positive and negative
any any yes
Rihaczek DFno; linear
driftcomplex any any reduced
Benefits & Limitations of the Representation
Ability to propagate
Modeling wave optics
Simplicity of
computation
Adaptability to
current pipe line
Near Field Far Field
Traditional LF
x-shear novery
simplehigh no yes
Observable LF
not x-shear
yes modest low yes yes
Augmented LF
x-shear yes modest high no yes
WDF x-shear yes modest low yes yes
Rihaczek DF
x-shear yes
better than WDF,
not as simple as
LF
low no yes
MotivationMotivation
• What is the difference between a hologram What is the difference between a hologram and a lenticular screen?and a lenticular screen?
• How they capture ‘phase’ of a wavefront for How they capture ‘phase’ of a wavefront for telescope applications?telescope applications?
• What is ‘wavefront coding’ lens for extended What is ‘wavefront coding’ lens for extended depth of field imaging?depth of field imaging?
Application - Wavefront Coding
Dowski and Cathey 1995
same aberrant blur regardless of depth of focus
cubicphase plate
pointin scene
small changein blur shape
Can they be part of Computer Vision?Moving away from 2D images or 4D lightfields?
Wavefront coding: WLC mobile phone cameras
Holography:Reference targets
Rendering: New perspective
projection methods
Gaussian beam lasers:Modern active illumination
Rotating PSF: Depth from defocus
QuickTime™ and aBMP decompressor
are needed to see this picture.
Raskar, Camera Culture, MIT Media Lab
Computational Photography
1. Epsilon Photography– Low-level Vision: Pixels– Multiphotos by bracketing (HDR, panorama)– ‘Ultimate camera’
2. Coded Photography– Mid-Level Cues:
• Regions, Edges, Motion, Direct/global– Single/few snapshot
• Reversible encoding of data, Lightfield– Additional sensors/optics/illum– ‘Smart Camera’
3. Essence Photography– Not mimic human eye– Beyond single view/illum– ‘New artform’
http://computationalphotography.org
ResourcesResources
• WebsiteWebsite– http://scripts.mit.edu/~raskar/lightfields/http://scripts.mit.edu/~raskar/lightfields/– Or follow http://cvpr2009.org tutorial pagesOr follow http://cvpr2009.org tutorial pages
• Key new papersKey new papers• Wigner Distributions and How They Relate to the Light FieldWigner Distributions and How They Relate to the Light Field
Zhengyun Zhang Zhengyun Zhang and Marc Levoy, ICCP 2009 (best paper)and Marc Levoy, ICCP 2009 (best paper)
• Augmenting Light Field to Model Wave Optics Effects , Augmenting Light Field to Model Wave Optics Effects , Se Baek OhSe Baek Oh, Barbastathis, , Barbastathis, RaskarRaskar (in Preparation) (in Preparation)
• Quasi light fields: extending the light field to coherent radiation , Quasi light fields: extending the light field to coherent radiation , Anthony AccardiAnthony Accardi, Wornell (in Preparation), Wornell (in Preparation)
WDF
TraditionTraditional light al light
fieldfield
TraditionTraditional light al light
fieldfield
Augmented LFObservable LF
RihaczekDistribution Function
AcknowledgementsAcknowledgements
• DarthmuthDarthmuth– Marcus Testorf, Marcus Testorf,
• MITMIT– Ankit Mohan, Ahmed Kirmani, Jaewon KimAnkit Mohan, Ahmed Kirmani, Jaewon Kim– George BarbastathisGeorge Barbastathis
• StanfordStanford– Marc Levoy, Ren Ng, Andrew AdamsMarc Levoy, Ren Ng, Andrew Adams
• AdobeAdobe– Todor Georgiev, Todor Georgiev,
• MERLMERL– Ashok Veeraraghavan, Amit AgrawalAshok Veeraraghavan, Amit Agrawal
MIT Media LabMIT Media Lab
Camera CultureCamera Culture
Ramesh RaskarRamesh Raskar
MIT Media LabMIT Media Lab
http:// CameraCulture . info/http:// CameraCulture . info/
Light FieldsLight Fields______ Light FieldsLight Fields______
Light Fields in Ray and Wave Optics
Introduction to Light Fields: Ramesh Raskar
Wigner Distribution Function to explain Light Fields: Zhengyun Zhang
Augmenting LF to explain Wigner Distribution Function: Se Baek Oh
Q&A
Break
Light Fields with Coherent Light: Anthony Accardi
New Opportunities and Applications: Raskar and Oh
Q&A: All