Post on 22-Dec-2015
Virtual Computational Camera
Changyin Zhou
changyin@cs.columbia.edu
Computational Photography, Spring 2009
What is Camera?
Lens, Lens Array, Coded Aperture, 3D Phase plate ….
- Camera is a projection: [4D light field] [2D image]
2D Sensor
4DLight Field
Shape, BRDF
Illumination
- Optical Elements are transforms: [4D Light Field] [4D Light Field]
Input: 4D light field Input: Transforms … Ouput: 2D image
Motivation
From Scratch(using Matlab, Python, C, …)
- Ray tracing
- Not well structured for optical design
- Not easy to develop from scratch (little intuition, time-consuming, …)
Optical Simulation Software(Zemax, ASAP…)
- Optical Element Based- Define Optical Elements Physically (e.g. curvatures of surface, refractive index)
Works as a real system in most cases
Disadvantage:
- No light field …- A big gap between physics and math- Designed for conventional optical elements
• Coded aperture?• Focal Sweep?
Two Typical Ways to Build Virtual Cameras
Build An Abstract-Level Virtual Camera
Input: 4D light field Input: Transforms… Output: 2D image
An Abstract-LevelVirtual Camera
1. Light Field Based
2. “Object-Oriented”: Optical Elements
3. Concept-Level: Defined mathematically
Purpose:
Serve Computational Camera Research
Features:
Scene:4D Light Field
Optical Element:Transform of 4D Light Field
Sensor:Projection: 4D Light Field 2D Image
4. Pipeline
Build An Abstract-Level Virtual Camera
Scene:LF(u, v, s, t)
Lens:
function outLF = Lens(inLF, arg);
For each [u, v] outLF{u, v} =
interp2(X, Y, inLF{u, v}, X-u/f, Y-v/f);end
Any other element:
function outLF = Other(inLF, arg);
….….….
Sensor:
function outIM = sensor(inLF, inDist);
For each [u, v] outIM(round(XX(1, :)*desU), round(YY(:, 1)'*desV) … = outIM(round(XX(1, :)*desU), round(YY(:, 1)'*desV)… + interp2(XX, YY, inLF{u, v}, … round(XX*desU)/desU, round(YY*desV)/desV);
end
Build An Abstract-Level Virtual Camera
Function VirtualCamera(parameters);
LF = loadLF(filename);LF = lens(LF, arg);LF = codedAperture(LF, coding, arg);LF = propagation(LF, distance);
LF = otherOptics(LF, arg);
LF = propagation(LF, distance);
IM = sensor(LF);
END
Camera
Challenges
• Resolution– Huge data (Constrained to 10 x 10 x 1000 x 1000 input light fi
eld in this project)
– Angular/spatial resolution balance could be different at every layer.
• An effective Framework (user interface, data structure, function interface, …)
• Ray Interpolation
Thank you!