Flexible Voxels for Motion-Aware Videography
1
Flexible Voxels for Motion-Aware Videography Mohit Gupta Srinivasa G. Narasimhan Amit Agrawal Sampling of the Space-Time Volume Motion-Aware Camera Ashok Veeraraghavan Post Capture Motion-Aware Interpretation of Data Hardware Implementation: Fast Per-pixel Shutter High Frame Rate (500 fps) for Fast Moving Regions High Spatial Resolution (1 MP) for Static Regions Post-Capture control of frame rate and spatial resolution Time Camera Sensor Plane Integration Time Conventional Sampling Scheme: Frame 1Frame 2 Frame N Fat and short voxels Low SR, High TR Flexible voxels Motion Aware Video Thin and long voxels High SR, Low TR x t y Integration Time Time Camera Sensor Plane Our Sampling Scheme: Frame 1Frame 2 Frame N Why is Building Such a Camera Hard? Motion-Aware video requires flexible voxels and a priori knowledge of the scene Image Plane Projecto r Image Plane Camera Scene Beam Splitte r Time Pixel 1 Pixel 2 Pixel K Camera Integration Projector Pattern Motion-Aware Video Results Fan Rotating (‘Wagon- wheel effect’) Multi-Use Light Engine (MULE) Pico-Projector (1440Hz.) ($350) Co-located projector camera setup Pixel on Pixel off Fast optical modulation Future work: Passive implementation using LCoS [Mukaigawa et al] Input Frame (Captured at 7.5 FPS) Anisotropic diffusion Motion-Aware reconstruction using motion information Captured Data Different Spatio-temporal Interpretations Re- binning TR = 1X SR = 1X TR = 2X SR = ½ X TR = 4X SR = ¼X Time Space Captured Frame Simple re-binning results in aliasing artifacts High SR High TR Diffusion Kernels Optical Flow Red = Fast, Green = Slow, Blue = Stationary Diffusion Kernels Aligned Along Flow Directions Motion-Aware Reconstruction High SR High TR Reduced Aliasing Increasing Temporal Resolution Coded Motion Blur Reconstructed Frame (60 FPS) (1 out of 8) High TR High SR
-
Upload
lewis-cain -
Category
Documents
-
view
27 -
download
0
description
Flexible Voxels for Motion-Aware Videography. Mohit Gupta. Amit Agrawal. Ashok Veeraraghavan. Srinivasa G. Narasimhan. y. t. Sampling of the Space-Time Volume. Motion-Aware Camera. Why is Building Such a Camera Hard?. x. Conventional Sampling Scheme:. High SR, Low TR. - PowerPoint PPT Presentation
Transcript of Flexible Voxels for Motion-Aware Videography
Flexible Voxels for Motion-Aware Videography Mohit Gupta Srinivasa G. NarasimhanAmit Agrawal
Sampling of the Space-Time VolumeMotion-Aware Camera
Ashok Veeraraghavan
Post Capture Motion-Aware Interpretation of Data
Hardware Implementation: Fast Per-pixel Shutter
High Frame Rate (500 fps) for Fast Moving Regions
High Spatial Resolution (1 MP) for Static Regions
Post-Capture control of frame rate and spatial resolution
Time
Camera
Sensor Plane
Integration Time
Conventional Sampling Scheme:Frame 1 Frame 2 Frame N
Fat and short voxels
Low SR, High TR
Flexible voxels
Motion Aware Video
Thin and long voxels
High SR, Low TR
x
ty
Integration Time Time
Camera
Sensor Plane
Our Sampling Scheme:Frame 1 Frame 2 Frame N
Why is Building Such a Camera Hard?
Motion-Aware video requires flexible voxels and a priori knowledge of the scene
Image Plane
ProjectorImage Plane
Camera
Scene
Beam Splitter
Time
Pixel 1
Pixel 2
Pixel K
Camera Integration
Projector Pattern
Motion-Aware Video Results Fan Rotating (‘Wagon-wheel effect’)
Multi-Use Light Engine (MULE)
Pico-Projector (1440Hz.) ($350)
Co-located projector camera setup
Pixel on
Pixel off
Fast optical modulationFuture work: Passive
implementation using LCoS
[Mukaigawa et al]
Input Frame (Captured at 7.5 FPS)
Anisotropic diffusion Motion-Aware reconstruction using motion information
Captured Data Different Spatio-temporal Interpretations
Re-binningTR = 1X SR = 1X
TR = 2X SR = ½ X
TR = 4X SR = ¼X
Time
Spa
ce
Captured Frame
Simple re-binning results in aliasing artifacts
High SR High TR
Diffusion Kernels
Optical Flow Red = Fast, Green = Slow,
Blue = Stationary
Diffusion Kernels Aligned Along Flow Directions
Motion-Aware Reconstruction
High SR High TR
Reduced AliasingIncreasing Temporal Resolution
Coded Motion Blur
Reconstructed Frame (60 FPS) (1 out of 8)
High TR High SR
