Frédo Durand, George Drettakis, Joëlle Thollot and Claude ...
Frequency Analysis and Sheared Reconstruction for Rendering Motion Blur Kevin Egan Yu-Ting Tseng...
-
date post
21-Dec-2015 -
Category
Documents
-
view
226 -
download
0
Transcript of Frequency Analysis and Sheared Reconstruction for Rendering Motion Blur Kevin Egan Yu-Ting Tseng...
Frequency Analysis and Sheared Reconstruction for Rendering Motion Blur
Frequency Analysis and Sheared Reconstruction for Rendering Motion Blur
Kevin Egan
Yu-Ting Tseng
Nicolas Holzschuch
Frédo Durand
Ravi Ramamoorthi
Columbia University
Columbia University
Grenoble University
MIT CSAIL
University of California, Berkeley
OverviewOverview
Introduction
• Overview of Frequency Analysis
• Sampling and Sheared Filter
• Implementation
• Results
Motion BlurMotion Blur
• Objects move while camera shutter is open
– Image is “blurred” over time
• Expensive for special effects
• Necessary to remove “strobing” in animations
A Simple ApproachA Simple Approach
• For each pixel
– Sample many different moments in time
t = 1.00
t = 0.75
t = 0.50
t = 0.25
t = 0.00
t [0.0, 1.0]
Standard MethodStandard Method
• Use axis-aligned pixel filter at each pixel
• Requires many samples
SPACE
TIM
E
space-time samples
axis-aligned filter
Our MethodOur Method
• Use a different filter shape at each pixel
• Filter sheared in space-time
• Fewer samples and faster renders
SPACE
TIM
E
space-time samples
sheared filter
Previous WorkPrevious Work
• Plenoptic Sampling
Chai et al., 2000
Sheared reconstruction filter in space-angle
• A Frequency Analysis of Light Transport
Durand et al., 2005
Analysis of transport, reflection and occlusion
• We extend both to space-time
Previous WorkPrevious Work
• Multidimensional Adaptive Sampling
Hachisuka et al., 2008
Anisotropic reconstruction based on contrast
Our method based on local freq estimates
• Spatial Anti-Aliasing for Animation Sequences with Spatio-Temporal Filtering
Shinya, 1993
Sheared filter across multiple images
We use speed bounds, derive sampling rates
OverviewOverview
• Introduction
Overview of Frequency Analysis
• Sampling and Sheared Filter
• Implementation
• Results
Space-Time Analysis GoalsSpace-Time Analysis Goals
• Uniform motion leads to sparse freq content
• Analyze shutter filter’s effect on freq content
• Design a filter customized to freq content
Shear in Space-TimeShear in Space-Time
x
y t
x
f(x, t)
• Object moving with low velocity
f(x, y)
shear
Large Shear in Space-TimeLarge Shear in Space-Time
x
y t
x
• Object moving with high velocity
f(x, y) f(x, t)
Camera Shutter FilterCamera Shutter Filter
• Applying shutter blurs across time
x
y t
x
f(x, y) f(x, t)
Basic Example – Fourier DomainBasic Example – Fourier Domain
• Fourier spectrum, zero velocity
t
x
f(x, t) F(Ωx, Ωt)texture
bandwidth
Ωt
Ωx
Basic Example – Fourier DomainBasic Example – Fourier Domain
• Low velocity, small shear in both domains
f(x, t) F(Ωx, Ωt)
t
x
slope = -speed
Ωt
Ωx
Basic Example – Fourier DomainBasic Example – Fourier Domain
• Large shear
f(x, t) F(Ωx, Ωt)
t
x Ωt
Ωx
Basic Example – Fourier DomainBasic Example – Fourier Domain
• Non-linear motion, wedge shaped spectra
f(x, t)
Ωt
Ωx
F(Ωx, Ωt)
t
x
shutter bandlimits in
time
-min speed
-max speed
shutter applies blur across time
indirectly bandlimits in
space
Main InsightsMain Insights
• Common case = double wedge spectra
• Shutter indirectly removes spatial freqs
• Moving reflections and shadows in paper
Ωt
Ωx
double wedge spectrum moving reflection
OverviewOverview
• Introduction and Simple Example
• Overview of Frequency Analysis
Sampling and Sheared Filter
• Implementation
• Results
Sampling and Filtering GoalsSampling and Filtering Goals
• Minimal sampling rates to prevent aliasing
• Derive shape of new reconstruction filter
Sampling in Fourier DomainSampling in Fourier Domain
Ωt
Ωxt
x
• Sampling produces replicas in Fourier domain
• Sparse sampling produces dense replicas
Fourier DomainSpace-Time Domain
Standard Reconstruction FilteringStandard Reconstruction Filtering
• Standard filter, dense sampling (slow)
Ωt
no aliasing
Ωx
Fourier Domainreplicas
Standard Reconstruction FilterStandard Reconstruction Filter
• Standard filter, sparse sampling (fast)
Ωt
aliasing
Ωx
Fourier Domain
Sheared Reconstruction FilterSheared Reconstruction Filter
• Our sheared filter, sparse sampling (fast)
Ωt
Ωx
No aliasing!
Fourier Domain
Sheared Reconstruction FilterSheared Reconstruction Filter
• Compact shape in Fourier = wide space-time
t
x
Space-Time Domain
Ωt
Ωx
Fourier Domain
Main InsightsMain Insights
• Sheared filter allows for many fewer samples
• Paper derives sampling rates
– for constant velocity same cost as a static image
OverviewOverview
• Introduction
• Overview of Frequency Analysis
• Sampling and Sheared Filter
Implementation
• Results
Our MethodOur Method
1. Compute bounds for signal speeds
2. Compute filter shapes and sampling rates
3. Render samples and reconstruct image
Implementation: Stage 1Implementation: Stage 1
• Sparse sampling to compute velocity bounds
max speedmin speed
Implementation: Stage 2Implementation: Stage 2
• Calculate filter widths and sampling rates
filter width
max speed
min speed
Implementation: Stage 2Implementation: Stage 2
• Uniform velocities, wide filter, low samples
filter width
max speed
min speed
Implementation: Stage 2Implementation: Stage 2
• Static surface, small filter, low samples
filter width
max speed
min speed
Implementation: Stage 2Implementation: Stage 2
• Varying velocities, small filter, high samples
filter width
max speed
min speed
Implementation: Stage 2Implementation: Stage 2
• Then compute sampling densities
samples per pixel
Uniform velocities =low sample count
Implementation: Stage 2Implementation: Stage 2
• Then compute sampling densities
samples per pixel
Varying velocities =high sample count
Implementation: Stage 3Implementation: Stage 3
• Render sample locations in space-time
• Apply wide sheared filters to nearby samples
t
x
sheared filter overlaps samples in multiple pixels
pixels
Implementation: Stage 3Implementation: Stage 3
• Filters stretched along direction of motion
• Preserve frequencies orthogonal to motion
filter shapes
OverviewOverview
• Introduction
• Overview of Frequency Analysis
• Sheared Filter
• Implementation
Results
Ballerina InsetBallerina Inset
Stratified16 samples / pix
4 min 2 sec
Our Method8 samples / pix3 min 57 sec
Stratified64 samples / pix14 min 25 sec
Equal Time Equal Quality
LimitationsLimitations
• Currently filter along line segments
• Initial sampling may miss motion
• Need to calculate speed and bandlimits
• Multiple texture / reflection / shadow signals
ConclusionConclusion
• Derivation of filter shape and sampling rates
• conservative min/max bounds for speed
• packing spectra as tightly as possible• Implementation of sheared filter
• No explicit representation for spectrum
• Easily added to existing rendering pipelines
• Faster render times
• Space-time Fourier theory for rendering
• Double wedge shape using min/max bounds
• Analysis of shutter filter
AcknowledgementsAcknowledgements
• Funding Agencies
• ONR, NSF, Microsoft, Sloan Foundation, INRIA, LJK
• Equipment and Licenses
• Pixar, Intel, NVIDIA
Previous WorkPrevious Work
• Distributed Ray Tracing,
Cook et al., 1984
Sampling across image, time and lens
• The Reyes Image Rendering Architecture,
Cook et al., 1987
Separate shading from visibility