Gaussian KD-Tree for Fast High-Dimensional Filtering

A. Adams, N. Gelfand, J. Dolson, and M. Levoy, Stanford University,

SIGGRAPH 2009.

Edge-Preserving Filtering

Noise Suppression

Detail Enhancement

High Dynamic Range Imaging

Edge-Preserving Filtering for Image Analysis

Input Image

Base Image Detail Image

Edge-Preserving Vs. Edge-Blurring

Input Image

Edge-Preserving Base Image Edge-Blurring Base Image

Edge-Preserving Vs. Edge-Blurring

Edge-Preserving Enhanced Image Edge-Blurring Enhanced Image

Halo Artifacts

Gaussian Filtering

i

ii

i

i

i

yx

p

ipiviv

pixel ofposition : pixel of ueoutput val :ˆ pixel of ueintput val :

Gaussian Filtering

i

ii

i

i

i

yx

p

ipiviv

pixel ofposition : pixel of ueoutput val :ˆ pixel of ueintput val :

Bilateral Filtering

i

iii

i

i

yx

pip

iviv

, pixel ofposition :

pixel of ueoutput val :ˆ pixel of ueintput val :

Output InputSpace Weight Range Weight

Space Weight Range Weight

x

y

Intensity

Bilateral Filtering

i

iii

i

i

yx

pip

iviv

, pixel ofposition :

pixel of ueoutput val :ˆ pixel of ueintput val :

Output InputBilateral Weight

Space Weight Range Weight

x

y

Intensity

Bilateral Filtering

Input Image Gaussian: σp = 12

Bilateral: σp = 12, σc = 0.15

Computational Complexity of Bilateral Filtering

• O(n2d)– Image Size: n– Maximum Filter Size: n– Dimension: d

• High Computational Complexity

Input

x

y

Intensity

Novel Methods

• Bilateral Grid– J. Chen, S. Paris, and F. Durand, “Real-time

edgeaware image processing with the bilateral grid,” ACM Transactions on Graphics (Proc. SIGGRAPH 07).

• Gaussian KD-Tree– A. Adams, N. Gelfand, J. Dolson, and M. Levoy,

“Gaussian KD-Trees for Fast High-Dimensional Filtering,” ACM Transactions on Graphics (Proc. SIGGRAPH 09).

High-Dimensional Filtering

x

y

Intensity

A Two-Dimensional Example

x

ISpaceRange

Signal Kernel

x

IOutput Signal

Kernel

Gaussian Filtering

x

ISpace

Signal Output Signal

Bilateral Filtering

Large Kernel Size High Computational Complexity!

Bilateral Grid Downsampling

x

I

Signal

Bilateral Grid

x

I

Signal

Spatial Grid

Traditional Spatial Downsampling

x

I

Signal

Bilateral Grid

Bilateral Grid Downsamplingx

I Bilateral GridKernel

Bilateral Filter on the Bilateral GridImage scanline

space

inte

nsityBilateral

Grid

space

inte

nsity

Bilateral Filter on the Bilateral GridImage scanline

Filtered scanline

Slice: query gridwith input image

BilateralGrid

Gaussian blur grid values

space

inte

nsity

Bilateral Filtering for Color Image

Bilateral Filtering Based on Luminance Bilateral Filtering Based on Color

Bilateral Grid for Color Image

Tiiiiii

i

i

i

bgryxp

ipiviv

pixel ofposition : pixel of ueoutput val :ˆ pixel of ueintput val :

Image

High-Dimensional Grid (5d grid)

High Memory Usage Cost

Gaussian KD-Tree

• Low Computational Complexity• Low Memory Usage

Gaussian KD-Tree

• Building The Tree• Querying The Tree

Building The Tree

Space

Intensity

Bounding Box

Longest Dimension, η1d

η1min η1

max

η1cut

η1

Gaussian KD-Tree

Building The Tree

Space

Intensity

η2d

η2min η2

max

η2cut

η1

Gaussian KD-Tree

η2

η2

Building The Tree

Space

Intensity

η3d

η3min

η3max

η3cut

η1

Gaussian KD-Tree

η2

η3

η3

Building The Tree

Space

Intensity

η4d

η4min η4

max

η4cut

η1

Gaussian KD-Tree

η2

η4

η3

η4

Building The Tree

Space

Intensity

Inner Node• Cutting Dimension• Min, Max Bound• Left, Right Child

η1

Gaussian KD-Tree

η2 η3

η4

……………….

Building The Tree

Space

Intensity

Leaf Node• Position

Querying The Tree

η1

Gaussian KD-Tree

η2 η3

η4

……………….

High-Dimensional Space

Image Pixel Querying

Querying The Tree

Gaussian KD Tree

Inner Node

Leaf Node

Image Pixel

Different Weighting to Leaf Nodes

Splatting

1-D Example of Splatting

SpaceQuerying Position

Space

Querying Position

ηcut

Sample Distribution

ηcut

SplattingSplatting

1-D Example of Splatting

SpaceQuerying Position

Space

Querying Position

ηcut

Sample Distribution

ηcut

SplattingSplatting

Correcting Weights for Splatting

q

pi

Querying The Tree

Gaussian KD Tree

Inner Node

Leaf Node

Image Pixel

Sample Splitting to Leaf Nodes

Samples

Blurring The Leaf Nodes

Slicing

Summary

x

y

r,g,b

Input Image

Gaussian KD Tree

High-Dimensional Space

Resolution Reduction

• Monte-Carlo Sampling• Weighted Importance Sampling

Applications

• Bilateral Filtering

Naïve Bilateral Filtering 5-D Bilateral Grid

3-D Bilateral GridKD-Tree

Complexity and Performance Analysis

Filter Size

Large

Small

5D Grid

Gaussian KD-Tree

Naïve

Applications

• Non-local Mean Filtering

Input Image Output Image

Non-local Mean Filtering

Target Patch

Searching Patches

…..

ii

i

i

pipiviv

pixel ofposition : pixel of ueoutput val :ˆ pixel of ueintput val :

Patch

Non-local Mean Filtering with PCA

Patch Examples 16 Leading Eigenvectors

http://www.ceremade.dauphine.fr/~peyre/numerical-tour/tours/denoising_nl_means/

Non-local Mean Filtering

Target Patch

Searching Patches

…..

ii

i

i

pipiviv

pixel ofposition : pixel of ueoutput val :ˆ pixel of ueintput val :

Patch

High-Dimensional Space

Non-local Mean Filtering with Gaussian KD-Tree

Gaussian KD Tree

Inner Node

Leaf Node

Image Pixel

Different Weighting to Leaf Nodes

High-Dimensional Space

Applications

• Non-local Mean Filtering

Input Image Output Image

Applications

• Geometry Filtering

Input Model Output Model with Gaussian Filtering

Output Model with Non-local Mean

Conclusions

• Novel methods of non-linear filter.– Bilateral grid and Gaussian kd-tree

• High-dimensional non-linear filter.– Edge preserving smoothing

• Computational Complexity Reduction– Importance sampling