Content-aware detection of JPEG grid inconsistencies for ...
Real-Time Edge-Aware Image Processing with the Bilateral Grid
Transcript of Real-Time Edge-Aware Image Processing with the Bilateral Grid
![Page 1: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/1.jpg)
Real-Time Edge-Aware Image Processing with the Bilateral Grid
SIGGRAPH '07Jiawen Chen, Sylvain Paris, Frédo Durand
CSAIL - MIT
By Orazio Gallo, 01/29/08
![Page 2: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/2.jpg)
Motivation
● Many computational photography applications require piecewise smooth filtering
● Bilateral filtering is effective but slow: minutes per megapixel
● Many approximations and optimizations have already been proposed (~seconds per megapixel)
● Real time?
![Page 3: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/3.jpg)
Paper's Main Contribution
● The bilateral grid: a 3D data structure that naturally embeds edge information and allows for parallelization
● Now everything is real-time!● “Edge-aware” algorithms for different image
processing tasks
![Page 4: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/4.jpg)
Talk Overview
● Super-quick overview of bilateral filtering (?)● The data structure and its use● Example apps
![Page 5: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/5.jpg)
Bilateral Filtering
Edge + noise De-noised edge
x
(Adapted from Durand and Forsey 2002)
![Page 6: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/6.jpg)
More on Bilateral Filtering
Input
Output
space
p
q
p
inte
nsity
inte
nsity
![Page 7: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/7.jpg)
More on Bilateral Filtering
Input
Output
space
p
qinte
nsity
inte
nsity
p
![Page 8: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/8.jpg)
Bilateral Grid
The Bilateral GridStandard 2D Image
x
y
![Page 9: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/9.jpg)
• Nearest neighbor
• Bicubic
2D vs. Bilateral Grid Downsampling
x
y
Downsampling in 2D
![Page 10: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/10.jpg)
x
y
Downsampling in 2D Bilateral Grid
Extra dimension compensates for aggressive down-sampling!!
2D vs. Bilateral Grid Downsampling
(70 x 70 x 10 for an 8 megapixel image)
![Page 11: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/11.jpg)
Processing and Slicing● Processing:
any function that takes a 3D function as input!
● Slicing – getting back an image:use the original image to access the grid. The intensity level is obtained by interpolating along the z direction in the grid.
![Page 12: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/12.jpg)
Bilateral Filter
![Page 13: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/13.jpg)
Bilateral Filter on the Bilateral GridImage scanline
space
inte
nsityBilateral
Grid
![Page 14: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/14.jpg)
space
inte
nsity
Bilateral Filter on the Bilateral GridImage scanline
Filtered scanline
Slice: query gridwith input image
BilateralGrid
Gaussian blur grid values
space
inte
nsity
![Page 15: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/15.jpg)
Bilateral Filter on the Bilateral Grid
s is the geometric spread
r is the photometric spread
![Page 16: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/16.jpg)
Bilateral Filter Performance
Image size: 2 MPixels● CPU
– Brute force: 10 minutes– State of the art ’06: 1 second [Weiss 06, Paris 06]
● Our Bilateral Grid with GPU– 2004 card (NV40): 28 ms (36 Hz)– 2006 card (G80): 9 ms (111 Hz)
![Page 17: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/17.jpg)
Bilateral Filter Performance
Show:✔ bf.wmv
![Page 18: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/18.jpg)
Grid Painting
![Page 19: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/19.jpg)
Grid Painting
![Page 20: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/20.jpg)
Grid PaintingImage scanline
space
inte
nsityBilateral
Grid
![Page 21: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/21.jpg)
space
inte
nsity
Grid PaintingImage scanline
Output brush stroke
Query grid with input image: slicing
BilateralGrid
![Page 22: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/22.jpg)
Grid Painting
Show:✔ gp_3d.wmv✔ gp_recut.wmv
![Page 23: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/23.jpg)
Tone Mapping for HDRI
![Page 24: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/24.jpg)
HDRI and Tone Mapping
● CRT displays 100:1● LCD displays up to ~1'000:1● Photographic prints up to ~1'000:1● Real world scenes: > 100'000:1
![Page 25: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/25.jpg)
HDRI and Tone Mapping
From Wikipedia
![Page 26: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/26.jpg)
HDRI and Tone Mapping
From Wikipedia
![Page 27: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/27.jpg)
HDRI and Tone Mapping
![Page 28: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/28.jpg)
The algorithm [Durand and Dorsey 02]
Color ratiosDetail
Large scale
Preserve
Linear remapping
exp
Log luminance
![Page 29: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/29.jpg)
User-Driven, Local Tone Mapping
![Page 30: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/30.jpg)
User-Driven, Local Tone Mapping
Show:✔ ToneMapping.wmv
![Page 31: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/31.jpg)
Other Applications: Histogram Equalization
Matlab's adapthisteq (1.5s) Their result (100ms)
1760 x 2140, HDR
![Page 32: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/32.jpg)
Other Applications: Transfer of Photographic Look [Bae 06]
● Artistic rendition from model– 2 bilateral filters
● Remove compression artifacts● Split into large-scale and small-scale layers
– 2 histogram transfers● Match statistics on layers
● 1.5 megapixels at 30 Hz
Model
Input
Output
![Page 33: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/33.jpg)
Summary● Definition of the Bilateral Grid:
– Intuition, creation, processing, and slicing● Applications
– Bilateral filtering– Grid painting– Local tone mapping– Local histogram equalization– Transfer of Photographic look– ...
![Page 34: Real-Time Edge-Aware Image Processing with the Bilateral Grid](https://reader031.fdocuments.in/reader031/viewer/2022020704/61fb62ad2e268c58cd5d8b94/html5/thumbnails/34.jpg)
thanks!