CS 691 Computational Photography

Instructor: Gianfranco DorettoImage Pyramids

Linear image transformations

• In analyzing images, it’s often useful to make a change of basis.

Basis

fUF

Vectorized image

transformed image

Spatial Domain Basis

Basis functions:

……

……

..

Tells you where things are….

… but no concept of what it is

Identity transform

= *

Pixel domain image

Spatial bases are local: each transform coefficient depends on one pixel location.

Pixel domain

image

1000…01000…00100…

…0001…00010

000100…

…0001000…

€

rF

€

rf

€

U

Fourier Domain Basis

Basis functions:

Tells you what (frequency) is in the image….

… but not where it is

……

……

……

Fourier transform

= *

Pixel domain image

Fourier bases are global: each transform coefficient depends on all pixel locations.

Fourier transform

€

rF

€

rf

€

U

Image Analysis

• Want representation that combines what and where.

Image Pyramids

Overview

• Gaussian Pyramid• Laplacian Pyramid

Image Pyramids

Known as a Gaussian Pyramid [Burt and Adelson, 1983]• In computer graphics, a mip map [Williams, 1983]• A precursor to wavelet transform

Gaussian pyramid construction

filter mask

Repeat• Filter (keep filter the same size)• Subsample (by a factor of two)

Until minimum resolution reached • can specify desired number of levels (e.g., 3-level pyramid)

Total number of pixels in pyramid?• 1 + ¼ + 1/16 + 1/32…….. = 4/3

Over-complete representation

A bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s nose

What are they good for?

• Improve Search– Search over translations

• Classic coarse-to-fine strategy

– Search over scale• Template matching• E.g. find a face at different scales

• Pre-computation– Need to access image at different blur levels– Useful for texture mapping at different resolutions (called mip-

mapping)

• Compression– Capture important structures with fewer bytes

• Denoising– Model statistics of pyramid sub-bands– Image blending

Gaussian pyramid

= *Pixel domainimage

Overcomplete representation. Low-pass filters, sampled appropriately for their blur.

Gaussian pyramid

€

rF

€

rf

€

U

Overview

• Gaussian Pyramid• Laplacian Pyramid

What does blurring take away?

original

What does blurring take away?

smoothed (5x5 Gaussian)

High-Pass filter

smoothed – original

Band-pass filtering

• Laplacian Pyramid (subband images)

• Created from Gaussian pyramid by subtraction

Gaussian Pyramid (low-pass images)

Laplacian filter

GaussianUnit impulse Laplacian of Gaussian

- ≅

Laplacian pyramid algorithm

blur

_

subsample blur

_

subsampleblur

_

subsample

Can we reconstruct the original?

interpolateinterpolateinterpolate

Need this!

The Laplacian Pyramid

• Synthesis– preserve difference between upsampled

Gaussian pyramid level and Gaussian pyramid level

– band pass filter - each level represents spatial frequencies (largely) unrepresented at other levels

• Analysis– reconstruct Gaussian pyramid, take top

layer

Laplacian pyramid

Laplacian pyramid

= *Pixel domain image

Overcomplete representation. Transformed pixels represent bandpassed image information.

Laplacian pyramid

Hybrid Image in Laplacian Pyramid

High frequency Low frequencyExtra points for project 1

Slide Credits

• This set of sides also contains contributionskindly made available by the following authors– Alexei Efros– Svetlana Lazebnik– Frédo Durand– Bill Freeman– Steve Seitz– Derek Hoiem– David Forsyth