Processing Digital Camera Images WS 11/12

  English/Deutsch?

Language

  Responsible:

•  Prof. Dr. Marc Alexa, Prof. Dr. Bernd Bickel

  Main contact:

•  Mathias Eitz - m.eitz@tu-berlin.de

•  EN 721

  Administration:

•  Helga Kallan - sekr@cg.cs-tu.berlin.de

•  EN 716

We

  Have to work in teams - introduce yourself…

•  CS Student?

•  Graphics/Vision classes taken

•  Programming experience

•  Why interested in this course

  Register for course on www.cg.tu-berlin.de

You

  Basic Image Processing

•  Convolution, Morphological operators, Filters

•  But might need those as building blocks for projects

  Photoshop

•  Won’t learn how to use Photoshop

•  Theory of some advanced PS features might be covered (Panorama function of CS3 uses recent research results)

  Photography

•  Won’t learn how to take good pictures

•  But might need to take pictures for some projects

What this course is not about

  Computational Photography

•  Epsilon Photography

•  Take multiple pics with varying camera parameters

•  E.g.: HDR, Panorama

•  Coded Photography

•  Modifying camera lens to achieve a certain goal

•  E.g.: Deblurring, light field capture, relighting, dark flash

•  Data driven Vision and Graphics

•  Beyond the human eye

•  E.g.: Photo tourism, scene completion

What this course is about

  Problem: limited field of view of digital cameras

•  Compact camera: 50x35º

•  Human: 200x135º

•  Panorama 360x180º

Panoramic Images

  Early panoramic images:

•  View from the top of Lookout Mountain, Tennessee1864, by George N. Barnard

Panoramic Images

Image: Wikipedia

  Problem: limited contrast of digital cameras

High Dynamic Range Imaging

Images: Fattal et al.

[Fattal et al.]

  Solution:

•  Take multiple pictures, vary exposure time

•  “Combine” shots to produce a single “HDR” image

High Dynamic Range Imaging

  “Combine” exposures into high dynamic range map

•  Each pixel represents real world radiance

  New problem: how to display on LCD?

Recovering HDR

  Approach [Fattal et al.]

•  Attenuate strong contrast edges

•  Reconstruct image

Tone mapping

Gradient Domain Imaging

  Usually, we directly manipulate (RGB) pixels

  Working in gradient domain can have advantages:

•  Modifying image edges becomes easier

•  Blending becomes easier

  Problem: unwanted motion blur

Fluttered Shutter [Raskar et al.]

  Traditional camera: shutter is always open

Fluttered Shutter

  Fluttered Shutter: Distinct open close pattern

Fluttered Shutter

  Deblurred result

Fluttered Shutter

  Problem: How to resize an image?

•  Scaling/Cropping/Seam Carving?

Seam Carving [Avidan & Shamir]

scaled cropped seam carved

Seam Carving

Patchmatch [Barnes et al.]

What can we do with millions of

images?

  Replace unwanted object in an image

Scene Completion [Hays et al.]

  Find similar scenes from large image collection

  Copy & blend missing area

Scene Completion

Photo Tourism [Snavely et al.]

Live demo: http://phototour.cs.washington.edu

  Overview/Introduction of topics: 18.10., 25.10.

  Basic Talks: 01.11. ~ 15.11.

•  Teams start their project by preparing basic talk

•  Talk covers important aspect that is useful in itself

  Project roadmap: 22.11.

•  Slides presented in class: list milestones, relevant papers

  Implementation project: 22.11. – 07.02.

•  Project presentations & demos: 07.02., 14.02.

  Documentation (online): 21.02.

Approximate schedule

  Attendance mandatory, we keep a list

  Grading

•  Attendance & participation: 10%

•  Basic talk: 20%

•  Project roadmap: 10%

•  Project (implementation): 40%

•  Documentation (online): 20%

Rules I

  Reuse existing resources, but

•  Must give referencs

•  Example: you may use illustrations in your talk taken from other slides, but add a note such as [image: taken from Alexa et al.]

•  For some of your projects rather complete implementations are available on the web – use them for inspiration, do not copy!

•  If in doubt, ask Mathias

Rules II

  Check last years’ projects

•  Links are on the course website

•  Browse similar course homepages @ MIT, Stanford, Brown

  Check our project list (online Friday)

•  Find at least two projects you are interested in

•  After class contact potential team members

  Register for this course

Todo until next week

  Similar courses:

•  http://www.cs.brown.edu/courses/csci2951-b/

•  http://www.cs.brown.edu/courses/csci1290/

•  http://graphics.cs.cmu.edu/courses/15-463/

•  http://www.cc.gatech.edu/classes/AY2005/cs4803cp_summer/

•  http://graphics.stanford.edu/courses/cs448-04-spring/

•  http://people.csail.mit.edu/fredo/PhotoSeminar05/

•  http://cs.nyu.edu/~fergus/teaching/comp_photo/

•  http://www.cs.unc.edu/~lazebnik/research/fall08/

•  http://cameraculture.media.mit.edu/Fall2010ComputationalCamera

Resources - I

  Finding papers:

•  http://scholar.google.com/

  SIGGRAPH coursenotes

•  http://web.media.mit.edu/~raskar/photo/

  Books:

•  http://szeliski.org/Book/

•  http://www.amazon.com/dp/1568813139/

Resources - II

  Image/Video References:

•  Connelly Barnes, Eli Shechtman, Adam Finkelstein, Dan B Goldman - PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing���ACM Transactions on Graphics (Proc. SIGGRAPH), 2009

•  Shai Avidan and Ariel Shamir. Seam carving for content-aware image resizing. ACM Transactions on Graphics (Proc. SIGGRAPH), 2007

•  Noah Snavely, Steven M. Seitz, Richard Szeliski, Photo tourism: Exploring photo collections in 3D, ACM Transactions on Graphics (Proc. SIGGRAPH), 2006

•  Ramesh Raskar, Amit Agrawal, and Jack Tumblin, Coded Exposure Photography: Motion Deblurring using Fluttered Shutter, ACM SIGGRAPH, 2006

•  James Hays, Alexei A. Efros. Scene Completion Using Millions of Photographs. ACM Transactions on Graphics (Proc. SIGGRAPH), 2007

•  R. Fattal, D. Lischinski and M. Werman,Gradient Domain High Dynamic Range Compression, ACM Transactions on Graphics (Proc. SIGGRAPH), 2002

Resources – III