Daniel Cremers

Computer Science & Mathematics

TU Munich

Variational Methods and

Partial Differential Equations

2 Daniel Cremers Variational Methods and Partial Differential Equations

Spatially Dense Reconstruction

infinite-dimensional optimization

3 Daniel Cremers Variational Methods and Partial Differential Equations

Which path is the fastest?

4 Daniel Cremers Variational Methods and Partial Differential Equations

Bernoulli & The Brachistochrone

Johann Bernoulli (1667-1748)

5 Daniel Cremers Variational Methods and Partial Differential Equations

Image segmentation:

Optimization in Computer Vision

Kass et al. ’88, Mumford, Shah ’89, Caselles et al. ‘95,

Kichenassamy et al. ‘95, Paragios, Deriche ’99,

Chan, Vese ‘01, Tsai et al. ‘01, …

Multiview stereo reconstruction:

Faugeras, Keriven ’98, Duan et al. ‘04, Yezzi, Soatto ‘03,

Labatut et al. ’07, Kolev et al. IJCV ‘09…

Optical flow estimation:

Horn, Schunck ‘81, Nagel, Enkelmann ‘86, Black, Anandan ‘93,

Alvarez et al. ‘99, Brox et al. ‘04, Zach et al. ‘07, Sun et al. ‘08,

Wedel et al. ’09, Werlberger et al. ‘10, …

6 Daniel Cremers Variational Methods and Partial Differential Equations

Compute solutions via energy minimization

Optimization and Variational Methods

7 Daniel Cremers Variational Methods and Partial Differential Equations

Inverse Problems: Denoising

input image denoised

Rudin, Osher, Fatemi, Physica D ‘92

For compute:

8 Daniel Cremers Variational Methods and Partial Differential Equations

Variational Methods & PDEs

Extremality principle as a necessary condition:

But: How to define the derivative with respect to a function?

For consider the functional

9 Daniel Cremers Variational Methods and Partial Differential Equations

The Gateaux Derivative

Derivative in “direction” :

Taylor expansion:

10 Daniel Cremers Variational Methods and Partial Differential Equations

The Gateaux Derivative

Integration by parts:

Necessary optimality condition:

Euler-Lagrange equation

11 Daniel Cremers Variational Methods and Partial Differential Equations

Leonhard Euler

(1703-1783)

Joseph-Louis Lagrange

(1736 – 1813)

Variational Methods

12 Daniel Cremers Variational Methods and Partial Differential Equations

Gradient Descent

Iteratively walk “down-hill”:

direction of

steepest descent

For the energy

we obtain:

diffusion equation

13 Daniel Cremers Variational Methods and Partial Differential Equations

Diffusion

14 Daniel Cremers Variational Methods and Partial Differential Equations

Edge-preserving Denoising

Slightly modify the regularization (Rudin, Osher, Fatemi ‘92):

Gradient descent:

nonlinear diffusion

Perona, Malik ‘90, Rudin et al. ’92, Weickert ‘98

total variation

15 Daniel Cremers Variational Methods and Partial Differential Equations

Linear vs. Nonlinear Diffusion

16 Daniel Cremers Variational Methods and Partial Differential Equations

Variational Deblurring

original image deblurred image blurred image

Lions, Osher, Rudin, ‘92

17 Daniel Cremers Variational Methods and Partial Differential Equations

Variational Super-Resolution

Schoenemann, Cremers, IEEE TIP ’12

One of several input images Super-resolution estimate

18 Daniel Cremers Variational Methods and Partial Differential Equations

Variational Optical Flow

Input video Optical flow field

Horn & Schunck ‘81, Zach et al. DAGM ’07, Wedel et al. ICCV ’09

19 Daniel Cremers Variational Methods and Partial Differential Equations

*

* 60 fps @ 640x480

Input video Optical flow field

Variational Optical Flow

Horn & Schunck ‘81, Zach et al. DAGM ’07, Wedel et al. ICCV ’09

20 Daniel Cremers Variational Methods and Partial Differential Equations

one of two input images depth reconstruction

Variational Stereo Reconstruction

Pock , Schoenemann, Graber, Bischof, Cremers ECCV ’08

21 Daniel Cremers Variational Methods and Partial Differential Equations

Variational Scene Flow

Wedel & Cremers, “Scene Flow”, Springer 2011

22 Daniel Cremers Variational Methods and Partial Differential Equations

2 label segmentation 10 label segmentation

Variational Image Segmentation

Mumford, Shah ‘89, Chambolle, Cremers, Pock ’12:

23 Daniel Cremers Variational Methods and Partial Differential Equations

Prior Knowledge for Segmentation

24 Daniel Cremers Variational Methods and Partial Differential Equations

Cremers, Osher, Soatto, IJCV ’06

Statistical Interpretation

25 Daniel Cremers Variational Methods and Partial Differential Equations

Cremers, Osher, Soatto, IJCV ’06

Statistical Interpretation

26 Daniel Cremers Variational Methods and Partial Differential Equations

Dynamical Shape Priors

Cremers, IEEE PAMI ’06

Statistically synthesized embedding functions

27 Daniel Cremers Variational Methods and Partial Differential Equations

Cremers, IEEE PAMI ’06

Dynamical Shape Priors

28 Daniel Cremers Variational Methods and Partial Differential Equations

Dynamical Shape Priors

Cremers, IEEE PAMI ’06

29 Daniel Cremers Variational Methods and Partial Differential Equations

Cremers, IEEE PAMI ’06

Dynamical Shape Priors

Dynamical shape prior

30 Daniel Cremers Variational Methods and Partial Differential Equations

Dynamical Shape Priors

Cremers, IEEE PAMI ’06

Dynamical prior of shape and translation

31 Daniel Cremers Variational Methods and Partial Differential Equations

Variational Multiview Reconstruction

Kolev et al., IJCV ‘09

32 Daniel Cremers Variational Methods and Partial Differential Equations

Overview

variational methods

multiview reconstruction

scene flow

image segmentation

optical flow

statistical shape priors