CIME1

7/29/2019 CIME1

1/59

Martin Burger

Institut fr Numerische und Angewandte MathematikEuropean Institute for Molecular ImagingCeNoS

Total Variation andRelated Methods

7/29/2019 CIME1

2/59

Martin Burger

Total Variation 2

Cetraro, September 2008

Mathematical Imaging@WWU

Christoph Brune Alex Sawatzky Frank Wbbeling Thomas Ksters Martin

Benning

Brbel Schlake Marzena Franek Christina Stcker Mary Wolfram Thomas Grosser Jahn

Mller

7/29/2019 CIME1

3/59

Martin Burger

Total Variation 3


Imaging Basics: Whatand Why ?

- Denoising: Given a noisy version of an image, find a smooth

approximation (better appealing to the human eye or better

suited for some task, e.g. counting cells in a microscope)

- Decomposition: Given an image, decompose it into different

parts such as smooth structure, texture, edges, noise

- Deblurring: Given a blurred version of an image (also noisy)

find an approximation of the original image

- Inpainting: Given an image with holes, try to fill the holes as

a reasonable continuation

7/29/2019 CIME1

4/59

Martin Burger

Total Variation 4


Imaging Basics: Whatand Why ?- Segmentation: Find the edges / different objects in an image

- Reconstruction: Given indirect information about an image,

e.g. tomography, try to find an approximation of the image

Many of these tasks can be categorized as Inverse Problems:

reconstruction of the cause of an observed effect (via a

mathematical model relating them)

Diagnosis in medicine is a prototypical example"The grand thing is to be able to reason backwards."

Arthur Conan Doyle (A study in scarlet)

7/29/2019 CIME1

5/59

Martin Burger

Total Variation 5


Noisy ImagesNoise appears from measurement devices or transmission

loss

7/29/2019 CIME1

6/59

Martin Burger

Total Variation 6


Damaged ImagesCorrupted Pixels, dusted, scratches

7/29/2019 CIME1

7/59Martin Burger

Total Variation 7


Medical Imaging: CTClassical image reconstruction example:

computerized tomography (CT)

Mathematical Problem:

Reconstruction of a density

function from its line integrals

Inversion of the Radon transformcf. Natterer 86, Natterer-Wbbeling 02

7/29/2019 CIME1

8/59Martin Burger

Total Variation 8


Medical Imaging: CTClassical image reconstruction example:

computerized tomography (CT)

7/29/2019 CIME1

9/59Martin Burger

Total Variation 9


Medical Imaging: CT+ Low noise level

+ High spatial resolution

+ Exact reconstruction

+ Reasonable Costs

- Restricted to few seconds

(radiation exposure, 20 mSiewert)- No functional information

- Few mathematical challenges left

Soret, Bacharach, Buvat 07

CTCT

Schfers et al 07

7/29/2019 CIME1

10/59Martin Burger

Total Variation 10


Medical Imaging: MR+ Low noise level

+ High spatial resolution

+ Reconstruction by Fourier inversion

+ No radiation exposure

+ Good contrast in soft matter

- Low tracer sensitivity- Limited functional information

- Expensive

- Few mathematical challenges left Courtesy Carsten Wolters,University Hospital Mnster

7/29/2019 CIME1

11/59Martin Burger

Total Variation 11


Medical Imaging: Ultrasound+ Fast and cheap

+ Varying spatial resolution

+ Usually no reconstruction

+ No radiation exposure

- High noise level

- Bad contrast / bones

7/29/2019 CIME1

12/59Martin Burger

Total Variation 12


Imaging Examples: PET (Human / Small

animal)Positron-Emission-Tomography

Data: detecting decay events of an radioactive tracer

Decay events are random, but their rate is proportional to the

tracer uptake (Radon transform with random directions)

Imaging of molecular properties

7/29/2019 CIME1

13/59Martin Burger

Total Variation 13


Medical Imaging: PET+ High sensitivity

+ Long time (mins ~ 1 hour,

radiation exposure 8-12 mSiewert)

+ Functional information

+ Many open mathematical questions

- Few anatomical information

- High noise level and disturbing

effects(damping, scattering, )

- Low spatial resolution

Soret, Bacharach, Buvat 07

Schfers et al 07

PET

PET

7/29/2019 CIME1

14/59Martin Burger

Total Variation 14


Small Animal PET: Burning down the

Mouse

7/29/2019 CIME1

15/59Martin Burger

Total Variation 15


Image reconstruction in PETStochastic models needed: typically measurements drawn from

Poisson model

Image uequals density function (uptake) of tracer

Linear OperatorK equals Radon-transform

Possibly additional (Gaussian) measurement noise b

7/29/2019 CIME1

16/59

Martin Burger

Total Variation 16


Cameras and MicroscopesSame model with different K can be used for imaging withphotons (microscopy, CCD cameras, ..)

Typically the Poisson statistic is good (many photon counts),

measurement noise dominates

In some cases the

opposite is true !

7/29/2019 CIME1

17/59

Martin Burger

Total Variation 17


Low SNR

Bad statistics arising due to

lower

radioactive activity or isotopesdecaying fast (e.g. H2O

15)

Desireable forpatients

Desireable for certain

quantitative investigations (H2O15

is useful tracer for blood flow)

~10.000

Events

~600

Events

7/29/2019 CIME1

18/59

Martin Burger

Total Variation 18


Basic ParadigmsTypical imaging tasks are s0lved by a compromise between the

following two goals:

- Stay close to the data

- Among those close to the data choose the one that

corresponds best to a-priori ideas / knowledge

The measure of how close one wants to stay to data is the

SNR, respectively noise level. For zero noise / infinite SNR onewould

reproduce the data exactly. The higher the noise level / lower

the SNR the farther the solution can be from the data.

7/29/2019 CIME1

19/59

Martin Burger

Total Variation 19


Imaging modelsContinuum Discrete

Image:

Data:

Relation by (sometimes nonlinear Operator)

7/29/2019 CIME1

20/59

Martin Burger

Total Variation 20


Imaging modelsWe usually use an abstract treatment with an image space Xand a data space Y

Digital (discrete) model is nowadays the realistic one, however

there are several reasons to interpret it as a discretization of

an underlying continuum model:

- Images come with different resolution, should be

compareable

- Rich mathematical models in the continuum PDEs

- Robustness of numerical methods

7/29/2019 CIME1

21/59

Martin Burger

Total Variation 21


Relation between Image and DataDenoising:

Decomposition

:

7/29/2019 CIME1

22/59

Martin Burger

Total Variation 22


Relation between Image and DataDeblurring:

Inpainting: inpainting region

7/29/2019 CIME1

23/59

Martin Burger

Total Variation 23


Relation between Image and DataSegmentation:

Reconstruction:

7/29/2019 CIME1

24/59

Martin Burger

Total Variation 24


Bayes ParadigmThe two goals are translated into probabilities:

- Conditional data probability

- A-priori probability of an image in absence of data

7/29/2019 CIME1

25/59

Martin Burger

Total Variation 25


Bayes Paradigm and MAPTogether they create the a-posterior probability of an image

A-priori probability of data is a scaling factor and can be

ignoredA natural estimator is the one maximizing probability, the

maximum-aposteriori-probability (MAP) estimator

7/29/2019 CIME1

26/59

Martin Burger

Total Variation 26


MAP EstimatorMAP estimator can be computed by minimizing negative log-

likelihood:

A-priori probability can be related to a regularization term

7/29/2019 CIME1

27/59

Martin Burger

Total Variation 27


The log-likelihoodThe probability to observe data f if the exact image is u can berelated to the distribution of the noise

Example: additive Gaussian noise (pointwise)

7/29/2019 CIME1

28/59

Martin Burger

Total Variation 28


The log-likelihoodThe log-likelihood becomes a sum, which converges to an

integral in the continuum limit

7/29/2019 CIME1

29/59

Martin Burger

Total Variation 29


Variational modelThe above reasoning yields directly a standard variational

model

The MAP estimator is determined from minimizing the

functional

7/29/2019 CIME1

30/59

Martin Burger

Total Variation 30


Variational model IIOne can show that the above minimization is equivalent to

7/29/2019 CIME1

31/59

Martin Burger

Total Variation 31


Discrepancy principleThe second formulation is a (generalized) discrepancy principle

for Gaussian noise:

Minimize the regularization (maximize a-priori probability)

among

all images that give a data discrepancy of the order of the

variance

Alternatively this can be interpreted as a rule of choosing

Choose such that

7/29/2019 CIME1

32/59

Martin Burger

Total Variation 32


Image Space and RegularizationThe image space and a-priori probability are directly related,

X consists of all images with positive probability or,

equivalently,

finite regularization functional

What is the right choice ofR ?

7/29/2019 CIME1

33/59

Martin Burger

Total Variation 33


Image Space and RegularizationHow can we get reasonable regularization terms ?

Dependent on goals and expectations on the solution

Typical expectation: smoothness, in particular few oscillations

(high oscillations = noise, to be eliminated)

Few oscillations means small gradient variance, i.e.

7/29/2019 CIME1

34/59

Martin Burger

Total Variation 34


Denoising ExampleConsider MAP estimate for Gaussian noise with above

regularization

Unconstrained optimization, simple optimality condition

7/29/2019 CIME1

35/59

Martin Burger

Total Variation 35


Reminder: Gateaux-DerivativeGateaux derivative of a functional is the collection of all

directional derivatives

7/29/2019 CIME1

36/59

Martin Burger

Total Variation 36


Optimality ConditionCompute Gateaux-derivative

Optimality:

Weak form of:

7/29/2019 CIME1

37/59

Martin Burger

Total Variation 37


Elliptic RegularityWe were looking for a function in

7/29/2019 CIME1

38/59

Martin Burger

Total Variation 38


Elliptic RegularityRegularity theory for the Poisson equation implies

Hence uhas even second derivatives and may beoversmoothed

Note: derivatives go to infinity for

7/29/2019 CIME1

39/59

Martin Burger

Total Variation 39


Scale Space and Inverse Scale SpaceThe square root of the Lagrange parameter defines a scale

Hence uvaries at a scale of order

Smaller scales in fare suppressed

7/29/2019 CIME1

40/59

Martin Burger

Total Variation 40


Scale Space and Inverse Scale SpaceMultiple scales by iterating the variational method for small

Scale Space Methods (Diffusion Filters):

Start with noisy image (finest scales) and gradually coarsenscales until a certain minimal scale is reached

Inverse Scale Space Methods (Bregman Iterations):

Start with the most rough information about the image (largest

scale = whole image, i.e. start with mean value) and gradually

refine scales until a certain minimal scale is reached

7/29/2019 CIME1

41/59

Martin Burger

Total Variation 41


Variational MethodsVariational method can be interpreted as both a

- Scale space method:

- Inverse scale space method:

7/29/2019 CIME1

42/59

Martin Burger

Total Variation 42


Scale Space Methods: Diffusion filtersAlternative construction of a scale space method:

Reinterpret optimality condition

7/29/2019 CIME1

43/59

Martin Burger

Total Variation 43


Scale Space MethodsIterate the variational method using the previous result as data

7/29/2019 CIME1

44/59

Martin Burger

Total Variation 44


Scale Space MethodStart with

Evolve uby

Denoised result:

7/29/2019 CIME1

45/59

Martin Burger

Total Variation 45


Inverse Scale Space MethodAlternative by starting with coarsest scale

7/29/2019 CIME1

46/59

Martin Burger

Total Variation 46


Inverse Scale Space MethodOpposite limit (oversmoothing) yields flow

Denoised result:

Tdepends on noise level

7/29/2019 CIME1

47/59

Martin Burger

Total Variation 47


Variational MethodsAlternative smoothing via penalizing coefficients in orthonormal

bases

7/29/2019 CIME1

48/59

Martin Burger

Total Variation 48


Fourier SeriesRewrite functional

Equivalent minimization

7/29/2019 CIME1

49/59

Martin Burger

Total Variation 49


Fourier SeriesExplicit solution of the minimization problem

7/29/2019 CIME1

50/59

Martin Burger

Total Variation 50


Other Orthonormal Bases / WaveletsAnalogous approach for other orthonormal bases, minimization

of coefficients

7/29/2019 CIME1

51/59

Martin Burger

Total Variation 51


Problems with Quadratic RegularizationQuadratic regularization yields simple linear equations to solve,

but has several disadvantages

- Oversmoothing (see above)

- Edges are destroyed

- Bias from operatorA (see later)

Alternative: other functions of the gradient

7/29/2019 CIME1

52/59

Martin Burger

Total Variation 52


Nonquadratic RegularizationOptimality condition

Linearization for smooth and strictly convex G

7/29/2019 CIME1

53/59

Martin Burger

Total Variation 53


Total VariationOnly way to penalize oscillations without full elliptic regularity is

to choose Gnot smooth / not strictly convex

Canonical choice

7/29/2019 CIME1

54/59

Martin Burger

Total Variation 54


Minimization ProblemThe minimization problem

has no solution in general (more later)

Problem needs to be defined on a larger space

7/29/2019 CIME1

55/59

Martin Burger

Total Variation 55


Total VariationRigorous definition

7/29/2019 CIME1

56/59

Martin Burger

Total Variation 56


Why TV-Methods ?Cartooning

Linear Filter TV-Method

7/29/2019 CIME1

57/59

Martin Burger

Total Variation 57


Why TV-Methods ?Cartooning

ROF Model with increasing allowed

variance

7/29/2019 CIME1

58/59

Martin Burger

Total Variation 58


SparsityAnalogous approach in orthormal basis by penalization with

weighted 1-norm

7/29/2019 CIME1

59/59

Total Variation 59

SparsityMost coefficients will be zero (sparse solution), the shrinkage of

coefficients is a data-dependent

Total variation leads to sparsity in the gradient, hence gradientwill be zero in most points (usually in the others there are

edges)

CIME1

Documents

Transcript of CIME1