Laboratorio di Problemi Inversi Esercitazione 3...

Laboratorio di Problemi InversiEsercitazione 3: regolarizzazione iterativa, metodo di

Landweber

Luca Calatroni

Dipartimento di Matematica, Universita degli studi di Genova

May 18, 2016.

Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 1 / 15

Outline

1 Recap on the theory

2 Landweber regularisation implementation

Surprisingly: the inverse problem

We are always considering the following linear, ill-posed, denoising/deblurringinverse problem:

y = Bx + n

and try to solve it in a clever way (i.e. avoiding the ill-posedness).

So far, we have seen some standard methods to solve it (both from the point ofview of minimisation/spectral filtering approaches):

I TSVD: truncate the bad components;

I Tikhonov: regularise the bad components depending on λ (choice oflambda: generalised cross-validation,seen!).

. . . then, invert.

Iterative regularization: idea

Another approach consists in solving the least square problem:

2‖Bx − y‖2.

Taking the gradient we have that in correspondence of the minimum x there holds:

BT (Bx − y) = 0

We can now either use fixed-point iterations or used gradient descent methods towrite down a time-stepping:

Landweber iterationsFor every k compute x by iterating

xk+1 = xk − τBT (Bxk − y)

for an appropriate choice of τ .

Iterative regularization: idea

Another approach consists in solving the least square problem:

2‖Bx − y‖2.

Taking the gradient we have that in correspondence of the minimum x there holds:

BT (Bx − y) = 0

We can now either use fixed-point iterations or used gradient descent methods towrite down a time-stepping:

Landweber iterationsFor every k compute x by iterating

xk+1 = xk − τBT (Bxk − y)

for an appropriate choice of τ .

Outline

1 Recap on the theory

2 Landweber regularisation implementation

Work plan

We want to implement the Landweber iterative regularisationtechnique to solve the deblurring/denoising problem: easy!

Furthermore, we want to understand:

I its convergence properties, also when noise is in the image;

I its approximation properties;

I its computational limits.

Objectives

Let’s start to. . .

I As usual, load the Shepp-Logan phantom;

I Create Gaussian PSF, with σ2 = 5;

I Use Fourier to blur the images (recall: fft2, ifft2, fftshift).

I Assign to sigma the level of Gaussian noise you want to add (take it small)and add noise to the blurred image.

I Show the result.

How to choose the Landweber parameter?

Without optimising, the rule of thumbs corresponds to choose:

0 < τ <2

|F(PSF )|2=

‖B‖2

To Do’s:

I set-up τ appropriately;

I set-up max it as the maximum number of iterations (use 2000).

OK, now let us create a MATLAB function having as input whatever is needed +the original image (not realistic!) to compute residual and as output thedenoised/deblurred image + the vector of residuals.

[deb,res]=landweber deb(...,phantom)

How to choose the Landweber parameter?

Without optimising, the rule of thumbs corresponds to choose:

0 < τ <2

|F(PSF )|2=

‖B‖2

To Do’s:

I set-up τ appropriately;

I set-up max it as the maximum number of iterations (use 2000).

OK, now let us create a MATLAB function having as input whatever is needed +the original image (not realistic!) to compute residual and as output thedenoised/deblurred image + the vector of residuals.

[deb,res]=landweber deb(...,phantom)

How to do it? Essential steps

I Initialise: vector of relative error and array of the iterates.

I Iteratively (till maximum of iterations) apply Landweber.

Recall

How did we compute Bx using the PSF? What about BT z?

I Use: fftshift, fft2, ifft2. To get BT from B using the PSF, flipupside-down and left-to-right the PSF.

commands: fliplr, flipud

I Write the Landweber iteration and show the result every 50 iterations, storingalso the relative error:

err(k) =‖xk − xor‖2

‖xor‖2F

I Compute the deblurred/denoised solution as the one having minimum relativeerror → output.

Recall

‖xor‖2F

Recall

‖xor‖2F

Remark

Matrices VS. vectorsNote that even though we are thinking at the problem from a “matrix - vector ”perspective, we are still using matrices. . . the properties of the linear operatorsinvolved are the one we are exploiting to solve the problem efficiently!

Comments on the results

1 Show the denoised/deblurred result when a small choice of the noise level ismade. What do you observe? How fast is the method?

2 What if the noise level is larger? What do you observe on the error plot?How does this reflect on iterates?

Semi-convergence

There is a k∗ after which error amplifies and noise oscillations become more andmore visible.

3 Use scatter to find the minimum of the error plot.

Semi-convergence

Part II: approximating property of Landweber

No inversion!By construction, the Landweber method does not require any inversion step alongthe iterates: that’s good!

We can re write:

xk+1 = (I − τBTB)xk + τBT y = . . . = (I − τBTB)k+1x0 + τ

k∑i=0

(I − τBTB)iBT y

Now, if ‖I − τBTB‖ ≤ 1 the first term goes to zero as k →∞.We can then study the behaviour of the second term by looking at the polynomial:

Sk(t) = τk∑

(1− τ t)i =1− (1− τ t)k+1

No inversion!

The following convergence property holds true:

Sk(t)→ 1

tif |1− λt| < 1, for k →∞.

The inversion step is represented by the ratio 1/t which would imply forLandweber iterates:

xk → x = (BTB)−1BT y ,

but such inversion is avoided through the analytical identity above, provided theterms in the series satisfy

|1− τ t| < 1

Regularisation properties

1 Define the appropriate interval such that:

0 < |1− τ t| < 1 (1)

(suggestion: take the first extreme to be a bit bigger than 0)

2 Define a MATLAB handle computing the partial sums

Sk(t) =1− (1− τ t)k+1

depending on the parameter t and k.

3 Show on the same figure, the plots of Sk for different values of k ↗ andcompare with the graph of 1/t (representing the inversion we want to avoid).What do you observe?

4 Now increase the interval in (1) and do the same. What do you observe? Isthe approximating property holding for every t ∈ R?

0 < |1− τ t| < 1 (1)

Sk(t) =1− (1− τ t)k+1

depending on the parameter t and k .

0 < |1− τ t| < 1 (1)

Sk(t) =1− (1− τ t)k+1

depending on the parameter t and k .

End of Part I: deterministic approaches

See you tomorrow with

Figure: Thomas Bayes

Laboratorio di Problemi Inversi Esercitazione 3...

Documents

Transcript of Laboratorio di Problemi Inversi Esercitazione 3...