Laboratorio di Problemi InversiEsercitazione 3: regolarizzazione iterativa, metodo di
Landweber
Luca Calatroni
Dipartimento di Matematica, Universita degli studi di Genova
May 18, 2016.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 1 / 15
Outline
1 Recap on the theory
2 Landweber regularisation implementation
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 2 / 15
Surprisingly: the inverse problem
We are always considering the following linear, ill-posed, denoising/deblurringinverse problem:
y = Bx + n
and try to solve it in a clever way (i.e. avoiding the ill-posedness).
Recap
So far, we have seen some standard methods to solve it (both from the point ofview of minimisation/spectral filtering approaches):
I TSVD: truncate the bad components;
I Tikhonov: regularise the bad components depending on λ (choice oflambda: generalised cross-validation,seen!).
. . . then, invert.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 3 / 15
Iterative regularization: idea
Another approach consists in solving the least square problem:
minx
1
2‖Bx − y‖2.
Taking the gradient we have that in correspondence of the minimum x there holds:
BT (Bx − y) = 0
We can now either use fixed-point iterations or used gradient descent methods towrite down a time-stepping:
Landweber iterationsFor every k compute x by iterating
xk+1 = xk − τBT (Bxk − y)
for an appropriate choice of τ .
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 4 / 15
Iterative regularization: idea
Another approach consists in solving the least square problem:
minx
1
2‖Bx − y‖2.
Taking the gradient we have that in correspondence of the minimum x there holds:
BT (Bx − y) = 0
We can now either use fixed-point iterations or used gradient descent methods towrite down a time-stepping:
Landweber iterationsFor every k compute x by iterating
xk+1 = xk − τBT (Bxk − y)
for an appropriate choice of τ .
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 4 / 15
Outline
1 Recap on the theory
2 Landweber regularisation implementation
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 5 / 15
Work plan
We want to implement the Landweber iterative regularisationtechnique to solve the deblurring/denoising problem: easy!
Furthermore, we want to understand:
I its convergence properties, also when noise is in the image;
I its approximation properties;
I its computational limits.
Objectives
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 6 / 15
Let’s start to. . .
I As usual, load the Shepp-Logan phantom;
I Create Gaussian PSF, with σ2 = 5;
I Use Fourier to blur the images (recall: fft2, ifft2, fftshift).
I Assign to sigma the level of Gaussian noise you want to add (take it small)and add noise to the blurred image.
I Show the result.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 7 / 15
Let’s start to. . .
I As usual, load the Shepp-Logan phantom;
I Create Gaussian PSF, with σ2 = 5;
I Use Fourier to blur the images (recall: fft2, ifft2, fftshift).
I Assign to sigma the level of Gaussian noise you want to add (take it small)and add noise to the blurred image.
I Show the result.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 7 / 15
Let’s start to. . .
I As usual, load the Shepp-Logan phantom;
I Create Gaussian PSF, with σ2 = 5;
I Use Fourier to blur the images (recall: fft2, ifft2, fftshift).
I Assign to sigma the level of Gaussian noise you want to add (take it small)and add noise to the blurred image.
I Show the result.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 7 / 15
Let’s start to. . .
I As usual, load the Shepp-Logan phantom;
I Create Gaussian PSF, with σ2 = 5;
I Use Fourier to blur the images (recall: fft2, ifft2, fftshift).
I Assign to sigma the level of Gaussian noise you want to add (take it small)and add noise to the blurred image.
I Show the result.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 7 / 15
Let’s start to. . .
I As usual, load the Shepp-Logan phantom;
I Create Gaussian PSF, with σ2 = 5;
I Use Fourier to blur the images (recall: fft2, ifft2, fftshift).
I Assign to sigma the level of Gaussian noise you want to add (take it small)and add noise to the blurred image.
I Show the result.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 7 / 15
How to choose the Landweber parameter?
Without optimising, the rule of thumbs corresponds to choose:
0 < τ <2
|F(PSF )|2=
2
‖B‖2
To Do’s:
I set-up τ appropriately;
I set-up max it as the maximum number of iterations (use 2000).
OK, now let us create a MATLAB function having as input whatever is needed +the original image (not realistic!) to compute residual and as output thedenoised/deblurred image + the vector of residuals.
[deb,res]=landweber deb(...,phantom)
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 8 / 15
How to choose the Landweber parameter?
Without optimising, the rule of thumbs corresponds to choose:
0 < τ <2
|F(PSF )|2=
2
‖B‖2
To Do’s:
I set-up τ appropriately;
I set-up max it as the maximum number of iterations (use 2000).
OK, now let us create a MATLAB function having as input whatever is needed +the original image (not realistic!) to compute residual and as output thedenoised/deblurred image + the vector of residuals.
[deb,res]=landweber deb(...,phantom)
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 8 / 15
How to do it? Essential steps
I Initialise: vector of relative error and array of the iterates.
I Iteratively (till maximum of iterations) apply Landweber.
Recall
How did we compute Bx using the PSF? What about BT z?
I Use: fftshift, fft2, ifft2. To get BT from B using the PSF, flipupside-down and left-to-right the PSF.
commands: fliplr, flipud
I Write the Landweber iteration and show the result every 50 iterations, storingalso the relative error:
err(k) =‖xk − xor‖2
F
‖xor‖2F
I Compute the deblurred/denoised solution as the one having minimum relativeerror → output.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 9 / 15
How to do it? Essential steps
I Initialise: vector of relative error and array of the iterates.
I Iteratively (till maximum of iterations) apply Landweber.
Recall
How did we compute Bx using the PSF? What about BT z?
I Use: fftshift, fft2, ifft2. To get BT from B using the PSF, flipupside-down and left-to-right the PSF.
commands: fliplr, flipud
I Write the Landweber iteration and show the result every 50 iterations, storingalso the relative error:
err(k) =‖xk − xor‖2
F
‖xor‖2F
I Compute the deblurred/denoised solution as the one having minimum relativeerror → output.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 9 / 15
How to do it? Essential steps
I Initialise: vector of relative error and array of the iterates.
I Iteratively (till maximum of iterations) apply Landweber.
Recall
How did we compute Bx using the PSF? What about BT z?
I Use: fftshift, fft2, ifft2. To get BT from B using the PSF, flipupside-down and left-to-right the PSF.
commands: fliplr, flipud
I Write the Landweber iteration and show the result every 50 iterations, storingalso the relative error:
err(k) =‖xk − xor‖2
F
‖xor‖2F
I Compute the deblurred/denoised solution as the one having minimum relativeerror → output.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 9 / 15
Remark
Matrices VS. vectorsNote that even though we are thinking at the problem from a “matrix - vector ”perspective, we are still using matrices. . . the properties of the linear operatorsinvolved are the one we are exploiting to solve the problem efficiently!
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 10 / 15
Comments on the results
1 Show the denoised/deblurred result when a small choice of the noise level ismade. What do you observe? How fast is the method?
2 What if the noise level is larger? What do you observe on the error plot?How does this reflect on iterates?
Semi-convergence
There is a k∗ after which error amplifies and noise oscillations become more andmore visible.
3 Use scatter to find the minimum of the error plot.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 11 / 15
Comments on the results
1 Show the denoised/deblurred result when a small choice of the noise level ismade. What do you observe? How fast is the method?
2 What if the noise level is larger? What do you observe on the error plot?How does this reflect on iterates?
Semi-convergence
There is a k∗ after which error amplifies and noise oscillations become more andmore visible.
3 Use scatter to find the minimum of the error plot.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 11 / 15
Comments on the results
1 Show the denoised/deblurred result when a small choice of the noise level ismade. What do you observe? How fast is the method?
2 What if the noise level is larger? What do you observe on the error plot?How does this reflect on iterates?
Semi-convergence
There is a k∗ after which error amplifies and noise oscillations become more andmore visible.
3 Use scatter to find the minimum of the error plot.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 11 / 15
Comments on the results
1 Show the denoised/deblurred result when a small choice of the noise level ismade. What do you observe? How fast is the method?
2 What if the noise level is larger? What do you observe on the error plot?How does this reflect on iterates?
Semi-convergence
There is a k∗ after which error amplifies and noise oscillations become more andmore visible.
3 Use scatter to find the minimum of the error plot.
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 11 / 15
Part II: approximating property of Landweber
No inversion!By construction, the Landweber method does not require any inversion step alongthe iterates: that’s good!
We can re write:
xk+1 = (I − τBTB)xk + τBT y = . . . = (I − τBTB)k+1x0 + τ
k∑i=0
(I − τBTB)iBT y
Now, if ‖I − τBTB‖ ≤ 1 the first term goes to zero as k →∞.We can then study the behaviour of the second term by looking at the polynomial:
Sk(t) = τk∑
i=0
(1− τ t)i =1− (1− τ t)k+1
t
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 12 / 15
No inversion!
The following convergence property holds true:
Sk(t)→ 1
tif |1− λt| < 1, for k →∞.
The inversion step is represented by the ratio 1/t which would imply forLandweber iterates:
xk → x = (BTB)−1BT y ,
but such inversion is avoided through the analytical identity above, provided theterms in the series satisfy
|1− τ t| < 1
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 13 / 15
Regularisation properties
1 Define the appropriate interval such that:
0 < |1− τ t| < 1 (1)
(suggestion: take the first extreme to be a bit bigger than 0)
2 Define a MATLAB handle computing the partial sums
Sk(t) =1− (1− τ t)k+1
t
depending on the parameter t and k.
3 Show on the same figure, the plots of Sk for different values of k ↗ andcompare with the graph of 1/t (representing the inversion we want to avoid).What do you observe?
4 Now increase the interval in (1) and do the same. What do you observe? Isthe approximating property holding for every t ∈ R?
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 14 / 15
Regularisation properties
1 Define the appropriate interval such that:
0 < |1− τ t| < 1 (1)
(suggestion: take the first extreme to be a bit bigger than 0)
2 Define a MATLAB handle computing the partial sums
Sk(t) =1− (1− τ t)k+1
t
depending on the parameter t and k .
3 Show on the same figure, the plots of Sk for different values of k ↗ andcompare with the graph of 1/t (representing the inversion we want to avoid).What do you observe?
4 Now increase the interval in (1) and do the same. What do you observe? Isthe approximating property holding for every t ∈ R?
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 14 / 15
Regularisation properties
1 Define the appropriate interval such that:
0 < |1− τ t| < 1 (1)
(suggestion: take the first extreme to be a bit bigger than 0)
2 Define a MATLAB handle computing the partial sums
Sk(t) =1− (1− τ t)k+1
t
depending on the parameter t and k .
3 Show on the same figure, the plots of Sk for different values of k ↗ andcompare with the graph of 1/t (representing the inversion we want to avoid).What do you observe?
4 Now increase the interval in (1) and do the same. What do you observe? Isthe approximating property holding for every t ∈ R?
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 14 / 15
End of Part I: deterministic approaches
See you tomorrow with
Figure: Thomas Bayes
Luca Calatroni (DIMA, Unige) Esercitazione 3, Lab. Prob. Inv. May 18, 2016 15 / 15
Top Related