Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data...
-
Upload
aliza-steggall -
Category
Documents
-
view
217 -
download
0
Transcript of Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data...
![Page 1: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/1.jpg)
Some Blind Deconvolution Techniques in Image Processing
Tony Chan
Math Dept., UCLA
Astronomical Data Analysis Software & SystemsConference Series 2004
Pasadena, CA, October 24-27, 2004
Joint work with Frederick Park and
Andy M. Yip
![Page 2: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/2.jpg)
2
OutlinePart I:
Total Variation Blind Deconvolution
Part II:Simultaneous TV Image Inpainting and
Blind Deconvolution
Part III:Automatic Parameter Selection for TV
Blind Deconvolution
Caution: Our work not developed specifically for Astronomical images
![Page 3: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/3.jpg)
3
Blind Deconvolution Problem
= +
Observed image
Unknown true image
Unknown point spread function
Unknown noise
Goal: Given uobs, recover both uorig and k
obsu origu k
![Page 4: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/4.jpg)
4
Typical PSFsPSFs w/ sharp edges:
PSFs w/ smooth transitions
![Page 5: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/5.jpg)
5
Total Variation Regularization
dxxuuTV )()(
• Deconvolution ill-posed: need regularization
• Total variation Regularization:
dxxkkTV )()(
• The characteristic function of D with height h (jump):
• TV = Length(∂D)h• TV doesn’t penalize jumps• Co-area Formula:
D
h drdsfdxufnR
ru
)(||}{
![Page 6: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/6.jpg)
6
TV Blind Deconvolution Model
TVTVobsku
kuukukuF 21
2
,),(min
),(),(,1),(,0, yxkyxkdxdyyxkku Subject to:
Objective:
(C. and Wong (IEEE TIP, 1998))
1 determined by signal-to-noise ratio
2 parameterizes a family of solutions, corresponds to different spread of the reconstructed PSF
• Alternating Minimization Algorithm:
• Globally convergent with H1 regularization.
),(min),(
),(min ),(
111
1
kuFkuF
kuFkuF
n
k
nn
n
u
nn
![Page 7: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/7.jpg)
7
Blind v.s. non-Blind Deconvolution
Observed Image noise-free
• An out-of-focus blur is recovered automatically
• Recovered blind deconvolution images almost as good as non-blind
• Edges well-recovered in image and PSF
non-Blind
Recovered Image PSF
Blind
1 = 2106, 2 = 1.5105
Clean image
True PSF
![Page 8: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/8.jpg)
8
Blind v.s. non-Blind Deconvolution: High Noise
Observed Image SNR=5 dB
non-Blind
Clean image
True PSF
Blind
• An out-of-focus blur is recovered automatically
• Even in the presence of high noise level, recovered images from blind deconvolution are almost as good as those recovered with the exact PSF
1 = 2105, 2 = 1.5105
![Page 9: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/9.jpg)
9
Controlling Focal-Length Recovered Images are 1-parameter family w.r.t. 2
Recovered Blurring Functions(1 = 2106)
0 1107 1105 11042:
The parameter 2 controls the focal-length
![Page 10: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/10.jpg)
10
Generalizations to Multi-Channel Images
• Inter-Channel Blur Model– Color image (Katsaggelos et al, SPIE 1994):
Bobs
Gobs
Robs
B
G
R
kkk
kkk
kkk
u
u
u
u
u
u
HHH
HHH
HHH
noise
122
212
221
75
71
71
71
75
71
71
71
75
obsk unoiseuH
k1: within channel blur
k2: between channel blur
m-channel TV-norm (Color-TV)
(C. & Blomgren, IEEE TIP ‘98)
2
2
1
2)(
m
iTVim
kkTV
m
iTVim uuTV
1
2)(
![Page 11: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/11.jpg)
11
Original image
Out-of-focus blurred blind non-blind
Gaussian blurred blind non-blind
Examples of Multi-Channel Blind Deconvolution(C. and Wong (SPIE, 1997))
• Blind is as good as non-blind
• Gaussian blur is harder to recover (zero-crossings in frequency domain)
![Page 12: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/12.jpg)
12
TV Blind Deconvolution Patented!
![Page 13: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/13.jpg)
13
Outline
Part I:Total Variation Blind Deconvolution
Part II:Simultaneous TV Image Inpainting and
Blind Deconvolution
Part III:Automatic Parameter Selection for TV
Blind Deconvolution
![Page 14: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/14.jpg)
14
TV Inpainting Model(C. & Shen SIAP 2001)
EDE
dxdyuudxdyuuJ ,||2
||][ 20
,0)(||
0
uuu
ue
.0; ,,DzEz
e
Graffiti Removal
Scratch Removal
![Page 15: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/15.jpg)
15
Images Degraded by Blurring and Missing Regions
• Blur– Calibration errors of devices– Atmospheric turbulence– Motion of objects/camera
• Missing regions– Scratches– Occlusion– Defects in films/sensors
+
![Page 16: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/16.jpg)
16
Problems with Inpaint then Deblur
• Inpaint first reduce plausible solutions
• Should pick the solution using more information
Original Signal Blurring func.
Original Signal Blurring func.
Blurred Signal
Blurred Signal
=
=
Blurred + Occluded
Blurred + Occluded
=
![Page 17: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/17.jpg)
17
Problems with Deblur then Inpaint
• Different BC’s correspond to different image intensities in inpaint region.
• Most local BC’s do not respect global geometric structures
Original Occluded Support of PSF
Dirichlet Neumann Inpainting
![Page 18: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/18.jpg)
18
The Joint Model
• Do --- the region where the image is observed
• Di --- the region to be inpainted
• A natural combination of TV deblur + TV inpaint
• No BC’s needed for inpaint regions
• 2 parameters (can incorporate automatic parameter selection techniques)
Inpainting take place
Coupling of inpainting & deblur
![Page 19: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/19.jpg)
19
Simulation Results (1)Degraded Restored
Zoom-in
• The vertical strip is completed
• Not completed
• Use higher order inpainting methods
• E.g. Euler’s elastica, curvature driven diffusion
![Page 20: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/20.jpg)
20
Simulation Results (2)
Observed Restored
Deblur then inpaint
(many artifacts)
Inpaint then deblur
(many ringings)
Original
![Page 21: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/21.jpg)
21
Boundary Conditions for Regular Deblurring
Original image domain and
artificial boundary outside the scene
Dirichlet B.C.
Periodic B.C.
Neumann B.C.
Inpainting B.C.
![Page 22: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/22.jpg)
22
![Page 23: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/23.jpg)
23
Outline
Part I:Total Variation Blind Deconvolution
Part II:Simultaneous TV Image Inpainting and
Blind Deconvolution
Part III:Automatic Parameter Selection for TV
Blind Deconvolution (Ongoing Research)
![Page 24: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/24.jpg)
24
Automatic Blind Deblurring (ongoing research)
• Recovered images: 1-parameter family wrt 2
• Consider external info like sharpness to choose optimal 2
Problem: Find 2 automatically to recover best u & k
SNR = 15 dB20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
Clean image
observed image
![Page 25: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/25.jpg)
25
Motivation for Sharpness & Support
• Sharpest image has large gradients• Preference for gradients with small support
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
u
|| u
|| u
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
Support of
![Page 26: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/26.jpg)
26
Proposed Sharpness Evaluator
• F(u) small => sharp image with small support• F(u)=0 for piecewise constant images• F(u) penalizes smeared edges
||ofsupport of Area)( uuF
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
u
|| u20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
Support of
![Page 27: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/27.jpg)
27
Planets Example
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
Rel. errors in u (blue) and k (red) v.s. 2
Proposed Objective v.s. 2
Optimal Restored Image Auto-focused Image
The minimum of the sharpness function agrees with that of the
rel. errors of u and k
(minimizer of sharpness func.)
(minimizer of rel. error in u)
1=0.02 (optimal)
105
106
107
0
0.5
1
1.5
105
106
107
1500
2000
2500
3000
3500
4000
4500
5000
![Page 28: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/28.jpg)
28
Satellite ExampleRel. errors in u (blue) and
k (red) v.s. 2
Proposed Objective v.s. 2
Optimal Restored Image Auto-focused Image
The minimum of the sharpness function agrees with that of the
rel. errors of u and k
(minimizer of sharpness func.)
(minimizer of rel. error in u)
1=0.3 (optimal)
105
106
107
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
20 40 60 80 100 120
20
40
60
80
100
120
105
106
107
2000
3000
4000
5000
6000
7000
8000
9000
10000
11000
20 40 60 80 100 120
20
40
60
80
100
120
![Page 29: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/29.jpg)
29
Potential Applications to Astronomical Imaging
• TV Blind Deconvolution– TV/Sharp edges useful?– Auto-focus: appropriate objective function?– How to incorporate a priori domain knowledge?
• TV Blind Deconvolution + Inpainting– Other noise models: e.g. salt-and-pepper noise
![Page 30: Some Blind Deconvolution Techniques in Image Processing Tony Chan Math Dept., UCLA Astronomical Data Analysis Software & Systems Conference Series 2004.](https://reader035.fdocuments.in/reader035/viewer/2022062318/551999f85503463d068b4943/html5/thumbnails/30.jpg)
30
References
1. C. and C. K. Wong, Total Variation Blind Deconvolution, IEEE Transactions on Image Processing, 7(3):370-375, 1998.
2. C. and C. K. Wong, Multichannel Image Deconvolution by Total Variation Regularization, Proc. to the SPIE Symposium on Advanced Signal Processing: Algorithms, Architectures, and Implementations, vol. 3162, San Diego, CA, July 1997, Ed.: F. Luk.
3. C. and C. K. Wong, Convergence of the Alternating Minimization Algorithm for Blind Deconvolution, UCLA Mathematics Department CAM Report 99-19.
4. R. H. Chan, C. and C. K. Wong, Cosine Transform Based Preconditioners for Total Variation Deblurring, IEEE Trans. Image Proc., 8 (1999), pp. 1472-1478
5. C., A. Yip and F. Park, Simultaneous Total Variation Image Inpainting and Blind Deconvolution, UCLA Mathematics Department CAM Report 04-45.