Robust Super-Resolution Presented By: Sina Farsiu.
-
date post
20-Dec-2015 -
Category
Documents
-
view
223 -
download
6
Transcript of Robust Super-Resolution Presented By: Sina Farsiu.
![Page 1: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/1.jpg)
Robust Super-Resolution
Presented By:Sina Farsiu
![Page 2: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/2.jpg)
Project Goals
Understanding & simulation of Dr. Assaf Zomet, et.al. paper : “Robust Super-Resolution” [1]
Comparing the results obtained by this method to other methods
Enhancing the method introduced in: [1]
![Page 3: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/3.jpg)
Super-Resolution Objective
Generation of a high-resolution image from multiple low-resolution moving frames of a scene.
![Page 4: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/4.jpg)
Super-Resolution Formulation
nHXb
pk1 kkkkk nXFCDb
pppp n
n
X
FCD
FCD
b
b
11111
![Page 5: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/5.jpg)
Solutions for S-R Problem
No Noise:
With Noise
bHX 1
, min2
2ILLxHXb
X
bHIHHX TT 1)(
![Page 6: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/6.jpg)
Problem with These Solutions
In the presence of outliers (error in motion estimation, inaccurate blur model, pepper & salt noise, …), these methods do not work accurately. Robust S-R methods can help in these situations.
![Page 7: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/7.jpg)
Robust S-R Formulation
pk1 kkkkk nXFCDb
2
1)(
2
21
p
kkkkk XFCDbXL
)( kkkkTk
Tk
Tkk bXFCDDCFB
![Page 8: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/8.jpg)
Robust S-R Formulation
Robustness
p
kkBXL
1
)(
)(*1 XLXX nn
p1kk y)}(x,median{By)L(X)(x,
987654321 B B B B B B B B B median
![Page 9: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/9.jpg)
Why Median?
Median is an estimate of mean.Unlike regularization method only one of low resolution frames contributes to reconstruct each pixel in the high-resolution frame. So outliers in other frames are discarded in the reconstruction process.Claim: In the absence of additive noise to all frames median method works better than regularization method.
![Page 10: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/10.jpg)
What if noise is added to all frames?
Claim: If considerable amount of additive noise is present in all frames regularization method works as good or even better than median method.
![Page 11: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/11.jpg)
Median-Average Reconstruction
Instead of using
We can combine average and median operators to get better results.
p1kk y)}(x,median{By)L(X)(x,
1
987654321 B B B B B B B B B average
![Page 12: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/12.jpg)
Bias Detection Procedure
It is useful to detect the outliers in the low resolution frames.We can omit those outlier pixels in our procedures. The difficulty is to differentiate between aliasing and outlier effects.
![Page 13: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/13.jpg)
Bias Detection
Formulation
After thresholding non zero values are due to aliasing or outliers.
jkkkk
jk XFCDye
![Page 14: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/14.jpg)
Bias Detection Result
Due to outlier Due to aliasing
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
0 0 0 1 1-
0 1- 1 1- 1
0 1 1- 0 0
0 1- 1 0 0
![Page 15: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/15.jpg)
Bias detection Procedure
1: compute 2: Threshold3: Filter the result with a LPF4: Threshold5: Omit corresponding pixels from super-resolution procedure
jkkkk
jk XFCDye
![Page 16: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/16.jpg)
Bias Detection
B-D method works only for uniform gray level outliers.In many situations median operation in robust super-resolution eliminates the bias of the estimator.
![Page 17: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/17.jpg)
Original L-R Frame
![Page 18: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/18.jpg)
Robust S-R Reconstructed H-R Frame mse=0.0017
![Page 19: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/19.jpg)
Median Reconstruction after adding noise 0.0131
![Page 20: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/20.jpg)
Regularized S-R Reconstructed H-R Frame mse=0.0131`
![Page 21: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/21.jpg)
Regularized Reconstruction after adding noise mse=.0125
![Page 22: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/22.jpg)
Error Due to Outlier
![Page 23: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/23.jpg)
Error Due to Aliasing
![Page 24: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/24.jpg)
Conclusion
Robust super-resolution method is quite effective in the presence of outliers, and produces better results in comparison with regularization method. In the presence of additive noise in all low –resolution frames this method loses its superiority to the regularization method.
![Page 25: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/25.jpg)
Conclusion & Results
Combination of mean and median operators can help us in this situation.Proposed bias detection algorithm is an effective method to detect outliers. If outliers are the only source of error in the L-R frames(no additive noise), more iterations we use smaller mse we will achieve.
![Page 26: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/26.jpg)
Suggestions for Future Research
Combining regularization and robust super-resolution methods.Using bias detection results in regularization method.Using robust super-resolution method in frequency domain.
![Page 27: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/27.jpg)
Acknowledgment
Thanks to Dr.Assaf Zomet, Dr.Michael Elad, Dirk Robinson and Dr. Peyman Milanfar for their valuable advices & suggestions.
![Page 28: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/28.jpg)
references
"Robust Super Resolution", A. Zomet, A. Rav-Acha and S. Peleg Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, December 2001."Efficient Super-Resolution and Applications to Mosaics", A. Zomet and S. Peleg, Proceedings of the International Conference on Pattern Recognition (ICPR), Barcelona, September 2000.
![Page 29: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/29.jpg)
“A Computationally effective Image super-resolution Algorithm”, Nguyen, N., P. Milanfar, G.H. Golub, IEEE Transactions on Image Processing, vol. 10, no. 4, pp. 573-583, April 2001 “A Fast Super-Resolution Reconstruction Algorithm for Pure Transnational Motion and Common Space Invariant Blur”, M. Elad and Y. Hel-Or, the IEEE Trans. on Image Processing, Vol.10, no.8, pp.1187-93, August 2001.
![Page 30: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/30.jpg)
Thank You All
![Page 31: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/31.jpg)
Additional Simulatins
![Page 32: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/32.jpg)
Original H-R Frame
![Page 33: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/33.jpg)
Blured median
![Page 34: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/34.jpg)
Projected L-R frame
![Page 35: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/35.jpg)
![Page 36: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/36.jpg)
![Page 37: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/37.jpg)
![Page 38: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/38.jpg)
![Page 39: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/39.jpg)
L-R Frame
![Page 40: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/40.jpg)
Regularization with high noisemse=.0481
![Page 41: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/41.jpg)
Median with high noisemse=.0693
![Page 42: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/42.jpg)
Composite Median & AverageMSE=0.0592
![Page 43: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/43.jpg)
Original H.R. Frame
![Page 44: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/44.jpg)
L.R. Frame Before Adding Noise
![Page 45: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/45.jpg)
L.R. Frame After Adding Noise
![Page 46: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/46.jpg)
Regularized ReconstructionMSE=.0216
![Page 47: Robust Super-Resolution Presented By: Sina Farsiu.](https://reader035.fdocuments.in/reader035/viewer/2022062421/56649d415503460f94a1b4e8/html5/thumbnails/47.jpg)
Median ReconstructionMSE=.0077