UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation...
Transcript of UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation...
![Page 1: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/1.jpg)
UNIQUE: Unsupervised Image Quality Estimation
D.Temel, M. Prabhushankar, and G. AlRegib
Center for Signal and Information ProcessingSchool of Electrical and Computer Engineering
Georgia Institute of TechnologyAtlanta, GA
1
![Page 2: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/2.jpg)
Outline
I. Introduction
II. Literature Review
III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing Visualization
V. Validation
VI. Conclusion
2
![Page 3: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/3.jpg)
Application Average daily shared photos
390 Million
700 Million
70 Million
760 Million
[1]
[1] Adweek, http://www.adweek.com/socialtimes/how-many-photos-are-uploaded-to-snapchat-every-second/621488, Jun 2015[2]LG, “Ultra Clarity, Ultra Scale,” http://www.lg.com/levant_en/Mini-page-ultra/index[3] PetPixel, http://petapixel.com/2015/07/08/instagram-resolution-increase-heres-how-it-affects-image-quality-and-file-size/, July 8, 2015
Smart Capturing
Remote Assistance
[2]
[3]
I. IntroductionImage Quality Assessment : Why?
3
![Page 4: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/4.jpg)
Test setup Reference images [1] Distorted images [1] Subjective Scores
Bad1 veryannoying
Poor2 annoying
Fair3 slightlyannoying
Good4 distortion but not
annoying
VeryGood5 no
perceiveddistortion
[1] Sheikh, H.R., Wang, Z., Cormack, L. and Bovik, A.C., "LIVE Image Quality Assessment Database Release 2", http://live.ece.utexas.edu/research/quality.
Mean opinion scores
I. IntroductionImage Quality Assessment : In Practice
4
![Page 5: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/5.jpg)
Outline
I. Introduction
II. Literature Review
III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing Visualization
V. Validation
VI. Conclusion
5
![Page 6: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/6.jpg)
II. Literature Review Data-driven Image Quality Estimators
YEAR 2011 2012 2013 2014 2015 2016
QUALITYESTIMATORS LB
IQ
DIIV
INE
CO
RN
IA
BRIS
QU
E
MLI
QM
CB/
SF
QAC
SPAR
Q
Tang
QAF
Kang
IQA-
CN
N++
Li
DLI
QA
Gao
CN
N-S
VR
UN
IQU
E
Visual system
Color
Do not require
Distortion specific data in the training
Labels in the training
Handcrafting features
6
![Page 7: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/7.jpg)
Outline
I. Introduction
II. Literature Review
III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing Visualization
V. Validation
VI. Conclusion
7
![Page 8: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/8.jpg)
D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” the IEEE Signal Processing Letters, vol.23, no.10, pp.1414-1418.
III. UNIQUE: Unsupervised Image Quality EstimationOverview of UNIQUE
8
![Page 9: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/9.jpg)
D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” the IEEE Signal Processing Letters, vol.23, no.10, pp.1414-1418.
III. UNIQUE: Unsupervised Image Quality EstimationPreprocessing
9
![Page 10: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/10.jpg)
D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” the IEEE Signal Processing Letters, vol.23, no.10, pp.1414-1418.
III. UNIQUE: Unsupervised Image Quality EstimationUnsupervised Learning Mechanism
10
![Page 11: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/11.jpg)
11
III. UNIQUE: Unsupervised Image Quality EstimationVisualization of Learnt Filters
D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” the IEEE Signal Processing Letters, vol.23, no.10, pp.1414-1418.
![Page 12: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/12.jpg)
D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” the IEEE Signal Processing Letters, vol.23, no.10, pp.1414-1418.
III. UNIQUE: Unsupervised Image Quality EstimationVisualization of Processed Images
12
![Page 13: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/13.jpg)
Outline
I. Introduction
II. Literature Review
III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing Visualization
V. Validation
VI. Conclusion
13
![Page 14: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/14.jpg)
LIVE MULTI TID Total
Compression 460 180 375 1015
Image Noise 174 180 1375 1729
Communication 174 250 424
Blur 174 315 250 739
Color - 375 375
Global - 250 250
Local - 250 250
LIVE database
TID 2013 database
𝐸𝐸[(𝑋𝑋 − 𝜇𝜇𝑋𝑋)(𝑌𝑌 − 𝜇𝜇𝑌𝑌)]𝜎𝜎𝑋𝑋𝜎𝜎𝑌𝑌 1 −
6∑𝑖𝑖=1𝑁𝑁 𝑥𝑥𝑖𝑖 − 𝑦𝑦𝑖𝑖 2
𝑁𝑁(𝑁𝑁2 − 1)
𝑋𝑋𝑖𝑖 ,𝑌𝑌𝑖𝑖 𝑥𝑥𝑖𝑖 ,𝑦𝑦𝑖𝑖𝐸𝐸 𝑋𝑋 − 𝑌𝑌 2
Pearson Linear Correlation Coefficient (PLCC)
Linearity
Root mean square error (RMSE)
Accuracy
Spearman Rank Correlation Coefficient (SRCC)
RankingPerformanceMetrics
Databases
𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑖𝑖𝑜𝑜𝑜𝑜𝑜𝑜𝑁𝑁𝑜𝑜𝑜𝑜𝑜𝑜𝑡𝑡𝑜𝑜
Outlier Ratio (OR)
Consistency
V. Image Quality Estimators• Validation
14
![Page 15: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/15.jpg)
15
V. Image Quality Estimators• Validation
![Page 16: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/16.jpg)
16
V. Image Quality Estimators• Validation
![Page 17: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/17.jpg)
Outline
I. Introduction
II. Literature Review
III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing Visualization
V. Validation
VI. Conclusion
17
![Page 18: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/18.jpg)
YEAR 2011 2012 2013 2014 2015 2016
QUALITYESTIMATORS LB
IQD
IIVIN
EC
OR
NIA
BRIS
QU
EM
LIQ
MC
B/SF
QAC
SPAR
QTa
ngQ
AFKa
ng
IQA-
CN
N++
Li
DLI
QA
Gao
CN
N-S
VRU
NIQ
UE
Visual systemColor
Do not require
Distortion specific data in the trainingLabels in the training
Handcrafting
VI. ConclusionContributions and Observations
To measure perceived quality Hand-crafting is not sufficient, we should also learn from the data.
Labels are not easy to find, we need to focus more on unsupervised approaches.
Color perception must be included in a comprehensive visual system model.
The best example is our visual system, we should model it as much as we can.18
![Page 19: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/19.jpg)
19
Thank You!
![Page 20: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/20.jpg)
20
Backup Slides
![Page 21: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/21.jpg)
81 121 169 400 625
IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUEMulti-model
Varying the number of neurons to learn global and local features
21
![Page 22: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/22.jpg)
Sharpness Threshold
IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUESharpness-weighted Multi-model
22
![Page 23: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/23.jpg)
IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUEVisualization
Images UNIQUE MS-UNIQUE
23
![Page 24: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/24.jpg)
V. Image Quality Estimators
0.60
0.65
0.70
0.75
0.806.55
7.05
7.55
8.05
8.55
0.55
0.75
0.95
1.15
TID13
OR
RM
SER
MSE
LIVE
TID13
• Validation
24
![Page 25: UNIQUE: Unsupervised Image Quality Estimation 2017...UNIQUE: Unsupervised Image Quality Estimation D.Temel, M. Prabhushankar, and G. AlRegib Center for Signal and Information Processing](https://reader030.fdocuments.in/reader030/viewer/2022040821/5e6ab097e8e9b974437ca6bd/html5/thumbnails/25.jpg)
V. Image Quality Estimators
0.90
0.92
0.94
0.96
SRC
C
LIVE
0.70
0.75
0.80
0.85
SRC
C
TID13
0.87
0.89
0.91
0.93
0.95
PLC
C
LIVE
0.70
0.75
0.80
0.85
PLC
C
TID13
• Validation
25