A new methodology to estimate the impact of H.264 artefacts on subjective video quality
-
Upload
stephane-pechard -
Category
Technology
-
view
795 -
download
1
description
Transcript of A new methodology to estimate the impact of H.264 artefacts on subjective video quality
![Page 1: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/1.jpg)
A new methodology to estimate the impact of H.264 artefacts on
subjective video quality
Stéphane Péchard, Patrick Le Callet, Mathieu Carnec, Dominique Barba
Université de Nantes – IRCCyN laboratory – IVC teamPolytech’Nantes, rue Christian Pauc, 44306 Nantes, France
Third International Workshop on Video Processing and Quality Metrics for Consumer ElectronicsScottsdale, 2007-01-26
![Page 2: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/2.jpg)
2
Introduction
• Codec => Coding artefacts
• Quality loss due to artefacts
=> Useful for quality metrics or better coding …
• Possible practical approach– Artefact classification– Annoyance or quality loss contribution per artefact type
![Page 3: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/3.jpg)
3
Farias and al. methodology Farias VPQM 05
• Artefacts type set (blockiness, blur, ringing, …)• Generation of synthetic artefacts
– Strength parameter– Applied with the same strength on a whole part of the
sequence• Subjective assessment => Annoyance curve per artefact type regarding the
strength => Content dependency
Alternative approach : VPQM07⇒ H.264 coding, Subjective assessment : quality scale, blur
scale, blockiness scale …⇒ No direct control of artefacts strength
![Page 4: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/4.jpg)
4
Proposed approach
• H.264 artefacts due to quantization/decision • effects are different regarding the local content
(edge, texture, …)• different perceived annoyance depending on the
local spatio-temporal activity of the content
• H264 distortions only in selected coherent spatio-temporal regions => define content categories
• Subjective quality assessment⇒ Quality loss curve per local content category
(e.g. effects of H264 on each category)
⇒ Strength ?
![Page 5: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/5.jpg)
5
Outline
• Spatio temporal segmentation • distorted sequences generation• subjective quality assessment of sequences• Quality assessment : Combining categories• Towards quality loss function per content
category
![Page 6: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/6.jpg)
6
The approach
temporalsegmentation classification
H.264 codingC-distortedsequencesgeneration
unlabeledholes filling
bordersprocessing
source
categories masks sequence
partly-distorted sequencesusable for subjective tests
Ci
……
![Page 7: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/7.jpg)
7
Spatio temporal classification
2 steps
- temporal segmentation :reliability regarding the motion => temporal tubes
- tube classification :Regarding spatial content
![Page 8: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/8.jpg)
8
Segmentation of sequences
temporalsegmentation classification
H.264 codingClass-distorted
sequencesgeneration
unlabeledholes filling
bordersprocessing
source
partly-distorted sequencesusable for subjective tests
Ci
……
![Page 9: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/9.jpg)
9
Segmentation of sequences
• per group of five successive frame, the center frame is divided into blocks
• motion estimation of each block using the two previous frames and the two next frames
(motion estimation performed on a multi-resolution representation)
i+1 i+2i-1 ii-2
![Page 10: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/10.jpg)
10
Segmentation of sequences
• temporal tracking of each block of frame i defines a spatio-temporal “tube” over the five frames
• a tube is oriented along the local motion
i+1 i+2i-1 ii-2
![Page 11: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/11.jpg)
11
Classification
temporalsegmentation classification
H.264 codingC-distortedsequencesgeneration
unlabeledholes filling
bordersprocessing
source
partly-distorted sequencesusable for subjective tests
Ci
……
![Page 12: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/12.jpg)
12
Definition of content categories
• low luminance smooth areas;• high luminance smooth areas;• fine textured areas;• edges;• strong textured areas
HVS has different perception of impairments depending on the local spatio-temporal content.
![Page 13: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/13.jpg)
13
Classification
• 4 spatial gradients means per tube (directions : 0, 90, 45 and 135°)
• plot in spatial space P (0 and 90°) => C1, C2, C3 and C4
• 2nd step : space P’ (45 and 135°) used to discriminate C5 in P
• frontier determined to obtain relevant classification
![Page 14: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/14.jpg)
14
Classification
• global tracking of moving objects over the whole sequence
• tubes are classified then merged by categories
smooth areas with low luminancesmooth areas with high luminancefine textured areasedgesstrong textured areas
![Page 15: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/15.jpg)
15
Unlabeled holes filling and tube intersections
temporalsegmentation classification
H.264 codingC-distortedsequencesgeneration
unlabeledholes filling
bordersprocessing
source
partly-distorted sequencesusable for subjective tests
Ci
……
![Page 16: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/16.jpg)
16
Unlabeled holes filling and tube intersection
• every pixel of the source has one and only one label• unlabeled holes :
– gradient value => class– closest tube
• Insection pixels : same
![Page 17: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/17.jpg)
17
Borders processing
temporalsegmentation classification
H.264 codingC-distortedsequencesgeneration
unlabeledholes filling
bordersprocessing
source
partly-distorted sequencesusable for subjective tests
Ci
……
![Page 18: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/18.jpg)
18
Borders processing
• borders between original and distorted large regions are treated so as to smooth the transitions
beforeborders
processing
afterbordersprocessing
![Page 19: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/19.jpg)
19
H.264 coding and class-distorted sequences generation
temporalsegmentation classification
H.264 codingC-distortedsequencesgeneration
unlabeledholes filling
bordersprocessing
source
category masks sequence
partly-distorted sequencesusable for subjective tests
Ci
……
![Page 20: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/20.jpg)
20
Partly-distorted sequences generation
H.264 sequences at different bitrates
categories sequence
original sequence C1
C2
C3
C4
C5
5 sequences per bitrate
![Page 21: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/21.jpg)
21
Original sequence (first frame)
![Page 22: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/22.jpg)
22
One caregory distorted sequence (first frame)
![Page 23: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/23.jpg)
23
Subjective quality assessment
• SAMVIQ protocol with at least 15 validated observers and normalized conditions
• 1920x1080 HDTV Philips LCD display
• Doremi V1-UHD 1080i HDTV player
![Page 24: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/24.jpg)
24
Subjective quality assessment
• 11 sequences in a SAMVIQ session:– 5 Ci-only distorted at a certain bitrate B
– entirely distorted sequence at B– entirely distorted sequence at low bitrate– entirely distorted sequence at intermediate
bitrate– explicit and hidden references
![Page 25: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/25.jpg)
25
Sequences uncompressed HDTV sequences from SVT
Above marathon Captain Dance in the woods Duck fly
C5 50 % C2 78 % C3 54 % C5 60 %
Fountain man Group disorder Rendezvous Ulriksdals
C2 71 % C2+C3+C1 95 % C5 56 % C2+C3 80 %
![Page 26: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/26.jpg)
26
example on sequence Ulriksdalscoded at 1 Mbps
0
10
20
30
40
50
60
70
80
90
1 2 3 4 5
Classes MOS(Sj,Bk) MOSref
![Page 27: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/27.jpg)
27
• ∆MOS(Ci, Sj ,Bk) = MOSref - MOS(Ci, Sj ,Bk) is the quality loss induced by distortions in category Ci
∆MOS(C4)
∆MOS(C5)
∆MOS(C3)
∆MOS(C1)
∆MOS(C2)
MOSref
MOS(Sj,Bk)
DMOS(Sj,Bk)
MOS(C4)
MOS(C5)
MOS(C3)MOS(C1)
MOS(C2)
• MOS(Ci, Sj ,Bk) for each sequence Sj, each category Ci at each bitrate Bk
• DMOS(Sj ,Bk) = MOSref – MOS(Sj ,Bk) is the quality difference between the reference and the entirely distorted sequence
DMOS and ∆MOS
![Page 28: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/28.jpg)
28
• relations use sums of ∆MOS
0.5472∆MOS(C4)
0.7664∆MOS(C2)
0.7094∆MOS(C3)
0.6400∆MOS(C5)
……
0.5349∆MOS(C1)
0.9058∆MOS(C1) + ∆MOS(C2) + ∆MOS(C3)
+ ∆MOS(C4) + ∆MOS(C5)
0.9094∆MOS(C2) + ∆MOS(C3) + ∆MOS(C4)0.9440∆MOS(C2) + ∆MOS(C5)0.9485∆MOS(C2)+ ∆MOS(C4) + ∆MOS(C5)
CCCombination
Possible relation between global DMOS and category ∆MOS?
![Page 29: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/29.jpg)
29
Non linear functions
• DMOSp = maxi(∆MOSi)
– CC = 0.9467• DMOSp = maxi(∆MOSi) + maxj(∆MOSj) with j≠i
– CC = 0.9530
• Correlation exists between global DMOS and category ∆MOS=> DMOS could be predicted from quality per
category
![Page 30: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/30.jpg)
30
Towards a quality loss model
• How to control the distortion level of a given class ?– Farias approach :strength of synthetic artefact
• Factors implied in the quality loss of category Ci:– distortions themselves– motion– proportion of the category– spatial localisation (not considered here)
![Page 31: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/31.jpg)
31
Distortion strength for category C1
• distortion strength = f(M,P,E)With all along the sequence :– M the mean motion of the category;– P the mean proportion of the category;– E the MSE on the category;
• M decreases the distortion strength while P and E increase DSproposed model for f
DS = (1 — M/Mt)×P×E
![Page 32: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/32.jpg)
32
Quality loss function for category C1
• Psychometic function as a prediction of ∆MOS1
φ(DS) = (a×DSb)/(c+DSb)
• correlation between φ(DS) and ∆MOS1 : 0.9514
• RMSE = 5.25• good predictor of the loss of quality induced by
category C1
![Page 33: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/33.jpg)
33
Quality loss function for class C1
=> Possible prediction of ∆MOS1
![Page 34: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/34.jpg)
34
Conclusion
• design of a new methodology to estimate the impact of H.264 artefacts on subjective video quality
• One distortion type but– Effect related to local content– possibility to relate the global loss to loss per
category– quality loss function for category C1
• Other categories and objective models
![Page 35: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/35.jpg)
35
Results: segmentation statistics
60.7016.750.3650.06C5 (%)
10.703.021.430.94C4 (%)
19.5053.856.8127.79C3 (%)
8.9722.5778.2617.45C2 (%)
0.133.8013.143.75C1 (%)
Duck flyDance in the woodsCaptainAbove marathon
Séquence
![Page 36: A new methodology to estimate the impact of H.264 artefacts on subjective video quality](https://reader034.fdocuments.in/reader034/viewer/2022052507/555840ced8b42acb078b4fc2/html5/thumbnails/36.jpg)
36
Results: segmentation statistics
3.3056.924.543.93C5 (%)
1.362.051.791.45C4 (%)
40.4819.8729.8013.37C3 (%)
41.3112.3838.5870.71C2 (%)
13.548.7825.2810.52C1 (%)
UlriksdalsRendezvousGroup disorderFountain man
Séquence