Perceptual Quality Driven 3-D Video over...
Transcript of Perceptual Quality Driven 3-D Video over...
![Page 1: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/1.jpg)
Perceptual Quality Driven 3-D Video over Networks
C.T.E.R. Hewage
Supervisors: Dr. Stewart Worrall and Dr. Safak Dogan
![Page 2: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/2.jpg)
2
Outline Introduction
Aim and Objectives
Efficient Coding Approaches for Stereoscopic Video
Objective Quality Measures for Stereoscopic Video
Efficient Transmission Strategies for Backward Compatible Stereoscopic Video Applications
Error Concealment Techniques for 3-D Video
Conclusion
Future Works
List of Publications
![Page 3: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/3.jpg)
3
Introduction
3-D Video – Provides the sense of “being there” 3-D Video communications – Provides more natural
conditions for human interaction Tele-presence, Tele-immersion, 3D TV
NodeB
RNC
LTE
WiMAX
3DTV
Mobile 3DTV
![Page 4: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/4.jpg)
4
Stereoscopic Video
The left and right views are fused in the visual cortex of the brain to perceive the depth of a scene
•Advantages of stereoscopic video over other representations of 3-D video
• Simple representation (easy camera arrangement)
• Cost effective display systems
• Easy to adapt for existing audio-visual communication technologies
![Page 5: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/5.jpg)
5
Stereoscopic Video Capture
Left and Right View Camera Simple
Cost effective
Colour plus Depth Camera
128
255
0
Znear
Zfar(a) (b)
(a)Colour image
(b)Per-pixel depth image.
![Page 6: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/6.jpg)
6
Problem Definition
3-D Video delivery beyond conventional video requires enormous system resources such as storage and bandwidth.
Development of 3-D video technologies start from the beginning would not be a feasible and cost effective solution.
The initial development of 3-D video technologies should be backward compatible for existing 2-D video applications.
Quality evaluation of 3-D video can be done accurately using subjective evaluation tests.
The transmission aspects of 3-D video has not received much attention to date.
![Page 7: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/7.jpg)
7
Aim and Objectives Aim: To enable backward compatible 3-D
video services over bandwidth limited and unreliable communication networks using a perceptual quality driven approach.
Objectives: Explore efficient methods of encoding 3-D video using existing
method of 2-D video compression
Propose objective quality metrics for coded 3-D video in a range of compression ratios and packet loss rates.
Propose, develop and test methods of error resilience over errorprone wireless channel conditions.
Propose, develop and test error concealment methods for 3-D video data transmitted over unreliable communication networks
![Page 8: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/8.jpg)
8
Outline Introduction
Aim and Objectives
Efficient Coding Approaches for Stereoscopic Video
Objective Quality Measures for Stereoscopic Video
Efficient Transmission Strategies for Backward Compatible Stereoscopic Video Applications
Error Concealment Techniques for 3-D Video
Conclusion
Future Works
List of Publications
![Page 9: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/9.jpg)
9
Efficient Encoding Approaches for Stereoscopic Video
Exploration of effcient encoding configurations for 3-D video using existing video coding standards Compression efficiency
Compatibility with end-to-end communication technologies
Asymmetric coding of colour plus depth video Coarsely quantized depth images
Temporarly down-sampled depth images
Performance analysis of stereoscopic video transmission over IP
![Page 10: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/10.jpg)
10
Exploration of efficient encoding configurations
Parallel encoding configurations
Single-output encoding configurations
H.264/AVC
CODEC
H.264/AVC
CODECColour
Depth
Colour
Base Layer
EnhancementLayerDepth
MultiLayer
CODEC
Side by Side Image
Colour plus Depth
H.264/AVC
Encoder
![Page 11: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/11.jpg)
11
Stereoscopic video encoding with MPEG4-MAC
Global MAC Bit-
Stream
MPEG-4
MAC
EncoderColour
Depth
Shape
•A single output bit-stream•A monochrome depth map•The binary shape of the video object need to be transmitted•Backward compatibility
H.264/SVC
EncoderBase Layer
EnhancementLayer
Colour
Depth
Global SVC
Bit-stream
•A single output bit-stream•Backward compatibility•Asymmetric coding support•Possibility of layered depth images (LDI)•Inter layer prediction can be used depending on the correlation of the stereo image pair
![Page 12: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/12.jpg)
12
“Orbi” Colour (Left) and Depth (Right) video sequences
R-D Performance of SSV coding using MPEG-4 MAC, H.264/AVC and scalable H.264/AVC
![Page 13: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/13.jpg)
13
Efficient Encoding Approaches for Stereoscopic Video
Exploration of effcient encoding configurations for 3-D video using existing video coding standards Compression efficiency
Compatibility with end-to-end communication technologies
Asymmetric coding of colour plus depth video Coarsely quantized depth images
Temporarly down-sampled depth images
Performance analysis of stereoscopic video transmission over IP
![Page 14: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/14.jpg)
14
Asymmetric coding of colour plus depth stereoscopic video
Mixed-resolution encoding concept for stereoscopic video is based on the response of Human Visual System (HVS).
Asymmetric coding of colour and depth
map video??? Coarsely quantized depth maps
Temporally down-sampled depth maps
![Page 15: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/15.jpg)
15
Coarsely Quantized Depth Map Sequences
514843383328231813Mode 5
514641363126211611Mode 4
49443934292419149Mode 3
47423732272217127Mode 2
45403530252015105Mode 1
Depth
45403530252015105Colour
QPType of video
For depth image coding at the enhancement layer, five Quantization Parameter (QP) modes are used. A similar set of QP values are used to obtain R-D results for the colour image coding at the base layer.
Coarse Grain Scalability (CGS) of scalable H.264/AVC is utilized to obtain different R-D performances for colour and depth sequences
•JSVM Version 8.13
•IPPP…IPP… sequence
•I frame every 75 frames
![Page 16: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/16.jpg)
16
Image Quality and Percentage Depth Bitrate @ overall bitrate of 1Mbps
38.4436.24Depth using Mode 5
39.5337.15Depth using Mode 440.6438.08Depth using Mode 3
41.6238.77Depth using Mode 2 42.6439.25Depth using Mode 1 37.4837.01Colour
InterviewOrbiY-PSNR (dB)
Image Sequence
0.1436.8736.81Mode 50.1836.9436.87Mode 40.2436.9536.90Mode 30.3136.7436.70Mode 20.4036.3236.30Mode 1
RightLeft
Percentage depth bitrate
(%)
Image Quality: PSNR (dB)Mode
0.1737.1136.98Mode 50.2237.1937.14Mode 40.2837.0736.99Mode 30.3436.8736.86Mode 20.4136.7636.70Mode 1
RightLeft
Percentage depth bitrate
(%)
Image Quality: PSNR (dB)Mode
![Page 17: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/17.jpg)
17
Efficient Encoding Approaches for Stereoscopic Video
Exploration of effcient encoding configurations for 3-D video using existing video coding standards Compression efficiency
Compatibility with end-to-end communication technologies
Asymmetric coding of colour plus depth video Coarsely quantized depth images
Temporarly down-sampled depth images
Performance analysis of stereoscopic video transmission over IP
![Page 18: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/18.jpg)
18
Stereoscopic Video Performance Over IP
The error sensitivities of different components of immersive media may be diverse in nature.
The losses in colour image sequence may have greater impact on the perceived quality than the damage caused by a corrupted depth image sequence.
The error performance analysis can be utilized to propose novel UPA and UEP mechanisms for 3-D video communication applications.
![Page 19: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/19.jpg)
19
Stereoscopic Video Performance Over IP
Orbi Interview
![Page 20: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/20.jpg)
20
Outline Introduction
Aim and Objectives
Efficient Coding Approaches for Stereoscopic Video
Objective Quality Measures for Stereoscopic Video
Efficient Transmission Strategies for Backward Compatible Stereoscopic Video Applications
Error Concealment Techniques for 3-D Video
Conclusion
Future Works
List of Publications
![Page 21: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/21.jpg)
21
Objective Quality Measures for Stereoscopic Video
3-D video quality = {image quality, depth perception, presence, naturalness, eye strain, etc.}
Subjective evaluation tests are widely used to obtain human response for 3-D perception. Time consuming.
Enormous effort and large number of subjects are required.
Controlled test environments are required.
Does objective quality measures of 3-D video represent the true quality as perceived by human observers?
Objective quality metrics for 3-D video enable researchers, developers to evaluate the quality more efficiently.
![Page 22: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/22.jpg)
22
Objective Quality Measures for Stereoscopic Video
The 3-D video quality is measured using subjective and objective quality metrics for a range of compression rates and packet loss rates. Subjective tests are conducted according to the ITU-
Recommendation BT.1438 (using DSCQS method).• Two perceptual attributes namely, overall image quality and depth
perception are measured subjectively. PSNR, SSIM and VQM measures of colour, depth and rendered left
and right image sequences are considered as objective quality measures.
Then the relationships (correlations) between objective and subjective quality ratings are derived using a symmetrical logistic function as described in the following.
p = 1 / [1 + exp ( D - DM ) G ]
where, p is the normalized opinion score, D is the distortion parameter, DM and G are constants and G may be positive or negative.
![Page 23: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/23.jpg)
23
Quality of Asymmetrically Coded Colour Plus Depth Map Video
Orbi Breakdance
![Page 24: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/24.jpg)
24
Correlation Between Objective and Subjective Quality Measures
![Page 25: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/25.jpg)
25
Correlation Between Objective and Subjective Quality Measures
0.21730.06590.71950.55500.10540.7528Average SSIM of the Rendered Left and Right Views
0.76020.12330.01880.79100.21170.0015Depth SSIM0.21120.06500.72740.53470.10340.7618Colour SSIM
0.12150.04930.84320.27740.07450.8764Average VQM of the Rendered Left and Right
0.12240.04900.84210.24600.07010.8904Colour VQM
0.17290.05880.77680.59360.10900.7356Average PSNR of the Rendered Left and Right views
0.77500.21190.000010.77140.12420.0043Depth PSNR0.14010.05290.81920.42650.09240.8100Colour PSNR
SSERMSECCSSERMSECC
Depth PerceptionOverall Image QualityObjective Quality Model
![Page 26: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/26.jpg)
26
Perceived Quality under Transmission Errors
Orbi Breakdance
![Page 27: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/27.jpg)
27
Correlation Between Objective and Subjective Quality Measures
0.142500.08900.63100.28270.12530.5099Average SSIM of Rendered Left and Right Views
0.14740.09050.61830.29270.12750.4926Colour SSIM
0.065540.06030.83030.11280.07920.8045Average VQM of Rendered Left and Right
0.080390.06680.79180.12410.08300.7848Colour VQM
0.043100.04890.88840.11230.07890.8054Average PSNR of Rendered Left and Right views
0.044260.04960.88540.11580.08020.7992Colour PSNR
SSERMSECCSSERMSECC
Depth PerceptionOverall Image QualityObjective Quality Model
![Page 28: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/28.jpg)
28
Outline Introduction
Aim and Objectives
Efficient Coding Approaches for Stereoscopic Video
Objective Quality Measures for Stereoscopic Video
Efficient Transmission Strategies for Backward Compatible Stereoscopic Video Applications
Error Concealment Techniques for 3-D Video
Conclusion
Future Works
List of Publications
![Page 29: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/29.jpg)
29
Efficient Transmission Strategies for Backward Compatible Stereoscopic Video Applications
3-D video should be protected over unreliable and error
prone communication channels
UEP is an effective method to provide unbalanced
protection for video delivery over networks.
However adding redundant information to 3-D video
stream is not a feasible approach.
Hence in this study, UEP is implemented by
varying the transmission power for colour and depth map streams
mapping colour and depth map bit-streams into different medium
access priority classes.
![Page 30: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/30.jpg)
30
Efficient Transmission Strategies for Backward Compatible Stereoscopic Video Applications
In this study colour bit-stream is assigned higher protection compared to the depth bit-stream
Why?
![Page 31: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/31.jpg)
31
Unequal Error Protection for Backward Compatible 3-D video Transmission over WiMAX
The UPA module dynamically allocates differentiated transmissionpower for colour and depth map streams based on the generated lookup tables.
The colour bit-stream will be survived under transmission errors than the depth map stream due to the high power associated with the colour video packets.
3-D video input
WiMAX
TransmissionSystem
Decoder
Depth video Packets
Colour video Packets3-D Video
Encoder based on
SVC
Extractor Module
Power control
UPAModule
Feedback
![Page 32: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/32.jpg)
32
Performance of the UEP Scheme
8 8.5 9 9.5 10 10.5 11 11.5 12 12.5 13 13.517
19
21
23
25
27
29
31
33
35
37
39
40.5
SNR (dB)
Y−
PS
NR
(dB
)
Colour without UEPColour with UEPDepth without UEPDepth with UEP
8 9 10 11 12 1322
24
26
28
30
32
34
36
38
40
42
SNR (dB)Y
−P
SN
R (
dB)
Colour without UEPColour with UEPDepth without UEPDepth with UEP
Orbi Interview
![Page 33: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/33.jpg)
33
Performance of the UEP Scheme
Without UEP With UEP
Without UEP With UEP
![Page 34: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/34.jpg)
34
Performance of the UEP Scheme
Perception of Overall Image Quality Using SSCQS Method
0.5
1
1.5
2
2.5
3
3.5
9.95 10.5 11.05 11.6 12.15 12.7 13.25
SNR (dB)
MO
S
Overall Image Quality with UEP Overall Image Quality without UEP
![Page 35: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/35.jpg)
35
Performance of the UEP Scheme
Depth Perception using SSCQS Method
0.5
1
1.5
2
2.5
3
3.5
9.95 10.5 11.05 11.6 12.15 12.7 13.25
SNR (dB)
MO
S
Depth Perception with UEP Depth Perception without UEP
![Page 36: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/36.jpg)
36
Outline Introduction
Aim and Objectives
Efficient Coding Approaches for Stereoscopic Video
Objective Quality Measures for Stereoscopic Video
Efficient Transmission Strategies for Backward Compatible Stereoscopic Video Applications
Error Concealment Techniques for 3-D Video
Conclusion
Future Works
List of Publications
![Page 37: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/37.jpg)
37
Error Concealment Techniques for 3-D Video
3-D error concealment methods can utilize the information from the same video stream as well as from the corresponding video stream to recover the missing information. A Novel Frame Concealment Method for Depth Maps
Using Corresponding Colour Motion VectorsCompression efficiency
Error Concealment Scheme for Stereoscopic Video Using the Shared Motion Information Send by the Encoder
3-D Video Concealment Using Associated Shape Information
![Page 38: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/38.jpg)
38
Frame Concealment
128
255
0
Znear
Zfar(a) (b)
0 10 20 30 40 50 60 70 80 90 100−0.05
0
0.05
0.1
0.15
0.2
Frame Number
r(A
,B)
Interview
Horizontal componentVertical component
![Page 39: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/39.jpg)
39
Depth Frame Concealment
MVs
. . .
. . .
Depth frame n Depth frame n+1
Colour frame n Colour frame n+1
Proposed method
JSVM-FC
![Page 40: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/40.jpg)
40
GlobalSVC Bitstream
Error Concealment Scheme for Stereoscopic Video Using the Shared Motion Information
SVC Encoder
Colour MVs
Base layer Encoder
Enhancement layer encoder
Colour
Depth
. . .
. . .
MVsMVs
#1 Depth frame #2 Depth frame
#1 Colour frame #2 Colour frame
40.2741.0537.2539.69
Proposed MV
sharing scheme
42.3441.0538.1040.13Separate MVs
Depth (dB)
Colour (dB)
Depth (dB)
Colour (dB)
Interview-PSNROrbi-PSNR
Method
![Page 41: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/41.jpg)
41
Performance of the Proposed MethodOrbi
Colour Depth
![Page 42: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/42.jpg)
42
Performance of the Proposed Method
Orbi Sequence @ 5% PLR with individual MVs
Orbi Sequence @ 5% PLR with shared MVs
![Page 43: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/43.jpg)
43
Outline Introduction
Aim and Objectives
Efficient Coding Approaches for Stereoscopic Video
Objective Quality Measures for Stereoscopic Video
Efficient Transmission Strategies for Backward Compatible Stereoscopic Video Applications
Error Concealment Techniques for 3-D Video
Conclusions
Future Works
List of Publications
![Page 44: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/44.jpg)
44
Conclusions 3-D Video Coding
Compression of Colour and depth using scalable H.264/AVC Asymmetric coding of colour plus depth video Stereoscopic video performance over IP
Objective Quality Metrics for 3-D video Quality Error Resilience Schemes for 3-D Video
UEP scheme for 3-D video transmission over WiMAX networks Prioritization scheme for 3-D video distribution over WLAN
Error Concealment Schemes for 3-D Video Depth frame concealment using associated colour motion
information Frame concealment scheme for 3-D video using shared motion
information 3-D error concealment scheme based on associated shape
information
![Page 45: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/45.jpg)
45
Future Works Rate Adaptive 3D Video Coding
Asymmetric coding approached can be extended to implement open/close rate adaptive schemes for 3-D video transmission.
Generic 3-D Video Quality Metric Extend this quality evaluation to cover the end-to-end
chain of 3-D video technologies from 3-D capture to display.
Associate more perceptual attributes such as presence, naturalness and eye-strain.
The Proposed Error Resilience and Concealment Schemes Can be Extended for Multi-View Video
![Page 46: Perceptual Quality Driven 3-D Video over Networksinfo.ee.surrey.ac.uk/CCSR/Internal/Presentations/Eeriwarawe.Thusha… · • Cost effective display systems • Easy to adapt for](https://reader036.fdocuments.in/reader036/viewer/2022071007/5fc4c85262dc1b1f372c7676/html5/thumbnails/46.jpg)
46
List of Publications1. C.T.E.R. Hewage, S. Nasir, S. Worrall, S. Dogan and A.M. Kondoz, "Prioritized 3-D Video Transmission over IEEE
802.11e", submitted to 2008 IEEE International Symposium on Circuits and Systems, Taipei, Taiwan, May 2009.
2. C.T.E.R. Hewage, Z. Ahmad, S. Worrall, S. Dogan and A.M. Kondoz, "Unequal Error Protection for Backward Compatible 3-D Video Transmission over WiMAX", submitted to 2008 IEEE International Symposium on Circuits and Systems, Taipei, Taiwan, May 2009.
3. C.T.E.R. Hewage, S. Worrall, S. Dogan and A.M. Kondoz, "Prediction of stereoscopic video quality using objective quality models of 2-D video", Electronics Letters -- 31 July 2008 -- Volume 44, Issue 16, p. 963-965.
4. C.T.E.R. Hewage, S. Worrall, S. Dogan and A.M. Kondoz, "Quality Evaluation of Colour plus Depth Map Based Stereoscopic Video", submitted to IEEE Journal of Selected Topics in Signal Processing:, May 2008.
5. C.T.E.R. Hewage, S. Worrall, S. Dogan and A.M. Kondoz, "Frame Concealment Algorithm for Stereoscopic Video Using Motion Vector Sharing", Proceedings of IEEE International Conference on Multimedia & Expo 2008 (ICME2008), Hannover, Germany, June 2008, p. 485-488.
6. C.T.E.R. Hewage, S. Worrall, S. Dogan and A.M. Kondoz, "A Novel Frame Concealment Method for Depth Maps Using Corresponding Colour Motion Vectors", Proceedings of 2nd 3DTV conference (3DTV-CON'08-IEEE), Istanbul, Turkey, May 2008, p.149-152.
7. C.T.E.R. Hewage, S. Worrall, S. Dogan, H. Kodikara Arachchi and A.M. Kondoz, "Stereoscopic TV over IP", Proceedings of the 4th IET European Conference on Visual Media Production (CVMP'2007), London, UK, November 2007.
8. C.T.E.R. Hewage, H.A. Karim, S. Worrall, S. Dogan and A.M. Kondoz, "Comparison of Stereo Video Coding Support in MPEG-4 MAC, H.264/AVC and H.264/SVC", Proceedings of the 4th IET International Conference on Visual Information Engineering (VIE'2007), London, UK, 25-27 July 2007.
9. C.T.E.R. Hewage, H. Kodikara Arachchi, T. Masterton, A.C. Yu, H. Uzuner, S. Dogan and A.M. Kondoz, "Content Adaptation for Virtual Office Environment Using Scalable Video Coding", Proceedings of the 16th IST Mobile and Wireless Communications Summit (IST Summit'2007), Budapest, Hungary, 1-5 July 2007.