Post on 11-Jan-2016
description
1
NUS.SOC.CS5248Ooi Wei Tsang
Error Recovery
NUS.SOC.CS5248Ooi Wei Tsang
2
Problems with Internet
Limited BandwidthVarying ConditionsDelay JitterPacket LossDelayHeterogeneity :
NUS.SOC.CS5248Ooi Wei Tsang
3
Network
Encoder
Sender
Middlebox
Receiver
Decoder
You Are Here
NUS.SOC.CS5248Ooi Wei Tsang
4
Overview
Characteristics of the InternetGeneral techniquesError recovery for audioEffect of loss on MPEGError recovery for MPEG
5
NUS.SOC.CS5248Ooi Wei Tsang
Loss Characteristics of The Internet
NUS.SOC.CS5248Ooi Wei Tsang
6
Characteristics of Internet
60-70% of paths do not show any loss
Those with loss have an average of 4.5 – 6% packet loss
[Paxson97] End-to-end Internet packet dynamics
NUS.SOC.CS5248Ooi Wei Tsang
7
Packet Loss Pattern
1000
30
Number of Occurrences
Burst Length
NUS.SOC.CS5248Ooi Wei Tsang
8
Characteristics of Internet
Bursts of loss are typically short (2-3 consecutively loss packets)
Long burst do occur
Burst may occur periodically
NUS.SOC.CS5248Ooi Wei Tsang
9
Wireless Link
Loss rate measured in my office 10 ~ 20%
up to 50% reported!
NUS.SOC.CS5248Ooi Wei Tsang
10
Gilbert Model
GOOD BAD
11
NUS.SOC.CS5248Ooi Wei Tsang
A Survey of Packet-Loss Recovery Techniques for Streaming Audio
Perkins, Hodson and HardmanIEEE Network Magazine 1998
NUS.SOC.CS5248Ooi Wei Tsang
12
Retransmission
1 2 3 4
1 2 4
3
3
X
NUS.SOC.CS5248Ooi Wei Tsang
13
Redundant Data
1 2 3 4
1 2 4
2
2 3
3 4 5
53
NUS.SOC.CS5248Ooi Wei Tsang
14
2 4
Error Concealment
1 2 3 4
1
15
NUS.SOC.CS5248Ooi Wei Tsang
Retransmission
for audio
NUS.SOC.CS5248Ooi Wei Tsang
16
Pros/Cons of Retransmissions
NUS.SOC.CS5248Ooi Wei Tsang
17
Scalable Retransmission
On packet loss T = random(0, RTT) wait for T multicast NACK
On receiving NACK from others suppress own NACK
NUS.SOC.CS5248Ooi Wei Tsang
18
Retransmit when
Group size is small Loss rate is lowLarge latency acceptable
NUS.SOC.CS5248Ooi Wei Tsang
19
Selective Retransmission
Retransmission based on priority of packets
Important/urgent packets are retransmitted first
20
NUS.SOC.CS5248Ooi Wei Tsang
Redundant Data
for audio
NUS.SOC.CS5248Ooi Wei Tsang
21
Parity Forward Error Correction
1011001
1000010
0001001
XOR 0010010
NUS.SOC.CS5248Ooi Wei Tsang
22
Parity Forward Error Correction
1011001
1000010
0001001
0010010
NUS.SOC.CS5248Ooi Wei Tsang
23
Parity FEC Ordering
NUS.SOC.CS5248Ooi Wei Tsang
24
Parity FEC Ordering
NUS.SOC.CS5248Ooi Wei Tsang
25
Parity FEC Ordering
NUS.SOC.CS5248Ooi Wei Tsang
26
Parity FEC Ordering
NUS.SOC.CS5248Ooi Wei Tsang
27
Parity FEC Ordering
NUS.SOC.CS5248Ooi Wei Tsang
28
Reed-Solomon Code
RS(n,k)
RS
n k
NUS.SOC.CS5248Ooi Wei Tsang
29
Reed-Solomon Code
NUS.SOC.CS5248Ooi Wei Tsang
30
Media Specific FEC
1 2 3 4
1 2 4
2
2 3
3 4 5
53
NUS.SOC.CS5248Ooi Wei Tsang
31
Pros/Cons of Redundant Data
NUS.SOC.CS5248Ooi Wei Tsang
32
Interleaving
NUS.SOC.CS5248Ooi Wei Tsang
33
Pros/Cons of Interleaving
34
NUS.SOC.CS5248Ooi Wei Tsang
Error Concealment
NUS.SOC.CS5248Ooi Wei Tsang
35
Effect of Loss on Audio
Speech Human ears can interpolate Loss up to length of phoneme can
still be tolerable
NUS.SOC.CS5248Ooi Wei Tsang
36
Insertion-based Repair
SpliceSilence SubstitutionNoise SubstitutionRepetition
NUS.SOC.CS5248Ooi Wei Tsang
37
Other Repair Methods
InterpolationRegeneration
38
NUS.SOC.CS5248Ooi Wei Tsang
Colin’s Recommendations
NUS.SOC.CS5248Ooi Wei Tsang
39
Non-Interactive Apps
InterleavingFECRetransmission for unicast only
NUS.SOC.CS5248Ooi Wei Tsang
40
Interactive Applications
Media Specific FEC
NUS.SOC.CS5248Ooi Wei Tsang
41
Error Concealment
Repeat
42
NUS.SOC.CS5248Ooi Wei Tsang
Packet Loss Effects on MPEG Video Sent over the Public Internet
Jill Boyce and Robert GaglianelloACM Multimedia 1998
NUS.SOC.CS5248Ooi Wei Tsang
43
Measurement-based Study
Need to understand the problem before proposing solution
NUS.SOC.CS5248Ooi Wei Tsang
44
Data Gathering Method
From: NYC13 , Austin21, London18
To: Holmdel, NJ
NUS.SOC.CS5248Ooi Wei Tsang
45
Data Gathering Method
Video: Two 5-mins MPEG 30 fps 384 kbps and 1 Mbps IBBPBBPBBPBBPBB QSIF 176x112 and SIF 352x240 One row per slice
NUS.SOC.CS5248Ooi Wei Tsang
46
Average Packet Loss
0
2
4
6
8
10
12
14
Packet Loss Rate
(%)
New York Texas London
1Mbps384kbps
NUS.SOC.CS5248Ooi Wei Tsang
47
Frame Effected by Errors
% Frames in Error
Packet Loss Rate
48
NUS.SOC.CS5248Ooi Wei Tsang
Overview of Error Recovery for Video
NUS.SOC.CS5248Ooi Wei Tsang
49
Methods
Retransmission InterleavingError ConcealmentFECLimiting Error Propagation
Reference Frame Selection
NUS.SOC.CS5248Ooi Wei Tsang
50
Reference Frame Selection
I P B B P
X
I
NUS.SOC.CS5248Ooi Wei Tsang
51
Methods
Retransmission InterleavingError ConcealmentFECLimiting Error Propagation
Reference Frame Selection Changing Temporal Pattern
52
NUS.SOC.CS5248Ooi Wei Tsang
Error Control Techniques for Interactive Low Bitrate Video Transmission over The Internet
Injong RheeSIGCOMM ‘98
NUS.SOC.CS5248Ooi Wei Tsang
53
Basic Idea
“Better Late Than Never!”, orLate packet is still useful
NUS.SOC.CS5248Ooi Wei Tsang
54
MPEG Frame Pattern
I B B P B B P
I P B B P B B
NUS.SOC.CS5248Ooi Wei Tsang
55
H.261 Error Propagation
I P P P P P PX X X X X X
I P P P P P PX X X X
retransmission
loss
NUS.SOC.CS5248Ooi Wei Tsang
56
H.261 Frame Pattern
I P P P P P PX
loss
NUS.SOC.CS5248Ooi Wei Tsang
57
I P P P P P P
H.261 Frame Pattern
X
loss
X X
retransmission
NUS.SOC.CS5248Ooi Wei Tsang
58
PTDD
Periodic Temporal Dependency Distance
Large PTDD Higher Chance of Repair Longer Error Propagations Less Temporal Redundancy
NUS.SOC.CS5248Ooi Wei Tsang
59
QAL
I P P P
I P P P Base Layer
EnhancementLayer
NUS.SOC.CS5248Ooi Wei Tsang
60
QAL
I P P P
I P P P Base Layer + FEC
EnhancementLayer
X
NUS.SOC.CS5248Ooi Wei Tsang
61
QAL + PTDD
I P P P
I P P P Base Layer
EnhancementLayer
NUS.SOC.CS5248Ooi Wei Tsang
62
Error Propagation
NUS.SOC.CS5248Ooi Wei Tsang
63
Recovery from Error Propagation
NUS.SOC.CS5248Ooi Wei Tsang
64
Frame “quality”
PSNR =
NUS.SOC.CS5248Ooi Wei Tsang
65
PSNR vs. Loss Rate
Packet Loss Rate
PSNR
66
NUS.SOC.CS5248Ooi Wei Tsang
Today’s Summary
NUS.SOC.CS5248Ooi Wei Tsang
67
How to recover packet loss
RetransmissionFECError Concealment
NUS.SOC.CS5248Ooi Wei Tsang
68
Limit the damage of error
InterleavingKey frame selectionChange reference frame
69
NUS.SOC.CS5248Ooi Wei Tsang
Project Stage 2
Packet Reassembly and MPEG Decoding
NUS.SOC.CS5248Ooi Wei Tsang
70
Decoding Pipeline
MntRTPFileReader MntRTPDropper MntRTPMPEGReassembler
MntRTPMPEGFrameDecoder
MntPPMWriter
MntPSNRCalculator
MntRTPMPEGPacket
MntMPEGRTPReassemblerBuffer
MntRTPMPEGPacket
MntYUVFrame
MntYUVFrame