Error Control Techniques Error Control Techniques for Interactive Low-bit for Interactive Low-bit Rate Video Transmission Rate Video Transmission over the Internet.over the Internet.

Error Control Techniques Error Control Techniques for Interactive Low-bit for Interactive Low-bit Rate Video Transmission Rate Video Transmission over the Internet.over the Internet.Injong RheeDepartment of Computer ScienceNorth Carolina State University

Video Conferencing over Packet-Video Conferencing over Packet-Switching Networks.Switching Networks.Video Conferencing over Packet-Video Conferencing over Packet-Switching Networks.Switching Networks.

Packet Loss

•¡¡ Compressed video frame is packetized and transmitted.•¡¡ Packet loss can occur quite frequently due to congestion •and user mobility.

Motion Compensated CodingMotion Compensated CodingMotion Compensated CodingMotion Compensated Coding

IntraFrame(I-frame)

InterFrame(P-frame)

Temporal Dependency

• Most video codecs use it.• Each P-frame depends on its previous frames.• I-frame is periodically transmitted.

When a packet is lost,….When a packet is lost,….When a packet is lost,….When a packet is lost,….

IntraFrame(I-frame)

InterFrame(P-frame)

Packet Loss

Error Propagation

• Conventional Solution : Transmit I-frame more often.• However, compression efficiency reduces.

Myth: retransmission is not Myth: retransmission is not useful for interactive videouseful for interactive videoMyth: retransmission is not Myth: retransmission is not useful for interactive videouseful for interactive video

retransmit

NACK

lost

F0 F1 F2 F3 F1 (retransmit)

F0 F1 F2 F3

TIME

(F1 arrives after its display)

Conventionally, late packets are discarded.

Retransmission stops error Retransmission stops error propagation.propagation.Retransmission stops error Retransmission stops error propagation.propagation.

retransmit

NACK

lost

F0 F1 F2 F3 F1 (retransmit)

F0 F1 F2 F3

TIME

F1 is recovered here

Don’t discard: Use late packets to recover their reference frames

Handling retransmitted ‘late’ Handling retransmitted ‘late’ packets in decoderpackets in decoderHandling retransmitted ‘late’ Handling retransmitted ‘late’ packets in decoderpackets in decoder

Old Reference Frame

ReferenceFrameCurrent

Diff.erenceFrame +

Current FrameInfo

Retransmitted Information.

Inverse Quantization/DCTInverse Quantization/DCT

Motion Prediction Compensation

Motion Prediction Compensation

Display

Error PropagationError PropagationError PropagationError Propagation

Recovery from Error Recovery from Error PropagationPropagationRecovery from Error Recovery from Error PropagationPropagation

Error Recovery TechniqueError Recovery TechniqueError Recovery TechniqueError Recovery Technique

•Recovery from Error Spread using Continuous Update (RESCU)

•Focus on error spread/propagation

•Continuous Update– Retransmission (this paper)– Forward error correction

•Combined with Layer coding (this paper)

Extending Packet DeadlineExtending Packet DeadlineExtending Packet DeadlineExtending Packet Deadline

1Add more frame buffers at the decoder.

2Adjust Temporal Dependency Distance (TDD) of a frame.– TDD: frame intervals from that

frame to its temporally dependent frame.

Add more reference frame Add more reference frame buffers.buffers.Cascaded BufferingCascaded Buffering

Add more reference frame Add more reference frame buffers.buffers.Cascaded BufferingCascaded Buffering

•Minus: More computation and buffers at the decoder.

•Plus : No change in the codec and picture pattern.

Base Reference Frame

Reference Frame 1

Contains Full ImagePrediction Errors + MVsPrediction Errors + MVs

Reference Frame 2

Adjust Temporal Dependency Adjust Temporal Dependency

DistanceDistance Periodic TDDs (PTDD)Periodic TDDs (PTDD)

Adjust Temporal Dependency Adjust Temporal Dependency

DistanceDistance Periodic TDDs (PTDD)Periodic TDDs (PTDD)

•Changing Dependency.

• f

• PTDD can be dynamically adjusted.

+Reduced Buffers & computations.

—No protection for non-periodic frame.

Intra Inter Inter Periodic

Continuous UpdatesContinuous UpdatesRetransmission & Forward Error CorrectionRetransmission & Forward Error CorrectionContinuous UpdatesContinuous UpdatesRetransmission & Forward Error CorrectionRetransmission & Forward Error Correction

•Retransmission – Retransmit lost packets within

PTDD.– Retransmission delay requires

larger PTDD.

•Forward Error Correction– send parity packets over a PTDD

period.

Periodic frame Periodic frame

Parity Packet for periodic frame

Protecting non-periodic Protecting non-periodic framesframes RESCU + Quality Assurance LayeringRESCU + Quality Assurance Layering

Protecting non-periodic Protecting non-periodic framesframes RESCU + Quality Assurance LayeringRESCU + Quality Assurance Layering

• .

•HP stream is sent with forward error correction.

•RESCU contributes to reducing bit rates.

HP

LP

HP

LP

HP

LP

HP

LP

protecting non-periodic protecting non-periodic frames frames Change temporal dependencyChange temporal dependency

protecting non-periodic protecting non-periodic frames frames Change temporal dependencyChange temporal dependency

•Errors in non-periodic frames do not propagate.

•As PTDD increases,– Compression efficiency decreases.

Periodic Inter Inter Periodic

Compression EfficiencyCompression Efficiency(MPEG-4 Class A, AKIYO)(MPEG-4 Class A, AKIYO)Compression EfficiencyCompression Efficiency(MPEG-4 Class A, AKIYO)(MPEG-4 Class A, AKIYO)

AVG Bytes/Frame

AV

G P

SN

R (

dB

) Intra-H261

RESCU + QAL

RESCU

H.261

Error ResilienceError Resilience (H.261 v.s. RESCU) (H.261 v.s. RESCU)Error ResilienceError Resilience (H.261 v.s. RESCU) (H.261 v.s. RESCU)

RESCU + QALRESCU + QALRESCU + QALRESCU + QAL

Error ResilienceError Resiliencevarious loss groupsvarious loss groupsError ResilienceError Resiliencevarious loss groupsvarious loss groups

H.261

Intra-H.261

Loss Rate

PS

NR

RESCU + QAL

RESCU

ConclusionsConclusionsConclusionsConclusions

•RESCU accommodates delays in recovery (retransmission or FEC).

•error resilience for interactive video.

•high compression efficiency.– need I-frames less frequently.

•Good for bandwidth-constrained, lossy networks (e.g., mobile wireless networks).

•Much more work to be done.

Error ResilienceError ResilienceRetransmission over various network Retransmission over various network delaysdelays

Error ResilienceError ResilienceRetransmission over various network Retransmission over various network delaysdelays

50 ms100 ms

150 ms

200 ms

250 ms300 ms