H.264/AVC for Wireless Applications

Thomas Stockhammer, and Thomas Wiegand

Institute for Communications Engineering, Munich University of Technology, Germany

Image Processing Department, Fraunhofer Institute for Telecommunications Heinrich-Hertz-Institut

MoMuC 2003INTERNATIONAL WORKSHOP ON MOBILE MULTIMEDIA COMM

UNICATIONS

Outline

Introduction H.264/AVC

Compression Efficiency and Encoder Flexibility Features for Multimedia Messaging and

Wireless packet-based Streaming Features for Wireless Conversational Services-

Rate Control and Error Resilience Rate-Distortion Optimized Mode Selection

Selected Simulation Results Conclusion

Introduction

The primary goals for H264/AVC are improved coding efficiency and improved network adaptation.

H.264/AVC distinguishes between two different conceptual layers, the Video Coding Layer (VCL), and the Network Abstraction Layer (NAL).

Introduction

Due to the likely business models in emerging wireless systems, in which the end-user’s costs are proportional to the transmitted data volume, and also due to limited resources bandwidth and transmission power, compression efficiency is the main target for wireless video and multimedia applications.

This makes H.264/AVC coding an attractive candidate for all wireless applications including Multimedia Messaging Services (MMS), packet-switched streaming services (PSS) and conversational applications.

Introduction

For efficient transmission in different environments not only coding efficiency is relevant, but also the seamless and easy integration of the coded video into all current and possible future protocol and multiplex architectures.

The VCL specifies an efficient representation for the coded video signal.

The NAL defines the interface between the video codec itself and the outside world.

Structure of H.264/AVC video encoderC

on

trol D

ata Video Coding Layer

Data Partitioning

Network Abstraction Layer

H.320 MP4FF H.323/IP MPEG-2 Etc.

Coded Macroblock

Coded Slice/Partition

H.264/AVCCompression Efficiency and Encoder Flexibility

Figure 1 H.264/AVC Encoder realization with coding options.

H.264/AVCFeatures for Multimedia Messaging and Wireless packet-based Streaming

MMS (Multimedia Messaging Service) Compression efficiency : due to the strict

separation of encoding, transmission and decoding

IDR (Instantaneous Decoder Refresh): for random access and fast forward.

Rate control: for constant video quality Reliable transmission: for wireless link

layer

H.264/AVCFeatures for Multimedia Messaging and Wireless packet-based Streaming

Streaming: online transmission and decoding

Short-term variances in the bit-rate With an appropriate setting of the initial delay

and receiver buffer a certain quality of service can be guaranteed.

Wireless channels commonly provide a constant bit-rate and reliable transmission by using an acknowledged mode within a window of a few seconds.

H.264/AVCFeatures for Multimedia Messaging and Wireless packet-based Streaming

Long-term variances in the bit-rate: due to distance, shadowing, re-newed resour

ce allocation Channel adaptive streaming technologies

Adaptive media playout Rate-Distortion optimized packet scheduling Frame dropping if the channel rate fluctuates

In small range: non-reference frames => temporal scalability

In large scale: I frames, SP (Switching Predictive) frames

H.264/AVCFeatures for Wireless Conversational Services-Rate Control and Error Resilience

The low delay constraint has two main impacts on the video transmitted over wireless bearer services with constant bit-rate. Fast quantization parameter adaptation Temporally backward references in MC

An error-resilient video coding standard suitable for conversational wireless services has to provide to combat two problems: it is necessary to minimize the visual effect of errors within

one frame, and as errors cannot be avoided, the well-known problem of s

patio-temporal error propagation in hybrid video coding has to be limited.

H.264/AVCFeatures for Wireless Conversational Services-Rate Control and Error Resilience

Error-resilience features included in the H.264/AVC standard Slice-Structured Coding

A slice is a sequence of MBs and provides spatially distinct resynchronization points within the video data for a single frame.

Advantages: Packet loss probability can be reduced if slices and transmis

sion packets are relatively small. re-synchronization possibility within one frame

Disadvantages: Increase packet overhead Loss of intra-frame prediction

Group-of-Block (GOB) and Slice Interleaving Reduce no coding overhead in the VCL, but the costly R

TP overhead of up to 40 bytes per packet.

H.264/AVCFeatures for Wireless Conversational Services-Rate Control and Error Resilience

Flexible MB Ordering (FMO) FMO permits the specification of different patter

ns for the mapping of MBs. Data partitioning

reduce visual artifacts resulting from packet losses, especially if prioritization or unequal error protection is provided by the network.

Encoding of single MBs for regions Multiple reference frames

limit error propagation

H.264/AVCRate-Distortion Optimized Mode Selection

The concept of selecting appropriate coding options in optimized encoder designs for many video coding standards is based on rate-distortion optimization algorithms

The Lagrange parameter for appropriate weighting of rate and distortion has to be selected appropriately.

In the H.264/AVC test model, the Lagrangian mode selection is used for motion vector search as well as MB mode and reference frame selection.

Selected Simulation ResultsCompression Efficiency

Figure 2 Coding performance of H.264/AVC codec compared to state-of-the-art video coding standards for QCIF test sequence foreman at frame rate 10 Hz.

Selected Simulation ResultsSlices and Channel-Adaptive Intra Updates

Figure 3 Cumulative distribution of decoded PSNR for differentNAL unit erasure rates for the estimation of the expected distortion in the encoder.

8%

30%

Channel statistics are taken into account into the selection of the coding option in the encoder.

Packet loss rate=4%Pure R-D

Selected Simulation ResultsSlices and Channel-Adaptive Intra Updates

Figure 4 Cumulative distribution of decoded PSNR for differenterror-resilience strategies: channel-optimized intra updates with and without slice structuring for different assumed loss probabilities p.

Slice structuring + Channel adaptive

Slice structuring

Channel adaptive

Selected Simulation ResultsExploiting Feedback in Video Encoding

Figure 5 Cumulative distribution of decoded PSNR for differenterror-resilience strategies: RD-optimized intra updates with slice structuring, and feedback mode with and without slice structuring for delay d=2 and d=4.

RD-optimized + slice structuring

Feedback (2) + slice structuring

Feedback (2)

Feedback (4) + slice structuring

Feedback (4)

Conclusion

In addition to excellent coding efficiency, the design of H.264/AVC also takes into account network adaptation providing large flexibility for its use in wireless applications.

In experimental results based on common test conditions it has been shown that in case without any feedback, several slices in combination with channel-adaptive rate-distortion optimized mode selection is a promising approach.

In case of available feedback, the application of multiple reference frames to exclude error propagation without slice structuring provides excellent results.