MPEG-4 AVC/H.264

International Telecommunication UnionInternational Multimedia Telecommunications Consortium

Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006

ITU-T

MPEGMPEG--4 AVC/H.264 4 AVC/H.264 Digital Video Compression StandardDigital Video Compression Standard

Ajay LuthraAdv. Tech., Motorola [email protected]

Joint ITU-T Workshop and IMTC Forum 2006 “H.323, SIP: is H.325 next?“San Diego, 9-11 May 2006 2

ITU-T

MPEG-4 AVC/H.264

o Historyo Technologyo Adoptiono Next Steps

o Start drinking from the fire hose


ITU-T

MPEG-4 AVC / H.264

ISO (MPEG)

ITU(VCEG)

JVT(Common Standard)

MPEG-4 AVC(Part 10)

H.264 / H.26L

Joint Video Team(Dec 2001)


ITU-T

AVC/H.264

…… Half the bit rate of MPEGHalf the bit rate of MPEG--22Better picture Better picture ……


ITU-T

o Objective• PSNR vs Bit Rate

o Subjective

AVC/H.264 Video Quality Tests


ITU-T

2022242628303234363840

0 500 1000 1500 2000 2500 3000 3500 4000Bitrate (kb/s)

PSN

R (d

B)

MP4 AVC (N=15, M=3)

MP2 (N=15, M=3)

MP4 ASP (N=15, M=3)

Note: PSNR at a given bit rate is encoder dependent and will vary from one encoder to another

MPEG-2, MPEG-4 ASP, AVC/H.264 (MP)Mobile & Calendar (CIF)


ITU-T

24

26

28

30

32

34

36

38

40

0 500 1000 1500 2000 2500 3000Bitrate (kb/s)

PSN

R (d

B)

MP4 AVC (N=15, M=3)

MP2 (N=15, M=3)

MP4 ASP (N=15, M=3)

MPEG-2, MPEG-4 ASP, AVC/H.264 (MP)BUS (CIF)


ITU-T

22

24

26

28

30

32

34

36

0 1000 2000 3000 4000 5000 6000Bitrate (kb/s)

PSN

R (d

B)

AVC (N=15, M=3)MP2 (N=15, M=3)MP4 (N=15, M=3)

MPEG-2, MPEG-4 ASP, AVC/H.264 (MP)Mobile & Calendar (HHR - 352x480)


ITU-T

M obil and Calendar (720x480)

24

26

28

30

32

34

36

400 2400 4400 6400 8400

Bitrate (kb/s)

PSNR

(dB)

MPEG-4 AVCMPEG 2

AVC/H.264 (MP) vs MPEG-2 (MP)


ITU-T

Fl_HD (1920x1088)

28

29

30

31

32

33

34

35

5000 9000 13000 17000

Bitrate (kb/s)

PSN

R (d

B)

MPEG-4 AVCMPEG 2

AVC/H.264 (MP) vs MPEG-2 (MP)


ITU-T~ % Bit rates required for the same PSNR ( ~ 32dB)

(Normalization: MPEG-2 = 100%)

6275

37

68

5033

100 100 100

0

20

40

60

80

100

120

MPEG-2 MPEG-4 ASP AVC/H.264

% Mobil & Calendar (CIF)

Mobil & Calendar (HHR)Bus (CIF)

Note: PSNR at a given bit rate is encoder dependent and will vary from one encoder to another

MPEG-2, MPEG-4 ASP, AVC/H.264 (MP)


ITU-T

Subjective Testing

o MPEG Tests• AVC/H.264 (Main Profile), MPEG-2 (Main Profile)

o Blu-Ray Disc Founders (BDF)• AVC/H.264 (High Profile), MPEG-2 (Main Profile)


ITU-T

MPEG Verification Test

o ITU-R 500-11• Double Stimulus Continuous Quality Scale

(DSCQS) Method

Seq. B Seq. A Seq. B

A B A* B*

Seq. A

Vote 1

2 s 10 s 2 s 10 s 2 s 10 s 2 s 10 s 4 s 1 s

Seq. B Seq. A Seq. B

A B A* B*

Seq. A

Vote 2

2 s 10 s 2 s 10 s 2 s 10 s 2 s 10 s 4 s 1 s

High

20

Low

40

60

80

1A 1B 2A 2B

- Mean Opinion Square and 95% Confidence Interval


ITU-T

Mobile & Calendar (SD)

1

2

3

4

5

AVC_4M

AVC_6M

AVC_3M

AVC_2M25

AVC_1M5

HiQ_6

M

HiQ_4

M

TM5_6M

HiQ_2

M25

TM5_4M

HiQ_3

M

TM5_3M

TM5_2M

25MPEG Verification Test


ITU-THarbour 720(60p)

1

2

3

4

5

AVC_10M AVC_20M TM5_20M TM5_10M HiQ_20M AVC_6M HiQ_10M TM5_6M HiQ_6M



ITU-T

New Mobile & Calendar 1080(30i)

1

2

3

4

5

AVC_10M AVC_20M HiQ_20M TM5_20M TM5_10M HiQ_10M



ITU-T

Blu-Ray Disc Founders (BDF)

o HDTV Subjective Testing


ITU-T

o High at 8 Mbps nominally beats MPEG-2 at 24 Mbpso Nominally transparent on 1080p24 at 16 Mbps

H.264/AVC FRExt 8Mbps

Original MPEG-224 Mbps

Mean Score

2

2.5

3

3.5

4

4.5

5

H.264/AVC FRExt

12Mbps

H.264/AVC FRExt

16Mbps

H.264/AVC FRExt

20Mbps

3.65 3.714.00

4.03 3.90

3.59

Blu-Ray Disc FoundersJVT_L033 - High Profile Tests (HDTV)


ITU-T

AVC/H.264

…… Half the bit rate of MPEGHalf the bit rate of MPEG--22

Better picture Better picture ……


ITU-To I, P, B, Bs, SP, SIo Improved Spatial Prediction

• 4x4• 8x8• 16x16

o Improved Temporal Prediction• Multiple reference frames• Variable block size MC - 7 Block Patterns

— 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x4• Up to 16 MVs per MB• Quarter Pixel Interpolation• Loop filter• Skip• Direct• Weighted

o Interlaced Video Coding Tools• Frame or Field predictions

o Context-based Adaptive Binary Arithmetic Coding (CABAC)o 4x4 or 8x8 Transformso Other

AVC / H.264


ITU-T

AVC/H.264 Coding ToolsMultiple Reference Frames

Current Picture

Decoded Picturesas Reference


ITU-T

AVC/H.264 Coding Tools

o Variable macroblock size

16

16

16

16

8

8

4 4 4 4MPEG-4AVC/H.264

MPEG-2


ITU-T

o Variable macroblock size



ITU-T


Legend: QCIF; Black Boundary Line – Skip; White – Intra; Gray - Inter (P)


ITU-T

AVC/H.264

o Motion estimation accuracy• ¼ Pixel


ITU-T

Built-in (In Loop) Deblocking Filter

No Deblocking Deblocking


ITU-T

+FMO+ASO+Red. Slices

AVC/H.264 Profiles

BaselineMainExtended

+ CABAC

+ SP, SI+ Data

Partitioning

+ B+ MBAFF

• I , P• CAVLC

High+ 8x8


ITU-T

AVC/H.264 Profiles

o High Profile• Highest compression or video quality at a given bit rate• Suitable for good quality entertainment video distribution

o Baseline Profile• Least complexity• Error resilient • Suitable for telephony, conferencing application


ITU-TLevel 1.0: QCIF @ 15frames/secLevel 1.1: QCIF @ 30 frames/sec, CIF @7.5 frames/secLevel 1.2: CIF @ 15 frames/secLevel 2.0: CIF @ 30 frames/secLevel 2.1: HHR @ 25 or 30 frames/secLevel 2.2: SDTV @ 15 frames/secLevel 3.0: SDTV: 720x480x30i, 720x576x25i

10 Mbps (max.), up to 5 (max. resolution) reference framesLevel 3.1: HDTV - 1280x720x30p, SVGA (800x600) 50+p Level 3.2: HDTV - 1280x720x60pLevel 4.0: HDTV (all formats) - 1920x1080x30i, 1280x720x60p, 2kx1kx30p

20 Mbps (max.), up to 4 (max. resolution) reference framesLevel 4.1: HDTV - 1920x1080x30i, 1280x720x60p, 2kx1kx30p

50 Mbps, up to 4 (max. resolution) reference framesLevel 4.2: S-HDTV - 1920x1080x60pLevel 5.0: S-HDTV/D-Cinema – 2kx1kx72p Level 5.1: S-HDTV/D-Cinema – 2kx1kx120p, 4kx2kx24p, 4kx2kx30p

AVC/H.264 – Levels

< 64 kbps

< 250 Mbps

> 5000 x


ITU-T

Transport of AVC / H.264

o Transport of MPEG-4 AVC using MPEG-2 System: ISO/IEC 13818-1 • PDAM (Proposed Draft AMendment) in May 2002• FPDAM (Final Proposed Draft AMendment) in Dec 2002• FDAM in July 2003• Approved AMD

o IP delivery• MPEG-2 TS over UDP/IP, or• RTP over IP


ITU-T

AVC/H.264

o Applications• Wherever you need to save bandwidth and/or storage capacity


ITU-T

AVC/H.264 Adoption

o 3GPP o 3GPP2 o ARIB (Japan) o ATSC o Blu-ray Disc o DLNA o DMB (Korea) o DVB o DVD Forum (HD-DVD)o IETF AVT - RTP payload spec

approved as RFC 3984o ISMA o SCTEo US DoD MISB - Adopted as US

government preferred codec up to 1080p


ITU-T

o Direct Broadcasting Satellite• DirectTV, BSkyB, DISH, Premiere, Euro 1080

o Terrestrial broadcasting• HDTV pay DTV in France

o IPTV• KPN in Netherlands, Belgacom in Belgium, SBC/ATT …

o Streaming• Quicktime

o Video Conferencingo Mobile TVo Portable Media Players

• Video iPOD, Cell Phoneso HD-DVDo Blu-Rayo Video Gameso Digital Cable

AVC/H.264 Commercial Deployment


ITU-T

o Products• NAB 2006 : 125+ companies

— Services— Encoders— Decoders— STB— ICs— Cores— Software— …

AVC/H.264 Commercial Deployment


ITU-T

o Extensions• Higher resolution and quality

— Studios, D-cinema— Very High Definition to home?

AVC/H.264 Standard


ITU-T

AVC/H.264 Profiles

o Higher Resolution and Quality— 4:2:0/10bits – High 10 bit (Hi10P)— 4:2:2/10bits – High 4:2:2 (Hi422)


ITU-T



• Scalable Video Coding (SVC)— Multiple layers— Potential alternative for seamless mobility

o Across networkso Across deviceso Across applications

AVC/H.264 Standard - Next Steps


ITU-T

Scalability

Single Layer Multiple Layers


ITU-T

Scalability

o Temporalo Spatialo SNRo Fine Granular (FGS)


ITU-T





• Multi-View Coding



ITU-T






• Multi-View Coding

o Next standard• Significant improvement over AVC/H.264 ?

— H.265?— MPEG-x ?

• Hunt is ON

MPEG-4 AVC/H.264

Documents

Transcript of MPEG-4 AVC/H.264