1

Topic: Advanced Video Coding Standard(Comparison of HEVC with H.264 and H.264 with MPEG-

2)

A PROJECT UNDER THE GUIDANCE OF DR. K. R. RAO

COURSE: EE5359 - MULTIMEDIAPROCESSING, SPRING 2015

SUBMITTED BY JAGRITI DHINGRA

UT ARLINGTON ID: 1001103750

EMAIL ID: jagriti.dhingra@mavs.uta.edu

2

ATSC: Advanced Televisions Systems CommitteeAVC: Advanced Video Coding. BD-BR: Bjontegaard Delta Bitrate. BD-PSNR: Bjontegaard Delta Peak Signal to Noise Ratio. CABAC: Context Adaptive Binary Arithmetic Coding. CTB: Coding Tree Block. CTU: Coding Tree Unit. CU: Coding Unit. DBF: De-blocking Filter. DCT: Discrete Cosine Transform. DVB: Digital Video Broadcast HEVC: High Efficiency Video Coding. HM: HEVC Test Model. ICME: International Conference on Multimedia and ExpoIEC: International Electro-technical Commission. ISDB: Integrated Services Digital Broadcasting ISO: International Organization for Standardization.

TABLE OF ACRONYMS

3

ITU-T: International Telecommunication Union- Telecommunication Standardization Sector. JCT: Joint Collaborative Team. JCT-VC: Joint Collaborative Team on Video Coding. JM: H.264 Test Model. JPEG: Joint Photographic Experts Group. MC: Motion Compensation. ME: Motion Estimation. MPEG: Moving Picture Experts Group. MSE: Mean Square Error. PB: Prediction Block. PSNR: Peak Signal to Noise Ratio. QP: Quantization Parameter SAO: Sample Adaptive Offset. SSIM: Structural Similarity Index. TB: Transform Block. TU: Transform Unit. VCEG: Visual Coding Experts Group.

4

Typical structure of an H.264/MPEG4-AVC video encoder.[1]

Figure 1

5

Advanced Video Coding

• H.264 or MPEG-4 Part 10, Advanced Video Coding (MPEG-4 AVC) is a video compression format that is currently one of the most commonly used formats for the recording, compression, and distribution of video content

• H.264 is perhaps best known as being one of the video encoding standards for Blu-ray Discs; all Blu-ray Disc players must be able to decode H.264.

• It covers all common video applications ranging from mobile services and videoconferencing to IPTV, HDTV, and HD video storage.[2]

• The H.264iAVC design [3] supports the coding of video (in 42:O chroma format) that contains either progressive or interlaced frames, which may be mixed together in the same sequence.

6

In H.264/AVC, three profiles are defined. Baseline, Main and X The Baseline profile supports all features in H.264iAVC except the following two feature sets: Set 1: B slices, weighted prediction, CABAC, field coding and macroblock adaptive switching between frame & field coding.

Set 2: SP and SI slices.

Main profile does not support the Flexible Macroblock Ordering (FMO) feature which is supported by the Baseline profile.

Profile X supports both sets of features on top of the Baseline profile, except for CABAC and macroblock adaptive switching between frame and field coding.

7

• H.264 design is divided into two layers i.e. Network Abstraction Layer VAL) and Video Coding Layer (VCL).

• NAL formats the VCL layer data into a format that is suitable for transmission by variety of transport layers whereas VCL does the compression of video.

• The video coding layer of H.264iAVC is similar in spirit to other standards such as MPEG-2 Video. It consists of a hybrid of temporal and spatial prediction, in conjunction with transform coding.

• Fig. 1 shows a block diagram of the video coding layer for a macroblock.

8

Figure 2: H.264 Encoder Block Diagram[4]

9

Video compression picture types

• There are three types of pictures (or frames) used in video compression: I‑frames, P‑frames and B‑frames.[7]

• An I‑frame is an 'Intra-coded picture', in effect a fully specified picture, like a conventional static image file. P‑frames and B‑frames hold only part of the image information, so they need less space to store than an I‑frame and thus improve video compression rates.[7]

Figure 2: Video compression types[7]

10

• A P‑frame ('Predicted picture') holds only the changes in the image from the previous frame. For example, in a scene where a car moves across a stationary background, only the car's movements need to be encoded. The encoder does not need to store the unchanging background pixels in the P‑frame, thus saving space. P‑frames are also known as delta‑frames.[7]

• A B‑frame ('Bi-predictive picture') saves even more space by using differences between the current frame and both the preceding and following frames to specify its content.[7]

11

H.264 standard• The standard was developed jointly in a partnership of VCEG and MPEG,

after earlier development work in the ITU-T as a VCEG project called H.26L. It is thus common to refer to the standard with names such as H.264/AVC, AVC/H.264, H.264/MPEG-4 AVC, or MPEG-4/H.264 AVC, to emphasize the common heritage.

• The standardization of the first version of H.264/AVC was to extend the original standard, the JVT then developed what was called the Fidelity Range Extensions (FRExt). These extensions enabled higher quality video coding by supporting increased sample bit depth precision and higher-resolution color information, including sampling structures known as Y'CbCr 4:2:2 (=YUV 4:2:2) and Y'CbCr 4:4:4.[2]

• The next major feature added to the standard was Scalable Video Coding (SVC). Specified in Annex G of H.264/AVC, SVC allows the construction of bitstreams that contain sub-bitstreams that also conform to the standard, including one such bitstream known as the "base layer" that can be decoded by a H.264/AVC codec that does not support SVC.[2]

12

HEVC standard

• High Efficiency Video Coding (HEVC) [12] is an international standard for video compression developed by a working group of ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Video Coding Experts Group). The main goal of HEVC standard is to significantly improve compression performance compared to existing standards (such as H.264/Advanced Video Coding [9]) in the range of 50% bit rate reduction at similar visual quality.[8]

Figure 4: Block Diagram of HEVC CODEC [12]

13

MPEG-2

• MPEG-2 is widely used as the format of digital television signals that are broadcast by terrestrial (over-the-air), cable, and direct broadcast satellite TV systems. It also specifies the format of movies and other programs that are distributed on DVD and similar disks. [10]

• The video section, part 2 of MPEG-2, is similar to the previous MPEG-1 standard, but also provides support for interlaced video; the format used by analog broadcast TV systems. MPEG-2 video is not optimized for low bit-rates, especially less than 1 Mbit/s at standard definition resolutions. However, it outperforms MPEG-1 at 3 Mbit/s and above. MPEG-2 is directed at broadcast formats at higher data rates of 4 Mbps (DVD) and 19 Mbps (HDTV).[11]

14

Table 1 is a tabulated form of the properties of the various MPEG-2 profiles. [13]

15

Table 2. MPEG-2 Levels [13]

16

Comparison Metrics• Peak Signal to Noise Ratio Peak signal-to-noise ratio (PSNR) [22] [26] is an expression for the ratio

between the maximum possible value (power) of a signal and the power of distorting noise that affects the quality of its representation. PSNR is most commonly used to measure the quality of reconstruction of lossy compression codecs. The signal in this case is the original data, and the noise is the error introduced by compression. When comparing compression codecs, PSNR is an approximation to human perception of reconstruction quality. Although a higher PSNR generally indicates that the reconstruction is of higher quality, in some cases it may not. One has to be extremely careful with the range of validity of this metric; it is only conclusively valid when it is used to compare results from the same codec (or codec type) and same content.

• PSNR is defined via the mean squared error (MSE). Given a noise-free m x n monochrome image I and its noisy approximation K, MSE is defined as:

17

18

Profiles used for Comparison

The HM 16.0 main profile [6] and JM 18.6 high profile [8] will be used for comparison in this project.

• Main profile in HEVC

This profile allows for a bit depth of 8-bits per sample with 4:2:0 chroma sampling, which is the most common type of video used with consumer devices.

• High Profile in H.264

This profile allows for a bit depth of 8-bits per sample with 4:2:0 Chroma sampling. It is used for broadcast and disc storage applications, particularly for high-definition television applications (for example, this is the profile adopted by the Blu-ray Disc storage format and the DVB HDTV broadcast service).

19

Test Sequences

20

Test Sequences [23]

21

Results

22

Graphs

23

24

25

Comparison: Similarities and Differences between MPEG-2 video and H.264 video

The MPEG-2 and H.264 video coding standards are compared with respect to their various aspects such as bit rate, block size, macroblock size, intra prediction, motion estimation blocks, quantization, motion vector prediction, intra prediction amongst various other.

26Table 3: Comparison between MPEG-2 and H.264

27

Performance comparison between MPEG-2 and H.264 using standard test streams – Simulation results• Test streams (foreman and carphone [47]) were encoded using the open-

source MPEG-2 codec [46] and the H.264 codec [45]. The results were compared against each other for parameters like the signal to noise ratio (SNR), GOP and compression ratio. CIF files were used for the “Foreman” and QCIF was used for the “Carphone” clip.

• The bit rate for H.264 encoding was taken as the standard one used by the codes. The bit rate for MPEG-2 encoding was adjusted on the basis of the bit rate of the H.264 encoding process.

• This helped to compare the two standards on a common plane. While the aim of this project is to compare the Main profiles of MPEG-2 and H.264, simulations were run for the Simple/Baseline profiles too.

• This was done in order to prove quantitatively that encoding using the Main profile for both MPEG-2 and H.264 gives a better compression ratio and better quality video than the Simple profile.

28

From the tables below, the following is concluded:For the same bit rate and video resolution, the PSNR (dB) values are greater for H.264 encoded videos than for the MPEG-2 encoded videos indicating better video quality. This can be verified from the screen shots.

The compression ratio for H.264 encoded video is also better than that for MPEG-2 encoded video inspite of better quality video. Compression ratio = original file size/compressed file sizeThe video quality for H.264 video is better than for MPEG-2 video for the Simple/Baseline profiles as well. Therefore, it can be concluded that H.264 video coding standard gives better compression and better video quality as compared to MPEG-2.

29

30

31

32

33

CONCLUSION

34

The results indicate that the emerging HEVC standard can provide a significant amount of increased coding efficiency compared to previous standards, including H.264/MPEG-2 and H.264/H.263 AVC. The syntax and coding structures of the various tested standards were explained, and the based encoder optimization was described. HEVC that is relevant to its coding efficiency.

35

References[1] “Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264/ISO/IEC 14 496-10 AVC,” in Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, JVTG050, 2003.

[2] “Generic Coding of Moving Pictures and Associated Audio Information - Part 2: Video,” ITU-T and ISO/IEC JTC 1, ITU-T Recommendation H.262 and ISO/IEC 13 818-2 (MPEG-2), 1994.

[3] IEEE SIGNAL PROCESSING MAGAZINE [148] MARCH 2007. https://www.ic.tberlin.de/fileadmin/fg121/publications/2007_42_h264avc_nutshell.pdf

[4] http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC

[5] I.E.G. Richardson, “Video Codec Design: Developing Image and Video Compression Systems”, Wiley, 2002.

[6] G. J. Sullivan et al, “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668, Dec. 2012

[7] IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, NO. 7, JULY 2003.

[8] T. Wiegand et al, “Overview of the H.264/AVC Video Coding Standard”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, pp. 560-576, July 2003.

[9] D. Marpe et al, “The H.264/MPEG4 advanced video coding standard and its applications”, IEEE Communications Magazine, Vol. 44, pp. 134-143, Aug. 2006.

[10] A. Puri et al, “Video coding using the H.264/MPEG-4 AVC compression standard”, Signal Processing: Image Communication, vol. 19, pp. 793-849, Oct. 2004.

36

[11] X. Li et al, “Rate-complexity-distortion evaluation for hybrid video coding”, IEEE International Conference on Multimedia and Expo (ICME), pp. 685-690, July 2010.

[12] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC-L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.it-sudparis.eu/jct/doc_end_user/current_document.php?id=7243

[13] JVT Draft ITU-T recommendation and final draft international standard of joint video specification

(ITU-T Rec. H.264-ISO/IEC 14496-10 AVC), March 2003, JVT-G050 available on http://ip.hhi.de/imagecom_G1/assets/pdfs/JVT-G050.pdf

[14] J. Vanne et al, “Comparative Rate-Distortion-Complexity Analysis of HEVC and AVC Video Codecs”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1885-1898, Dec. 2012.

[15] G. Bjontegaard, “Calculation of Average PSNR Differences between RD Curves”, document VCEG-M33, ITU-T SG 16/Q 6, Austin, TX, Apr. 2001.

[16] D. Grois et al, “Performance Comparison of H.265/ MPEG-HEVC, VP9, and H.264/ MPEG-AVC Encoders”, available on: http://iphome.hhi.de/marpe/download/Performance_HEVC_VP9_X264_PCS_2013_preprint.pdf

[17] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html

[18] H.264 tutorial by I.E.G. Richardson: https://www.vcodex.com/h264.html

[19] HEVC white paper-Ittiam Systems: http://www.ittiam.com/Downloads/en/documentation.aspx

[20] HEVC white paper-Ateme: http://www.ateme.com/an-introduction-to-uhdtv-and-hevc

37

[21] HEVC white paper-Elemental Technologies: http://www.elementaltechnologies.com/lp/hevc-h265-demystified-white-paper

[22] White paper on PSNR-NI: http://www.ni.com/white-paper/13306/en/

[23] Test Sequences: ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/

[24] Access to HM 16.0 Reference Software: http://hevc.hhi.fraunhofer.de/

[25] Access to JM 18.6 Reference Software: http://iphome.hhi.de/suehring/tml/

[26] Website on PSNR: http://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio

[27] Website on SSIM: http://en.wikipedia.org/wiki/Structural_similarity

[28] Access the website http://www-ee.uta.edu/Dip/Courses/EE5359/ and refer to the project by S. Kulkarni on “Transcoding from H.264/AVC to High Efficiency Video Coding (HEVC)”, University of Texas, Arlington, Spring 2013.

[29] Access the website http://www-ee.uta.edu/Dip/Courses/EE5359/ and refer to the project by H. B. Jain on “Comparative and performance analysis of HEVC and H.264 intra frame coding and JPEG 2000”, University of Texas, Arlington, Spring 2013.

[30] Access the website http://www-ee.uta.edu/Dip/Courses/EE5359/ and refer to the thesis by S. Vasudevan on “Fast intra prediction and fast residual quad-tree encoding implementation in HEVC”, University of Texas, Arlington, 2013.

38

[31] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T / ISO-IEC Document: JCTVC J0292r1, July 2012.

[32] Z. Wang et al, “Image Quality Assessment: From Error Visibility to Structural Similarity”, IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600-612, Apr. 2004.

[33] G. Sullivan et al, “Standardized Extensions of High Efficiency Video Coding (HEVC)”, IEEE Journal of selected topics in Signal Processing, Vol. 7, No. 6, pp. 1001-1016, Dec. 2013

[34] “MPEG-2”, Wikipedia, Feb. 14, 2008. Available at <http://en.wikipedia.org/wiki/Mpeg_2> [35] “H.262 : Information technology - Generic coding of moving pictures and associated audio information: Video”, International Telecommunication Union, 2000-02. [36] “MPEG-2 White paper”, Pinnacle Technical Documentation, Version 0.5, Pinnacle Systems, Feb. 29, 2000. [37] X. Li et al, “Rate-complexity-distortion evaluation for hybrid video coding”, IEEE International Conference on Multimedia and Expo (ICME), pp. 685-690, Jul. 2010. [38] G. Bjontegaard, “Calculation of Average PSNR Differences between RD Curves”, document VCEGM33, ITU-T SG 16/Q 6, Austin, TX, Apr. 2001.

39

[39] https://hevc.hhi.fraunhofer.de/trac/hevc JCT-VC Document. [40] R. Hoffner, “MPEG-4 Advanced Video Coding emerges”, Available at < http://www.tvtechnology.com/features/Tech-Corner/F_Hoffner- 03.09.05.shtml> [41] Implementation Studies Group, “Main Results of the AVC Complexity analysis”, MPEG document N4964, ISO/IEC JTC11/SC29/WG11, July 2002. [42] S. Wagston and A. Susin, “IP core for an H.264 Decoder SoC”, 2007, Available at< www.us.design-reuse.com/news/?id=15746&print=yes> [45] H.264 software (version 13.2) obtained from: http://iphome.hhi.de/suehring/tml/ 46] MPEG-2 software (version 12) obtained from:http://www.mpeg.org/MPEG/video/mssg-free-mpeg-software.html [47] Test streams (Foreman, News, Carphone) obtained from: http://www-ee.uta.edu/dip/Courses/EE5356/ee_5356.htm