Topics in Signal ProcessingInterim Report

Sujatha Gopalakrishnan

1001024145

Basic Acronyms

• HEVC- High Efficiency Video Coding• TU- Transform Unit• PU- Prediction Unit• CU- Coding Unit• JCT-VC - Joint Collaborative Team on Video Coding• Tmuc – Test Model under Consideration• HM- HEVC Test Model • DCT- Discrete Cosine Transform• MC- Motion Compensation• SAO- Sample Adaptive Offset• ME- Motion estimation• SAP- Sample based Angular Intra-Prediction• NLM- Non-Local Means

HEVC

• High Efficiency Video Coding (HEVC) [1][2] is a Video Compression standard, a successor to H.264/MPEG-4 AVC [22]. HEVC is said to double the data compression ratio compared to H.264/MPEG-4 AVC [1] at the same level of video quality [2].

• The design of most video coding standards is primarily aimed at having the highest coding efficiency.

• The main goal of the HEVC standardization effort is to enable significantly improved compression performance relative to existing standards, in the range of 50% bit rate reduction for equal perceptual video quality [10] [11].

Block Diagram HEVC EncoderFigure 1: HEVC Encoder [2]

Block Diagram HEVC DecoderFigure 2: HEVC Decoder [3]

HEVC Lossless Coding • The lossless coding mode of HEVC main profile bypasses transform

quantization and in-loop filters as shown in the fig.2 [4] [19].

• Comparing it with non-lossless coding mode, it has smallest quantization parameter value.

• Lossless coding mode provides perfect fidelity and average bit rate reduction.

• Outperforms the existing lossless compression solution such as JPEG-2000 [22] and JPEG-LS [22].

• It can prevent accumulation of quantization errors in repeated encoding and decoding operations of video editing

• In this method it is essential to preserve numerical video data with fewer bits.

• DCT coefficients i.e., float-point numbers have to be quantized instead of DCT.

• Lossless video coding is used when perfect preservation of video data is required [29].

• It employs Sample Angular-based Intra-Prediction (SAP) [4]. Same prediction mode signaling method. Same interpolation method of HEVC. Uses adjacent neighbors as reference shown in fig.8. Prediction residuals are coded with the entropy coder

in the spatial domain [29].

Block Diagram HEVC Lossless Coding Figure 3: HEVC lossless Algorithm Block Diagram [4]

Algorithm of Sample Based Angular Intra Prediction

• The SAP [4] is designed to better exploit the spatial redundancy in the lossless coding mode by generating intra prediction samples from adjacent neighbors. The design principle here is very similar to the sample-based DPCM in [21] [4] H.264/MPEG-4 AVC [20] [4] lossless coding, but SAP is fully harmonized with the HEVC block-based angular intra prediction, and can be applied to all the angular intra prediction modes specified in HEVC [4].

• As shown in the fig.7 SAP is performed sample by sample. The adjacent neighboring samples,  of the current sample   in the current PU are used for prediction.

SAP AlgorithmFigure 4: Algorithm of SAP [4]

DCT for HEVC lossless compression• DCT is applied to prediction residuals. • DCT coefficients are quantized.• Quantized DCT coefficients and quantization error are coded.• Coding of each unit is performed by Adaptive Quantization

parameters.

Improved HEVC lossless compression using Two-Stage coding• Block of residual signal is separated into two parts:• Part 1: Quantized DCT coefficients.• Part 2: Quantization error.• Quantized coefficients are used to reconstruct a lossy decoded

block which is subtracted from the residual block.• Quantization error is encoded as the spatial block.

Figure 5: Two-stage lossless coding [30]

Table 1: Coded block for two-stage coding [30]

Table 2: Performance of two-stage coding [30]

• Pixel-based averaging predictor

• Lossless video compression of noisy video content can be improved if the noise within the video is considering the compression [32].

• In HEVC lossless coding block-wise processing is not needed, pixel-wise prediction could be performed for better spatial correlation within the image or a video signal [31].

• De-noised intra prediction scheme is used, where de-noising is performed by the predictor instead of removing the noise.

• Non-Local Means (NLM) algorithm is used for de-noising [33].

• Pixel-wise prediction is the combination of linear predictors with exponentially decaying weights from NLM algorithm.

• Developed predictor results in a weighted average of surrounding pixels.

• Other non-local predictors e.g., forward adaptive scheme where intra-frame motion compensation is performed [34] [32] or a backward adaptive scheme where template matching is performed for prediction [35] [36] [32], are designed for block-wise lossy prediction in H.264/AVC and thus are not efficient

for lossless compression.Figure 6: NLM algorithm for de-noising [32].

NLM Algorithm• The Non-Local Means (NLM) [31] algorithm has been introduced in

[32] for image de-noising. In NLM de-noising, the estimate for a de-noised version of a noisy pixel is established by averaging all the pixels in the local as well as non-local neighborhood. The process of NLM [31] de-noising is illustrated in Fig 9. In the illustrated ex-ample, the pixel g[i] should be de-noised, where i=(x,y) is the two-dimensional coordinate. Therefore all pixels in the support area S are averaged depending on their similarity to g[i].

• The similarity between the pixels is measured by a certain mean distance of the pixels in the surrounding area, which is illustrated with the square around the pixels g[j1], g[j2] and g[j3].

The averaging process is described by:

ΡNLM[i] = ∑j s ∊ w[I,j], g[j] [31]

Figure 7: Casual patches for NLM predictor [31]

Low-Complexity Pixel wise Predictor Implementation• Run time for prediction is proportional to the

neighborhood size or the patch size.• If the patch becomes larger, structural complexity

of the patch becomes higher, so it becomes harder to find similar patches.

• Hence patch and neighborhood sizes are reduced in NLM predictor.

• Results of the investigated parameter constellations a, b, h, dSSE and dSAD are shown in Table 3 [32].

Table 3: Compression results of the proposed pixel-wise prediction [32]

• Test Sequences

Sequences are obtained from [29] and experimented to obtain the performance based on various parameters described as follows.

Figure 8: Race horse sequence [29]

Figure 9: Basketball drill sequence [29]

• Test sequence 1:

Test Sequence Resolution Frame rate (fps)

RaceHorses_416x240_30.yuv

416 x 240 30

Test sequence

  Intra Profile

Random Access Profile

BD- % Bit rate reductio

n

BD- PSNR

 

 

Race horse

sequence

PSNR (dB) 34.2442 33.7342  

-22.4269

 

1.483Bitrate(kbps)

1842.6421

371.49

Encoding time(sec)

24.332 119.764

Decoding time(sec)

0.840 4.134

Test sequence 2:

Test Sequence Resolution Frame rate (fps)

BasketballDrill_832x480_50.yuv

832 x 480 50

Test sequence

  Intra Profile

Random Access Profile

BD- % Bit rate reduction

BD- PSNR

 

 

Basketball drill

PSNR (dB) 37.2947 35.7193  

-32.8763

 

1.956Bitrate(kbps) 5941.7732

817.87

Encoding time(sec)

97.539 347.891

Decoding time(sec)

1.2 4.28

Project Results BD-PSNR (dB) for the test sequences

Race horse (1.483) Basket ball drill (1.956)

0

0.5

1

1.5

2

2.5

BD- PSNR (dB) HEVC Lossless Coding

BD- PSNR (dB)

Test Sequences

BD- P

SNR(

dB)

BD-% Bit rate reduction for test sequences

Race horse (-22.4269)

Basket ball drill (-32.8763)

-35

-30

-25

-20

-15

-10

-5

0

BD- Bitrate % for HEVC Lossless coding

%BD- Bitrate

Test Sequenes

%BD

- Bitr

ate

Test sequence 1 [29]: Encoding and Decoding time(secs) for Intra and Random access profiles

Racehorse_Intra Racehorse_Random Access

Encoding Time (secs)

24.332 119.764

Decoding Time (secs)

0.84 4.134

10

30

50

70

90

110

130 Encoding Time vs Decoding Timese

cs

Test sequence 2 [29]: Encoding and Decoding time(secs) for Intra and Random access profiles

Basketball drill_Intra Basketball drill_Random Access

Encoding Time (secs)

97.539 347.891

Decoding Time (secs)

1.2 4.28

2575

125175225275325375

Encoding Time vs Decoding Time

secs

Future Work

• Future simulations will be conducted using the HM 16.0 [16] software for other test sequences [29]; results will be plotted and compared for the best results using the parameters of PSNR, Bit rate, encoding and decoding time. Also BD-PSNR and BD-%bit rate reduction values will be plotted. Theoretical analysis will be carried out on sample-based angular prediction for HEVC lossless coding.

References• [1] G.J. Sullivan et al, “Overview of the high efficiency video coding (HEVC)

standard”, IEEE Trans. CSVT, vol. 22, pp.1649-1668, Dec. 2012.

• [2] G.J. Sullivan et al, “Standardized Extensions of High Efficiency Video

Coding (HEVC)”, IEEE Journal of selected topics in Signal Processing, vol.7, pp.1001-1016, Dec. 2013.

 • [3] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T/ISO/IEC

Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC- J0292r1, July. 2012.

• [4] M. Zhou et al, “HEVC lossless coding and improvements”, IEEE Trans. CSVT, vol.22, pp.1839-1843, Dec. 2013.

• [5] N. Purnachand et al, "Fast Motion Estimation Algorithm for HEVC", IEEE Second International Conference on Consumer Electronics-Berlin (ICCE-Berlin), vol.11, pp.34-37, Sep. 2012.

• [6] P. Hanhart et al, “ Subjective quality evaluation of the upcoming HEVC video compression standard”, SPIE Optical Engineering+ Applications, International Society for Optics and Photonics, vol. 8499, pp.84990v-84990v, Aug. 2012.

• [7] M. Horowitz et al, “Informal subjective quality comparison of video compression performance of the HEVC and H.264/MPEG - 4 AVC standards for low delay applications”, SPIE Optical Engineering+ Applications. International Society for Optics and Photonics, vol.84990, pp.84990w-84990w, Aug. 2012.

• [8] A. Abdelazim, W. Masri and B. Noaman, "Motion estimation optimization tools for the emerging high efficiency video coding (HEVC)", IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, vol. 9029, pp. 902905-902905, Feb. 2014.

• [9] M. Jakubowski and G. Pastuszak, “Block-based motion estimation algorithms-a survey”, Journal of Opto-Electronics Review, vol. 21, pp.86-102, Mar. 2013.

• [10] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC-L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Jan. 2013, available on http://phenix.it-sudparis.eu/jct/doc_end_user/current_document.php?id=7243

• [11] J. Ascenso et al, "Improving Frame Interpolation with Spatial Motion Smoothing for Pixel Domain Distributed Video Coding", 5th EURASIP Conference on Speech and Image Processing, Multimedia Communications and Services, pp.1-6, Smolenice, Slovak Republic, July. 2005. 

• [12] X. Wang et al, “Paralleling Variable Block Size Motion Estimation of HEVC on Multicore CPU plus GPU platform”, IEEE International Conference on Image Processing (ICIP), vol.22, pp. 1836-1839, Sep. 2013.

• [13] Introduction to parallel computing https://computing.llnl.gov/tutorials/parallel_comp/#Whatis

• [14] L. Zhao et al, “Group-Based Fast mode decision algorithm for intra prediction in HEVC”, IEEE Eighth international Conference on Signal Image Technology and Internet based Systems. Article no.6115979, pp. 225-229, Nov. 2011.

• [15] V. Sze and M. Budagavi, "High Throughput CABAC Entropy Coding in HEVC", IEEE Transactions on Circuits and Systems for Video Technology, vol.22, no.12, pp.1778-1791, Dec 2012.

• [16] T.Nguyen et al, "Transform Coding Techniques in HEVC", IEEE Journal of Selected Topics in Signal Processing, vol.7, pp.978–989, Dec. 2013.

• [17] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html

• [18] HEVC Reference Software HM16.0. https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.0rc1/

• [19] B. Bross et al,“High Efficiency Video Coding (HEVC)Text Specification Draft 8”, JCT-VC document, JCTVC-J1003, Stockholm, Sweden, Jul. 2012.

http://phenix.it-sudparis.eu/jct/doc_end_user/current_document.php?id=6465

• [20] Joint Video Team, “Advanced Video Coding for Generic Audiovisual Services”, ITU-T Rec. H.264 and ISO/IEC, 14496-10 (MPEG-4) AVC, pp.H.100-H.869, Feb. 2014.

• [21] Y.L. Lee et al, "Improved lossless intra coding for H.264/MPEG-4 AVC", IEEE Trans. Image Process., vol.15, no.9, pp.2610-2615, Sep 2006.

• [22]K.R. Rao, D.N Kim and J.J Hwang, “High Efficiency Video Coding (HEVC) Revised/Updated Chapter from the book Video Coding Standards”–Springer 2014.

• [23] ITU-T website: http://www.itu.int/ITU-T/index.html

• [24] JCT-VC documents are publicly available at http://ftp3.itu.ch/av-arch/jctvc-site and http://phenix.it-sudparis.eu/jct/

• [25] V.Sze, M. Budagavi, and G.J. Sullivan, “High Efficiency Video Coding (HEVC) Algorithms and Architectures” Springer, 2014.

• [26] Software reference manual for HM:

https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/HM-9.2-dev/doc/software-manual.pdf

        

• [27] M. Wien, “High efficiency video coding: Tools and specification”, Springer, 2015.

• [28] I.E. Richardson, “Coding video: A practical guide to HEVC and beyond”, Wiley, 11 May 2015.

• [29] Video Sequences: http://ultravideo.cs.tut.fi/http://forum.doom9.org/archive/index.php/t-135034.html

[30] C. Xun and Q. Gu, "Improved HEVC lossless compression using Two-Stage coding with Sub-Frame level optimal quantization values", IEEE International Conference. Image Processing (ICIP), vol.23, pp. 5651-5655, Oct. 2014.

• [31] W. Eugen et al, "Pixel-based averaging predictor for HEVC lossless coding", IEEE International Conference. Image Processing (ICIP), vol.23, pp. 1806-1810, Sept 2013.

 • [32] E. Wige et al, "In-loop denoising of reference frames for lossless coding of noisy

image sequences" IEEE International Conference. Image Processing (ICIP), vol.19, pp. 461-464, Sept 2010.

• [33] A. Buades, B. Coll, and J.M. Morel, "A non-local algorithm for image denoising", IEEE Computer Society Conference. Computer Vision and Pattern Recognition (CVPR), vol.2, pp. 60-65, June 2005.

• [34] S.L. Yu and C. Chrysafis,” New intra prediction using intra-macroblock motion compensation”, Joint Video Team (JVT) of ISO/IEC MPEG &ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Std., doc. JVT-C151, Virginia, USA, Apr.2004.

 • [35] T. K. Tan, C. S. Boon, and Y. Suzuki, "Intra prediction by template matching",

IEEE International Conference. Image Processing (ICIP), vol.15, pp. 1693-1696, Oct. 2006.

[36] T.K. Tan et al, "Intra Prediction by Averaged Template Matching Predictors", IEEE. Consumer Communications and Networking Conference (CCNC), vol. 16, pp. 405-409, Jan 2007.

[37] V. Sze, M. Budagavi, “Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial),” IEEE International Symposium on Circuits and Systems (ISCAS), presented on June. 2014.

http://www.rle.mit.edu/eems/publications/tutorials/