Video Compress

ionMPEGMoving Picture Experts Group

What defines good video quality?

• Size of pictures

• Bitrate of channel medium (especially in real-time applications)

• Resolution of the original images and/or frames

• Frame rate of source AND frame rate of reproduction medium (24 frames per second is standard movie quality)

If any one of these factors is inferior, it can bottleneck the overall system and cause reduction

in video quality:

GIGO garbage in = garbage out !!!

MPEG Types• MPEG 1 :

• For lower resolution images• Bit streams upto 1.5Mbps

• MPEG 2:• Support higher resolution images• Bit streams of upto 15Mbps

• MPEG 4:• Support interactivity• Composed of tools, algorithms, profiles, MSDL

(MPEG Syntax Description Language)

RequirementsRandom Access, Reverse, Fast Forward, Search

At any point in the stream (within ½ second)Can reduce quality somewhat during this task, if needed

Audio/Video Synchronization

Robustness to errorsNot catastrophic if some bits are lostLends itself to Internet streaming

Coding/Decoding delay under 150 msFor interactive applications

Ability to EditModify/Replace frames

Basics of MPEGMany “moving” images or image sequences consist of a static background with one or more moving foreground objects. We can get coding advantage from this.

we code the first frame by baseline JPEG and use this frame as reference image.

Treat the second image block by block and compare each block with the same block in the reference image.

For blocks that have identical block in reference image, we only send a special code instead of whole code.

For other blocks, we just encode them as usual.

Basics of MPEGMotion Vectors

static background is a very special case, we should consider the displacement of the block.

Motion vector is used to inform decoder exactly where in the previous image to get the data.

Motion vector would be zero for a static background.

Basics of MPEGBlock Matching--how to find the matching block?

Matching criteria:In practice we couldn’t expect to find the exactly identical matching block, instead we look for close match.Most motion estimation schemes look for minimum mean square error(MMSE) between block.

Matching block size:How large the matching block will affect coding efficiencyblock size MPEG used: 16×16

N

nnn yxIyxI

NMSE

1

2' )),(),((1

Basics of MPEGSearch range:

It’s reasonable to consider an displacement of 360 pixles/s or about 60pixels/image in standard-definition television.In real-world scenes there is usually more or faster motion horizontally than vertically, generally the width of search area should be twice the height.Suggested search range: ±60 pixles × 30 pixles

MPEG CompressionCompression through

Spatial

Temporal

Spatial RedundancyTake advantage of similarity among most neighboring pixels

Loss of Resolution

Original (63 kb)

Low (7kb)

Very Low (4 kb)

QuestionWhen may spatial redundancy reduction be ineffective?

What kinds of images/movies?

AnswerWhen may spatial redundancy elimination be ineffective?

High-resolution images and displaysMay appear ‘coarse’

What kinds of images/movies?A varied image or ‘busy’ scene

Many colors, few adjacent

Temporal RedundancyTake advantage of similarity between successive frames

950 951 952

Temporal Redundancy Reduction

Motion estimation for different frames

X Z

Y

Available from earlier frame (X)

Available from later frame (Z)

QuestionWhen may temporal redundancy reduction be ineffective?

AnswerWhen may temporal redundancy reduction be ineffective?

Many scene changes

High motion

Typical Compression Techniques

Within-the-frame coding techniques

Between-the-frames coding techniquesBlock match motion compensation

Image stitching

Background sprite coding

Basics of MPEGTypes of pictures

I (intra) framecompressed using only intraframe coding

Moderate compression but faster random access

P (predicted) frameCoded with motion compression using past I frames or P frames

Can be used as reference pictures for additional motion compensation

B (bidirectional) frameCoded by motion compensation by either past or future I or P frames

D (DC) frameLimited use: encodes only DC components of intraframe coding

I IB P B P BB

P I

Bi-directional prediction

Unidirectional prediction

MPEG Compression Block Diagram

Framebeing coded

DCT QuantizeVariablelengthcode

Referenceframe

InverseDCT

De-quantize

Totransmitbuffer

A typical group of pictures in display order

A typical group of pictures in coding order

1 5 2 3 4 9 6 7 8 13 10 11 12

I P B B B P B B B P B B B

I B B B P B B B P B B B P

MPEG Compression

Translate the RGB system into a YUV system

Human perception is less sensitive to chrominance than to brightness

Translate brightness into chrominance and then the resolution does not have to be as good lower necessary bitrate

Coloring Scheme:

Normal JPEG Coloring Blocks

Red

Blue

Green

Yellow

Cr

Cb

Cg

Translation formulas:Y = Wr*R + Wb*B + Wg*GCr = Wr* (R - Y)Cb = Wb* (B - Y)Cg =Wg* (G - Y)

Image Sources Intra coding of macroblocks– just as what JPEG does– MPEG has two default quantization tables, one for

intra coding, another one for non-intra coding of residuals

Macroblock: composed of six blocks (4:2:0 or 4:1:1 format)• Four blocks of yellow (luminance)

• One block of Cb (chrominance)

• One block of Cr (chrominance)

Coding of Macroblock

Non-intra coding of macroblocks

The first step is to intra code the macroblock--just in case if we fail to find a reasonable match in motion estimation.

Then we use motion estimation to find the nearest match and get the motion vector. Only luminance samples are used in motion estimation.

Then each DCT block in macroblock will be treated separately. The residuals will be encode by DCT and quantization (use flat table) as in intra coding. DC along with AC

This process is applied to all six blocks in the macroblock

Motion vectors are coded predictively

Coding of Macroblock

P-framesIf the block can be skipped, we just send a “skip” codeotherwise, we compare the number of total bits of inter and intra coding, choose the more efficient one. Mark this block accordingly.

B-framescomparison among three methods of encoding

Coding of Macroblock

Non-intra coding of macroblocks

The first step is to intra code the macroblock--just in case if we fail to find a reasonable match in motion estimation.

Then we use motion estimation to find the nearest match and get the motion vector. Only luminance samples are used in motion estimation.

Then each DCT block in macroblock will be treated separately. The residuals will be encode by DCT and quantization (use flat table) as in intra coding. DC along with AC

This process is applied to all six blocks in the macroblock

Motion vectors are coded predictively

Coding of Macroblock

Frame

recorder

DCT

Quantize

Variable-length

coder

Transmit

buffer

Prediction

encode

r

De-

quantize

Inverse

DCT

Motion

predictor

Reference

frame

Rate

controllerIN OUT

Scale

factor

Buffer

fullness

Prediction

Motion vectors

DC

MPEG: Structure of the Coded Bit-Stream

Sequence layer: picture dimensions, pixel aspect ratio, picture rate, minimum buffer size, DCT quantization matrices

GOP layer: will have one I picture, start with I or B picture, end with I or P picture, has closed GOP flag, timing info, user data

Picture layer: temporal ref number, picture type, synchronization info, resolution, range of motion vectors

Slices: position of slice in picture, quantization scale factor

Macroblock: position, H and V motion vectors, which blocks are coded and transmitted

GOP-1 GOP-2 GOP-n

I B B B P B B..

Slice-1

Slice-2

…

Slice-N

mb-1mb-2 mb-n

0 12 3 4 5

Sequence layer

GOP layer

Picture layer

Slice layer

Macroblock layer

8x8 block

MPEG1 Audio Encoding Process

1. Break the signal into frames (a frame is usually several milliseconds in length)

2. Determine spectral energy distribution by taking the DFT, then break this distribubtion into sub-bands via filtering

3. Consider the specified bit rate (this will determine the number of bits per frame that can be used)

4. Compare the DFT signal to the human psychoacoutic model, perform masking

5. Use a modified Huffman code to achieve compression

6. After each frame has been compressed, place a header on each frame

7. Reassemble the new frames with headers back into a continuous bit stream

Image Sourceshttp://lea.hamradio.si/~s51kq/subsample.gif

http://www.scantips.com/basics9jb.html

http://en.wikipedia.org/wiki/JPEG

http://www.seas.gwu.edu/~ayoussef/cs225/transforms.html

Lots other. Sorry I forgot to save the sources.