City University of Hong Kong Moving Picture Expert Group - Established in 1988 by the Joint ISO/IEC...

City University of Hong KongCity University of Hong KongCity University of Hong KongCity University of Hong Kong

Moving Picture Expert Group - Established in 1988 by the Joint ISO/IEC Technical Committee on IT.

Mission - To develop standards for coded representation of motion pictures and audio at a bit rate of up to 1.5Mb/s.

MPEG-1 was issued in 1992.

MPEG-2 (1994) - higher quality (not lower than NTSC and PAL) with bit rates between 2-10Mb/s.

Applications - Digital CATV and Terrestrial digital broadcasting distribution, Video recording and retrieval.

Lossy compression

Trade off image quality with bit rate according to objective or subjective criteria

Video sequences usually contains large statistical redundancies in both temporal and spatial directions

Intraframe coding

Interframe coding

Subsampling of Chrominance - Human eye is more sensitive to luminance than chrominance

Encoding of a single picture

Similar to JPEG

Discrete Cosine Transform- Converts spatial to frequency domain

Quantization of spectral coefficients

DPCM to encode DC terms

Zigzag scan to group zeros into long sequences, followed by run-length coding

Lossless, Variable Length Coding to encode AC coefficients

Remove temporal redundancies between frames

Use extensively in MPEG-1 and MPEG-2

Based on estimation of motion between video frames

Use of motion vectors to describe displacement of pixels from one frame to the next

Spatial correlation between motion vectors are high

One motion vector can represent the motion of a block of pixels.

Current framePrevious frame

Figure 1

For each image block in the current frame,

Find its nearest counterpart in the previous frame.

Record the displacement vector

Figure 2

Frame N-1 Frame N

SearchWindow

Previous BlockLocation

Current BlockLocation

Only the prediction error (residual) images are transmitted

Good prediction reduces information content in residual images

Partition the previous and the current images into non-overlapping square blocks of size NxN

Previous frame Current frame

e.g., N=8

Represent each block with a 2D matrix:

f(x,y) for previous frame

g(x,y) for current frame

yxgyxfN

gfD0 0

Difference between any two blocks is given by

yxgyxfN

gfD0 0

The lower the difference, the more similar is the pair of blocks

A motion vector is computed for ‘EVERY’ blocks in the current frame. HOW?

Each block in the current frame matched against all the blocks in the previous frame, the closest one is taken to be its counterpart.

MV=(-2,-3)x

MV=(-1,-3)x

MV=(-1,-2)x

The method is slow, especially if the image resolution and N are large.

A motion vector is computed for ‘EVERY’ blocks in the near neighborhood of the current frame.

For example, only the blocks that are adjacent to the current one is tested. The method is faster but the search area is restricted

Search Window

For example, only the blocks that are adjacent to the current one is tested. The method is faster but the search area is restricted

Assumption: changes between frames are small and are restricted within the search window.

Search Window

However, the search time is still long

Given a block in the current frame, search the best match in the previous frame along the vertical direction

Search Window

Search WindowBest

Given a block in the current frame, search the best match in the previous frame along the vertical direction

Search the best match in the previous frame along the horizontal direction

Search WindowSolution

Non-optimal solution with the assumption of smooth intensity distribution

Search the best match in the previous frame along the horizontal direction

Applications - multimedia and video transmission Based on JPEG and H.261 standards Flexible picture size and frame rate specified by

users Video source - Non-interlaced video signals. Minimum requirements on decoders

– resolution of 720X576– 30 frames/s – 1.86Mb/s

Applications - multimedia and video transmission Based on JPEG and H.261 standards Flexible picture size and frame rate specified by

users Video source - Non-interlaced video signals. Minimum requirements on decoders

– resolution of 720X576– 30 frames/s – 1.86Mb/s

Layer structure in MPEG bitstreamLayer structure in MPEG bitstream

Sequence Group Of Pictures (GOP) Picture Slice Macroblock Block

Video SequenceVideo Sequence

Group Of PicturesGroup Of Pictures

A PictureA Picture A SliceA Slice MacroblockMacroblock BlockBlock

Partitioning of images into Macroblocks (MB) Intraframe coding on one out of every K images Motion estimation on MBs Generate (K-1) predicted frames Encode residual error images Conditional Replenishment of Macroblocks

An image is partitioned into Macroblocks of size 16X16

1 MB = 4 luminance (Y) and 2 chrominance blocks (U,V)

The sampling ratio between Y, U and V is 4:1:1

An image is partitioned into Macroblocks of size 16X16

1 MB = 4 luminance (Y) and 2 chrominance blocks (U,V)

The sampling ratio between Y, U and V is 4:1:1

I P P PY1 Y2

Y:U:V = 4:1:11 2 3 4

I P P PY1 Y2

Y:U:V = 4:1:11 2 3 4

Figure 3

Chrominance format:

4:4:4, 4:2:2, 4:1:1, 4:2:0

Chrominance format:

4:4:4, 4:2:2, 4:1:1, 4:2:0Assuming 8bits for Y, U and V components

Assuming 8bits for Y, U and V components

4:4:44:4:44*8 (Y) + 4*8 (U) + 4*8 (V)

= 96 bits

4*8 (Y) + 4*8 (U) + 4*8 (V)

= 96 bits

Bits per pixel =

96/4 = 24 bpp

Bits per pixel =

96/4 = 24 bpp

Chrominance format:

4:4:4, 4:2:2, 4:1:1, 4:2:0

Chrominance format:

4:2:24:2:24*8 (Y) + 2*8 (U) + 2*8 (V)

= 64 bits

4*8 (Y) + 2*8 (U) + 2*8 (V)

= 64 bits

Bits per pixel =

64/4 = 16 bpp

Bits per pixel =

64/4 = 16 bpp

Chrominance format:

4:4:4, 4:2:2, 4:1:1, 4:2:0

Chrominance format:

4:1:14:1:14*8 (Y) + 1*8 (U) + 1*8 (V)

= 48 bits

4*8 (Y) + 1*8 (U) + 1*8 (V)

= 48 bits

Bits per pixel =

48/4 = 12 bpp

Bits per pixel =

48/4 = 12 bpp

Chrominance format:

4:4:4, 4:2:2, 4:1:1, 4:2:0

Chrominance format:

4:2:04:2:04*8 (Y) + 1*8 (U) + 1*8 (V)

= 48 bits

4*8 (Y) + 1*8 (U) + 1*8 (V)

= 48 bits

Bits per pixel =

48/4 = 12 bpp

Bits per pixel =

48/4 = 12 bpp

DCT Weighted (I-frame)/Uniform (P-frame) Quantization DPCM on DC terms Zigzag scan + runlength + VLC

Macroblock Block to beencoded

DPCMDC

ZigZagScanning

RunlengthEncoding

sz: Step Size

JPEG encoded DC

JPEG encoded ACFigure 4

Previous I or P frame is stored in both encoder and decoder

Motion Compensation is performed on a macroblock basis

One motion vector (mv) is generated for each macroblock

The mvs are coded and transmitted to the receiver

Previous I or P frame is stored in both encoder and decoder

Motion Compensation is performed on a macroblock basis

One motion vector (mv) is generated for each macroblock

The mvs are coded and transmitted to the receiver

Motion prediction error of pixels in each macroblock is calculated

Error blocks (size 8X8) are encoded in the same manner as those in the I-Picture

A video buffer plus step size adjustment maintain a constant target bit-rate

Motion prediction error of pixels in each macroblock is calculated

Error blocks (size 8X8) are encoded in the same manner as those in the I-Picture

A video buffer plus step size adjustment maintain a constant target bit-rate

x(n)Coder

Predictor

Decoder

epq(n)

e(n)• Current signal x(n) is predicted

from previous sample x(n-1). Predicted value is xp(n)

• Predicted error e(n)=x(n)-xp(n) is compressed (encode) and transmitted

• The encoded error is decoded and added back to xp(n) to reconstruct the current signal x(n). However there are loss in the codec and the reconstructed signal xr(n) is not identical to x(n)

• xr(n) is taken to predicted the next sample x(n+1)

Figure 5

Block to beencoded

RLCVLC

MC + FRAMESTORE

CONTROL

ENCODEDRESIDUAL

MOTIONVECTOR

I-Pictures are encoded independently I-Pictures can therefore be used as access point for

random access, fast-forward (FF) or fast-reverse (FR)

P-Pictures cannot be decoded alone, hence cannot be used as an access point

B-Pictures are constructed with the nearest I or P Pictures

Backward prediction requires the presence of the start and end frames, both can be used as access points

I-Pictures are encoded independently I-Pictures can therefore be used as access point for

random access, fast-forward (FF) or fast-reverse (FR)

P-Pictures cannot be decoded alone, hence cannot be used as an access point

B-Pictures are constructed with the nearest I or P Pictures

Backward prediction requires the presence of the start and end frames, both can be used as access points

Compression Random Access Coding DelayI Pictures only Low Highest LowI and P Pictures Medium Low MediumI, P and B Pictures High Medium High

Figure 6a

I P PB B B B

1 23 4 76 5Order of Coding

Figure 6b

I I II I I I

I I IP P P P

I I IP B P B

Compression Random Access Coding DelayI Pictures only Low Highest LowI and P Pictures Medium Low MediumI, P and B Pictures High Medium High

Only macroblocks that have been changed in the decoder are updated

Three types of MB are classified in MPEG standard Skipped MB - Zero motion vector, the MB is

neither encoded nor transmitted Inter MB - Motion Prediction is valid, the MB type

and address, motion vector and the coded DCT coefficients are transmitted

Intra MB - Encoded DCT coefficients of the MB are transmitted. No Motion Compensation is used

Only macroblocks that have been changed in the decoder are updated

Three types of MB are classified in MPEG standard Skipped MB - Zero motion vector, the MB is

neither encoded nor transmitted Inter MB - Motion Prediction is valid, the MB type

and address, motion vector and the coded DCT coefficients are transmitted

Intra MB - Encoded DCT coefficients of the MB are transmitted. No Motion Compensation is used

Pred-mcq

Pred-mc

Pred-m

Intra-q

Intra-d

Pred-cq

Pred-c

Skipped

Pred-mcq

Pred-mc

Pred-m

Intra-q

Intra-d

Pred-cq

Pred-c

Skipped

Non-zero motion vector,

Error coded with defined quantization

Pred-mcq

Pred-mc

Pred-m

Intra-q

Intra-d

Pred-cq

Pred-c

Skipped

Error coded with default quantization

Pred-mcq

Pred-mc

Pred-m

Intra-q

Intra-d

Pred-cq

Pred-c

Skipped

Error not coded

Pred-mcq

Pred-mc

Pred-m

Intra-q

Intra-d

Pred-cq

Pred-c

Skipped

Macroblock intra-coded with defined quantization

Pred-mcq

Pred-mc

Pred-m

Intra-q

Intra-d

Pred-cq

Pred-c

Skipped

MBMacroblock intra-coded with default quantization

Pred-mcq

Pred-mc

Pred-m

Intra-q

Intra-d

Pred-cq

Pred-c

Skipped

MB MV = 0 (not predicted)

Error coded with defined quantization

Pred-mcq

Pred-mc

Pred-m

Intra-q

Intra-d

Pred-cq

Pred-c

Skipped

MV = 0 (not predicted)

Error coded with default quantization

Pred-mcq

Pred-mc

Pred-m

Intra-q

Intra-d

Pred-cq

Pred-c

Skipped

Macroblock copied from predictor picture

Pred-f/b/i cq

Pred-f/b/i c

Skipped Pred-f/b/i

Intra-q

Intra-d

DecodedBlock

RLDVLD

MC+FRAMESTORE

MOTIONVECTOR

ENCODEDDCT DATA

Note: MC for P-Frames only

Reverse process of the encoderReverse process of the encoder

Figure 7

A superset of MPEG-1 and backward compatible to the latter

Support interlaced video signals Scalable video-coding property, can be decoded by

receivers with different capabilities Permits partial implementation defined by Profiles

and Levels A Profile defines a new set of algorithms added as a

superset to the algorithms in the profile that follows A Level specifies the range of parameters supported

by the implementation

A superset of MPEG-1 and backward compatible to the latter

Support interlaced video signals Scalable video-coding property, can be decoded by

receivers with different capabilities Permits partial implementation defined by Profiles

and Levels A Profile defines a new set of algorithms added as a

superset to the algorithms in the profile that follows A Level specifies the range of parameters supported

by the implementation

LEVEL Pel/Line Pels/Frame Frame Rate (f/s) Bit-rate (Mb/s)High 1920 1152 60 80

High 1440 1440 1152 60 60Main 720 576 30 15Low 352 288 30 4

PROFILE

Non-scalable Coding

Interlaced Video

B-Picture

4:2:0 YUV format

2 layers SNR ScalableCoding

MAIN +SNR

SCALABLE

2 layers SpatialScalable Coding

+SPATIAL

SCALABLESNR

SCALABLE4:0:0 YUV format

3 layers SNR andSpatial ScalableCoding+HIGH

SPATIALSCALABLE

4:2:2 YUV format

B-Picture PredictionMAIN -SIMPLE

Figure 8

Main Profile: MPEG-2 Non-scalable Coding Mode

A straightforward extension of MPEG-1 to accomodate interlaced video signals

Field/Frame MacroblocksTwo types of prediction

Frame Prediction: Prediction based on one or more previously decoded frames

Field Prediction : Prediction of individual field based on one or more previously decoded field

Main Profile: MPEG-2 Non-scalable Coding Mode

A straightforward extension of MPEG-1 to accomodate interlaced video signals

Field/Frame MacroblocksTwo types of prediction

Frame Prediction: Prediction based on one or more previously decoded frames

Field Prediction : Prediction of individual field based on one or more previously decoded field

Figure 9a

The four sub-blocks of a Frame

Macroblock

A stationary scene image

Figure 9b

Macroblock

A moving scene image

The object shape is changed with motion because of the interlacing mechanism

Same object may appear different in successive frames because of the above reason - prediction is not accurate

Simple image patterns may become complicated

More AC coefficients are required to describe each component in the frame Macroblock

The object shape is changed with motion because of the interlacing mechanism

Same object may appear different in successive frames because of the above reason - prediction is not accurate

Simple image patterns may become complicated

More AC coefficients are required to describe each component in the frame Macroblock

Figure 9c

Macroblock

A moving scene image

Compute Field-based Variance Compute Frame-based Variance If Field-based Variance < Frame-based Variance

MB coded with Field-based DCT

var1=0;

for(m=0;m<COL;m++)

for(n=0;n<ROW-2;n++)

D1=x(m,n)-x(m,n+1);

D2=x(m,n+1)-x(m,n+2)

var1+=(D1*D1)+(D2*D2);

var1=0;

for(m=0;m<COL;m++)

for(n=0;n<ROW-2;n++)

D1=x(m,n)-x(m,n+2);

D2=x(m,n+1)-x(m,n+3)

var1+=(D1*D1)+(D2*D2);

A top field is predicted from either previously coded top or bottom field with Motion Compensation (MC)

Bottom fields are predicted from previously coded top field with MC

Combine frame and field prediction is used in MPEG-2

A top field is predicted from either previously coded top or bottom field with Motion Compensation (MC)

Bottom fields are predicted from previously coded top field with MC

Combine frame and field prediction is used in MPEG-2

Provide interoperability between different services and systems

Base layer - encodes downscaled video at reduced bitstream

Enhance layer - encodes the difference between original signal and the upscaled base-layer video

Provide interoperability between different services and systems

Base layer - encodes downscaled video at reduced bitstream

Enhance layer - encodes the difference between original signal and the upscaled base-layer video

ENHANCEMENTENCODER

UPSCALING

BASE LAYERENCODER

DOWNSCALING

Video in

Enhancementlayer bitstream

Basic layerbitstream

ENHANCEMENTENCODER

UPSCALING

BASE LAYERENCODER

DOWNSCALING

Video in

Figure 10

ENHANCEMENTENCODER

UPSCALING

BASE LAYERENCODER

DOWNSCALING

Video in

ENHANCEMENTENCODER

UPSCALING

BASE LAYERENCODER

DOWNSCALING

Video in

A 2-layer DCT, VLC and MC encoder

Both layers encoded video signal at the same resolution

Base layer - DCT coefficients are coarsely quantized and is protected from transmission error

Enhancement layer: DCT coefficients are finely quantized and their difference with the base layer is transmitted

A 2-layer DCT, VLC and MC encoder

Both layers encoded video signal at the same resolution

Base layer - DCT coefficients are coarsely quantized and is protected from transmission error

Enhancement layer: DCT coefficients are finely quantized and their difference with the base layer is transmitted

x Quantizer

Quantized level L

De-quantizer

xQ=s * L

Step size s

x Quantizer

Quantized level L

De-quantizer

xQ=s * L

Step size s

Quantization error

x Quantizer

Quantized level L

De-quantizer

xQ=S * L

Coarse step size S

Quantization error E

Quantizer De-quantizer

Fine step size s

EQ=s * LQ

EQ can be used to compensate error in xQ

VBImageBlock Base Layer

Bit Stream

EnhancementLayer BitStream

Base LayerBit Stream

EnhancementLayer BitStream

ImageBlocks

Figure 11

Temporal prediction from previous frame

Estimation of mv of lost MB from neighbouring Mbs

Add mvs to MBs of I-Frame for error concealment

2 Layer Coding using Data Partitioning, Spatial and Frequency scalability

Temporal prediction from previous frame

Estimation of mv of lost MB from neighbouring Mbs

Add mvs to MBs of I-Frame for error concealment

2 Layer Coding using Data Partitioning, Spatial and Frequency scalability

Applications of MPEG Encoding StandardCable and Interactive TV DistributionSatellite and Digital Terrestrial TVBroadcastingRemote SurveillanceVideo Conferencing/TelephonyStandalone or Computer Based MultimediaSystemsHDTVDVDVCDDigital Camera

City University of Hong Kong Moving Picture Expert Group - Established in 1988 by the Joint ISO/IEC...

Documents

Transcript of City University of Hong Kong Moving Picture Expert Group - Established in 1988 by the Joint ISO/IEC...

IntroductiontMyn1 Introduction MPEG, Moving Picture Experts Group was started in 1988 as a working group within ISO/IEC with the aim of defining standards.

IS/IEC 519-1 (1984): Safety in Electroheat Installation ... · medium frequency induction furnace installations IEC Pub 519-3 (1988) Safety in electroheat installation - Part ...

IS/QC 250301 (1988): Inductor and Transformer Cores for … · 2018-11-15 · IS QC 250301 : 1988. IEC QC 250301 :I987 3. Marking 3.1 Core sets (see IEC Publication 723-4, Sub-clause

web.csulb.eduweb.csulb.edu/divisions/aa/grad_undergrad/senate/committees/iec/...Conservatorio Superior de Musica de Murcia—Spain (Gerry Brondial, Music) City University of Hong Kong

Nederlandse EN 61330 - nen.nl · IEC 664-1:1992 IEC 694:1980 IEC 726(mod):1982 IEC 905:1987 IEC 947-1:1988 ... I EC 551 1987 Determination of transformer and reactor (mod) sound levels

TR 102 520 - V1.1.2 - Human Factors (HF); Access symbols ......[9] IEC 60416 (1988): "General principles for the formulation of graphical symbols". [10] IEC 60417-1 (1988): "Graphical

cosgra sa - cranetex.com · iec 60034-2-1 iec 60034-5 iec 60034-6 iec 60034-7 iec 60034-8 iec 60034-9 iec 60034-11 iec 60034-12 iec 60034-14 iec 60034-30 iec 60085 iec 60038 iec 60072

IEC 62116,IEC 61727,IEC 61683,IEC 60068(1,2,14,30),IEC ... · 5 years IEC 62116,IEC 61727,IEC 61683,IEC 60068(1,2,14,30),IEC 62109-1/2 100 W W W W W W ARPC Anti Reverse Power Controller(optional)

2012 - pcpartner.com · of the Hong Kong Chinese General Chamber of Commerce (since 1988), director of Ocean Park Corporation (March 2006 to February 2012), member of Hong Kong Housing

HONG KONG LEGISLATIVE COUNCIL-21 April 1988 OFFICIAL ... · The Council will now resume and we will continue with the debate on the Appropriation Bill 1988. MR. PANG (in Cantonese):

HONG KONG LEGISLATIVE COUNCIL-20 April 1988 OFFICIAL … · 2005. 8. 11. · hong kong legislative council-20 april 1988 1097official report of proceedings wednesday, 20 april 1988

AC switches DC switches - Balaji Electricalsbalajielectricals.com/wp-content/themes/balaji/pdf/rotaryswitches.pdf · European : IEC-60947-1 : 1988 Canadian : CSA 22.2 No.14-1991 American

IEC 309.1-1988

T gri d con - Welcome to EWA Website 60364-1 IEC 60364-4 IEC 60364-4 IEC 60364-4 IEC 60364-4 IEC 60364-5 IEC 60364-5 IEC 60364-5 IEC 60364-6 IEC 60364-7 IEC 62548 IEC 62446-1 IEC 61724-1

TECHNICAL SPECIFICATION FOR CONTROL AND POWER CABLES … · 2018. 10. 27. · IEC 502 IS-7098:1985 (part 2) LT and 3.3 - 33kVXLPE cables IEC 502 IS - 1554:1988 (part 1) PVC Cables

TEST REPORT MEDICAL ELECTRICAL EQUIPMENTsupport.elmark.com.pl/advantech/download/ppc-153m... · Tested according to IEC 60601 -1 (2 ed 1988) + Amend. 1 (1991) + Amend. 2 (1995) +

21294 GLOW-STARTERS FOR FLUORESCENT LAMPSpontofocal... · IEC 60901: 1987, Single-capped fluorescent lamps - Safety and performance requirements IEC 60921: 1988, Ballasts for tubular

Gap Analysis Between the Second and Third Editions of IEC ... · IEC 60601-1 3rd edition cancels and replaces the second edition published in 1988, its Amendment 1 (1991) and Amendment

Unaudited Consolidated Interim Financial …...Director, CFO Asia Pacific Region, Hong Kong (1992-1997) Various other senior positions (1988-1992) 1987-1988 Prudential Bache Securities

Waste Management...Vibration IEC 68.2.6,shock IEC 68.2.29, Impact IEC 62262-IK09 Vibration IEC 68.2.6, Shock IEC 68.2.29, Impact IEC 62262-IK07 Vibration IEC 68.2.6, shock IEC 68.2.29