Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding...
Transcript of Overview: image and video coding standards...Bernd Girod: EE398B Image Communication II Video Coding...
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 1
Overview: Video Coding Standards
Video coding standards: applications and common structureRelevant standards organizationsITU-T Rec. H.261 ITU-T Rec. H.263ISO/IEC MPEG-1 ISO/IEC MPEG-2ISO/IEC MPEG-4Recent progress: H.264/AVC
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 2
ISO/IEC MPEG
MPEG-1 Standard (1991) (ISO/IEC 11172)Target bit-rate about 1.5 MbpsTypical image format CIF, no interlaceFrame rate 24 ... 30 fpsMain application: video storage for multimedia (e.g., on CD-ROM)
MPEG-2 Standard (1994) (ISO/IEC 13818)Extension for interlace, optimized for TV resolution (NTSC: 704 x 480 Pixel)Image quality similar to NTSC, PAL, SECAM at 4 - 8 MbpsHDTV at 20 Mbps
MPEG-4 Standard (1999) (ISO/IEC 14496)Object based codingWide-range of applications, with choices of interactivity, scalability, error resilience, etc.
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 3
MPEG-1/2: GOP Structure
"Group of Pictures" = “GOP“, GOP structure is very flexible
I-Picture P-Picture P-Picture
B-Pictures
time
1 3 4 2 6 7 8 5
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 4
MPEG-1 Encoder
Pre-processing DCT Weighting Quantization VLC Video
multiplex BufferPicture reordering
Inverse quantization
Inverse weighting
Inverse DCT
Picture store 1
Picture store 2
Motion compensation+
-
+
1/2
zero
Motion vectors, macroblock info, start codesVideo in
Bitstream
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 5
MPEG-1: coding of I-pictures
I-pictures: intraframe coded8x8 DCTArbitrary weighting matrix for coefficientsDifferential coding of DC-coefficientsUniform quantizationZig-zag-scan, run-level-codingEntropy codingUnfortunately, not quite JPEG
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 6
MPEG-1: coding of P-pictures
Motion-compensated prediction from an encoded I-picture or P-picture (DPCM)Half-pel accuracy of motion compensation, bilinear interpolationOne displacement vector per macroblockDifferential coding of displacement vectorsCoding of prediction error with 8x8-DCT, uniform threshold quantization, zig-zag-scan as in I-pictures
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 7
MPEG-1: coding of B-pictures
Motion-compensated prediction from two consecutive P- or I-pictures
either• only forward prediction (1 vector/macroblock)
or• only backward prediction (1 vector/macroblock)
or• Average of forward and backward prediction = interpolation (2
vectors/macroblock)
Half-pel accuracy of motion compensation, bilinear interpolationCoding of prediction error with 8x8-DCT, uniform quantization, zig-zag-scan as in I-pictures
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 8
MPEG-2 vs. MPEG-1
Efficiently compress interlaced digital video at broadcast quality
Field/frame picturesChroma samplingNew prediction modesField/frame DCTAdditional scan patterns for DCT coefficientsMotion compensation with blocks of size 16x8 pels
Improved coding efficiency by different quantization, VLC tablesVarious scalability modes
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 9
Coding of Interlaced Video (1)
Frame and field picture structures
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 10
Coding of Interlaced Video (2)
Field prediction for field pictures
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 11
Coding of Interlaced Video (3)
Field prediction for frame pictures
16
16
16
8
8
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 12
Coding of Interlaced Video (4)
Dual prime for P pictures
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 13
Coding of Interlaced Video (5)
Field/frame DCTAlternate Scan
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 14
MPEG-4
Support highly interactive multimedia applications as well as traditional applicationsAdvanced functionalities: interactivity, scalability, error resilience…Coding of natural and synthetic audio and video, as well as graphicsEnable the multiplexing of audiovisual objects and composition in a scene
Video on LANs, Internet videoWireless videoVideo databaseInteractive home shoppingVideo e-mail, home moviesVirtual reality games, flight simulation, multi-viewpoint training
‘TV/film’AV-data
‘Computer’Interactivity
‘Telecom’Wireless
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 15
MPEG-4: Scene with audiovisual objects
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 16
MPEG-4: Video coding
Basic video codingDefinition of Video Object (VO), Video Object Layer (VOL), VideoObject Plane (VOP) Improved coding efficiency vs. MPEG-1/2
• Based on H.263 baseline• Global motion compensation• Sprites• Quarter pixel motion compensation
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 17
MPEG-4: Video coding
Object-based video codingBinary shape coding
α-map shape codingPadding for block-based DCT of textureShape-adaptive DCT
DWT for still texture codingMesh animation, face and body animation
Bernd Girod: EE398B Image Communication II Video Coding Standards: MPEG-1,2,4 no. 18
Shape Adaptive DCT