Digital Multimedia, 2nd edition Nigel Chapman & Jenny Chapman Chapter 7 This presentation © 2004,...
-
Upload
arthur-chandler -
Category
Documents
-
view
229 -
download
5
Transcript of Digital Multimedia, 2nd edition Nigel Chapman & Jenny Chapman Chapter 7 This presentation © 2004,...
Digital Multimedia, 2nd editionNigel Chapman & Jenny Chapman
Chapter 7
This presentation © 2004, MacAvon Media Productions
Video
© 2004, MacAvon Media Productions
7
• PAL uncompressed
• 768x576 pixels per frame
• x 3 bytes per pixel (24 bit colour)
• x 25 frames per second
• ≈ 31 MB per second
• ≈ 1.85 GB per minute
Data Size
191–192
© 2004, MacAvon Media Productions
7
• NTSC uncompressed
• 640x480 pixels per frame
• x 3 bytes per pixel (24 bit colour)
• x 30 frames per second (approx)
• ≈ 26 MB per second
• ≈ 1.6 GB per minute
Data Size
191–192
© 2004, MacAvon Media Productions
7
• In the camera – DV + Firewire
• In the computer – video capture card
• Digitization in camera (DV) means less noise
• Less noise allows better compression
Digitization
193–195
© 2004, MacAvon Media Productions
7
• Play back a video stream as it arrives over a network (like broadcast TV), instead of downloading an entire video clip and playing it from disk (like renting a DVD)
Streamed Video
197
© 2004, MacAvon Media Productions
7
• Start playing a downloaded clip as soon as enough of it has arrived
• Starts when the (estimated) time to download the rest is equal to the duration of the clip
HTTP Streaming
198
© 2004, MacAvon Media Productions
7
• Digital video devices must conform to standards
• Digital standards must maintain compatibility with older analogue standards for broadcast TV
Video Standards
199
© 2004, MacAvon Media Productions
7
• Required for TV, so encountered in captured footage
• Each frame is divided into two fields
• Field 1: odd lines; Field 2: even lines
• Fields are transmitted one after the other
• Frame is built out of the interlaced fields
Interlacing
200
© 2004, MacAvon Media Productions
7
• PAL (Phase Alternating Line)
• Western Europe, Australia & New Zealand, China,…
• NTSC (National Television Standards Committee)
• North America, Japan, Taiwan, parts of South America,…
Analogue Standards
199
© 2004, MacAvon Media Productions
7
• SECAM (Séquential Couleur avec Mémoire)
• France and former Soviet Union
• Standard only used for transmission
• Uses PAL cameras etc
Analogue Standards
199
© 2004, MacAvon Media Productions
7
• Frame has 625 lines, 576 are picture
• 25 frames (50 fields) per second
•NTSC• Frame has 525 lines, 480 are picture
• 29.97 frames (59.94 fields) per second
• (Often quoted as 30 frames per second)
PAL
202
© 2004, MacAvon Media Productions
7
• Digital video standard, properly called Rec. ITU-R BT.601
• 720 luminance samples (Y), 2x360 colour difference samples (B−Y and R−Y) per line
• PAL 720x576 pixels; NTSC 720x480 pixels
• Pixels are not square
CCIR 601
202–203
© 2004, MacAvon Media Productions
7
• Used in CCIR 601
• Twice as many Y samples as each of the colour difference samples
• Co-sited: same pixel is used for all three samples
• Reduces data rate to just over 20MB per second
4:2:2 sub-sampling
203
© 2004, MacAvon Media Productions
7
• Consumer format, also known as mini-DV
• DVCAM, DVPRO use different tape formats, but generate the same data stream
• 4:1:1 chrominance sub-sampling
• Data rate constant 25Mbits per second
• Compression ratio 5:1
DV
204
© 2004, MacAvon Media Productions
7
• ISO/IEC Motion Picture Experts Group
• Series of standards including
• MPEG-1 intended for video CD
• MPEG-2 used in DVD and broadcast
• MPEG-4 for low bitrate multimedia
MPEG
204–206
© 2004, MacAvon Media Productions
7
• Profiles define subsets of the features of the data stream
• Levels define parameters such as frame size and data rate
• Each profile may be implemented at one or more levels
• Notation: profile@level, e.g. MP@ML
MPEG Profiles & Levels
204–205
© 2004, MacAvon Media Productions
7
• MPEG-2 Main Profile at Main Level (MP@ML) used for DVD video
• CCIR 601 scanning
• 4:2:0 chrominance sub-sampling
• 15 Mbits per second
• Most elaborate representation of MPEG-2 compressed data
MPEG-2 Profiles & Levels
205
© 2004, MacAvon Media Productions
7
• Designed to support a range of multimedia data at bit rates from 10kbps to >1.8Mbps
• Applications from mobile phones to HDTV
• Video codec becoming popular for Internet use, is incorporated in QuickTime, RealMedia and DivX
MPEG-4
205–206
© 2004, MacAvon Media Productions
7
• Visual Simple Profile (SP), suitable for low bandwidth streaming over Internet
• Visual Advanced Simple Profile (ASP) suitable for broadband streaming
• SP@L1 (Level 1 of Simple Profile), 64 kbps, 176x144 pixel frame
• ASP@L5, 8000 kbps, full CCIR 601 frame
MPEG-4 Profiles & Levels
205
© 2004, MacAvon Media Productions
7
• Spatial (intra-frame) compression
• Compress each frame in isolation, treating it as a bitmapped image
• Temporal (inter-frame) compression
• Compress sequences of frames by only storing differences between them
• Always some compression because of sub-sampling
Video Compression
206–208
© 2004, MacAvon Media Productions
7
• Image compression applied to each frame
• Can therefore be lossless or lossy, but lossless rarely produces sufficiently high compression ratios for volume of data
• Lossless compression implies a loss of quality if decompressed then recompressed
• Ideally, work with uncompressed video during post-production
Spatial Compression
207
© 2004, MacAvon Media Productions
7
• Key frames are spatially compressed only
• Key frames often regularly spaced (e.g. every 12 frames)
• Difference frames only store the differences between the frame and the preceding frame or most recent key frame
• Difference frames can be efficiently spatially compressed
Temporal Compression
207–208
© 2004, MacAvon Media Productions
7
• Purely spatial compression
• Apply JPEG to each frame
• Used by most analogue capture cards
• No standard, but MJPEG-A format widely supported
Motion JPEG
209–210
© 2004, MacAvon Media Productions
7
• Starts with chrominance sub-sampling of CCIR 601 frame
• Constant data rate 25Mbits per second
• Higher quality than MJPEG at same rate
• Apply DCT, quantization, run-length and Huffman coding on zig-zag sequence – like JPEG – to 8x8 blocks of pixels
DV Compression
210–211
© 2004, MacAvon Media Productions
7
• If little or no difference between fields (almost static frame), apply DCT to block containing alternate lines from odd and even fields
• If motion between fields, apply DCT to two 8x4 blocks (one from each field) separately, leading to more efficient compression of frames with motion
DV Compression
210–211
© 2004, MacAvon Media Productions
7
• Shuffling
• Construct video segments by taking 8x8 blocks from five different areas of the frame, to ‘average’ amount of detail
• Calculate coefficients for whole video segment, making more efficient use of available bytes
DV Compression
210–211
© 2004, MacAvon Media Productions
7
• Spatial compression based on quantization and coding of DCT coefficients
• Temporal compression based on motion compensation
• Record displacement of object plus changed pixels in area exposed by its movement
MPEG-1 Compression
211–212
© 2004, MacAvon Media Productions
7
• I-pictures purely intra-frame compressed
• P-pictures ‘predictive’
• Difference frames based on earlier I- or P-pictures
• B-pictures ‘bi-directionally predictive’
• Difference frames based on preceding and following I- or P-pictures
MPEG-1 Compression
212–213
© 2004, MacAvon Media Productions
7
• Group of Pictures (GOP)
• Repeating sequence of I-, P- and B-pictures
• Always begins with an I-picture
• Display order – frames in order they will be displayed
• Bitstream order – re-ordered so that every P- or B-picture comes after frames it depends on, allowing reconstruction of the complete frames
MPEG-1 Compression
213–214
© 2004, MacAvon Media Productions
7
• Source Input Format (SIF)
• 4:2:0 chroma sub-sampled
• 352x240 pixel frame
• MPEG-1 compressed SIF video at 30 frames per second has data rate of 1.86Mbits per second (CD video – 40mins of video at that rate)
• MPEG-1 can be scaled up to larger frames, but cannot handle interlacing
MPEG-1 Compression
214
© 2004, MacAvon Media Productions
7
• Standard defines an encoding for multimedia streams made up of different sorts of object – video, still images, animation, 3-D models…
• Higher profiles divide a scene into arbitrarily shaped video objects each one may be compressed and transmitted separately, scene is composed at the receiving end by combining them
• SP and ASP restricted to rectangular objects, usually complete frames
MPEG-4
215
© 2004, MacAvon Media Productions
7
• Refinement of MPEG-1 compression
• I-pictures compressed by quantizing and Huffman coding DCT coefficients
• Improved motion compensation leads to better quality than MPEG-1 at same bit rates
MPEG-4 Compression
215
© 2004, MacAvon Media Productions
7
• Simple Profile
• P-pictures only
• Efficient decompression, suitable for PDAs etc
• Advanced Simple Profile (ASP)
• B-pictures
• Global Motion Compensation
• Sub-pixel motion compensation
MPEG-4 Compression
215–216
© 2004, MacAvon Media Productions
7
• Cinepak – Longest established, high compression ratio, takes much longer to compress than to decompress
• Intel Indeo – Similar to Cinepak, but roughly 30% faster compression
• Sorenson – More recent, higher quality and better compression ratios than other two
• All three based on vector quantization
• Quality of all three inferior to MPEG-4
Older Codecs
216–219
© 2004, MacAvon Media Productions
7
• Divide each frame into small rectangular blocks (’vectors’)
• Code Book – collection of constant vectors representing typical patterns (edges, textures, flat colour,…)
• Compress by replacing each vector in image by index of vector from code book that most closely resembles it
Vector Quantization
216
© 2004, MacAvon Media Productions
7
• Making a constructed whole from a collection of parts
• Selection, trimming and organization of raw footage
• Apply transitions (e.g. dissolves) between shots
• Combination of picture with sound
• No changes made to the footage itself
Editing
223–230
© 2004, MacAvon Media Productions
7
• Changing or adding to the material
• Most changes are generalizations of image manipulation operations (e.g. colour correction, blurring and sharpening,…)
• Compositing – combining elements from different shots into a composite sequence
• Animating elements and combining animation with live action
Post-Production
230–236
© 2004, MacAvon Media Productions
7
• Compromises required to bring resource requirements of video within capabilities of delivery media (e.g. networks) and low-end machines
• Reduce frame size (e.g. downsample to quarter frame)
• Reduce frame rate (12fps is OK for smooth motion, flicker not a problem on computer)
• Reduce colour depth
Preparing for Delivery
236–237