CS 414 - Spring 2011
CS 414 – Multimedia Systems Design Lecture 5 – Digital Video Representation
Klara Nahrstedt
Spring 2011
CS 414 - Spring 2011
Administrative MP1 is out (January 28) Deadline of MP1 is February 16
Wednesday midnight – submit via compass Help-session for MP1 – Monday, February
2, 7pm , Room: 1111 SC
Outline
Human Visual System Visual Perception
ResolutionBrightnessTemporal Resolution
Television 3D Video
CS 414 - Spring 2011
Color and Visual System Color refers to how we
perceive a narrow band of electromagnetic energy source, object, observer
Visual system transforms light energy into sensory experience of sight
Human Visual System
Eyes, optic nerve, parts of the brain
Transforms electromagnetic energy
Human Visual System
Image Formation cornea, sclera, pupil,
iris, lens, retina, fovea Transduction
retina, rods, and cones Processing
optic nerve, brain
Retina and Fovea
Retina has photosensitive receptors at back of eye
Fovea is small, dense region of receptors only cones (no rods) gives visual acuity
Outside fovea fewer receptors overall larger proportion of rods
Fovea
Retina
Transduction (Retina)
Transform light to neural impulses
Receptors signal bipolar cells
Bipolar cells signal ganglion cells
Axons in the ganglion cells form optic nerve
RodsBipolar cellsConesGanglion
Optic nerve
Rods vs Cones
Contain photo-pigment Respond to low energy Enhance sensitivity Concentrated in retina,
but outside of fovea One type, sensitive to
grayscale changes
Contain photo-pigment Respond to high energy Enhance perception Concentrated in fovea,
exist sparsely in retina Three types, sensitive to
different wavelengths
Cones Rods
CS 414 - Spring 2011
Tri-stimulus Theory 3 types of cones (6 to 7 million of them)
Red = L cones, Green = M cones, Blue = S cones Ratio differentiates for each person E.g., Red (64%), Green (32%), rest S cones E.g., L(75.8%), M(20%), rest S cones E.g., L(50.6%), M(44.2%), rest S cones Source of information:
See ‘cone cell’ in wikipedia www.colorbasics.com/tristimulus/index.php
Each type most responsive to a narrow band red and green absorb most energy, blue the least
Light stimulates each set of cones differently, and the ratios produce sensation of color
Color Perception (Color Theory) Hue
distinguishes named colors, e.g., RGB
dominant wavelength of the light
Saturation Perceived intensity of a specific
color how far color is from a gray of
equal intensity Brightness (lightness)
perceived intensity
CS 414 - Spring 2011
Hue Scale
Saturation
Original
lightness
Source: Wikipedia
Visual Perception: Resolution and Brightness Spatial Resolution (depends
on: ) Image size Viewing distance
Brightness Perception of brightness is higher
than perception of color Different perception of primary
colors Relative brightness:
green:red:blue=59%:30%:11%
B/W vs. Color
CS 414 - Spring 2011Source: wikipedia
Visual Perception: Temporal Resolution
CS 414 - Spring 2011
Effects caused by inertia of human eye Perception of 16 frames/second as continuous
sequence Special Effect: Flicker
Temporal Resolution
FlickerPerceived if frame rate or refresh rate of
screen too low (<50Hz)Especially in large bright areas
Higher refresh rate requires:Higher scanning frequencyHigher bandwidth
CS 414 - Spring 2011
Visual Perception Influence
Viewing distance Display ratio (width/height – 4/3 for
conventional TV) Number of details still visible Intensity (luminance)
CS 414 - Spring 2011
Television History
1927, Hoover made a speech in Washington while viewers in NY could see, hear him
AT&T Bell Labs had the first “television” 18 fps, 2 x 3 inch
screen, 2500 pixels
Television Concepts Production (capture)
2D array of light energy to electrical signals signals must adhere to known, structured formats
Representation and Transmission popular formats include NTSC, PAL, SECAM, HDTV
Re-construction CRT technology and raster scanning display issues (refresh rates, temporal resolution) relies on principles of human visual system
CS 414 - Spring 2011
Video Representations
CompositeNTSC - 6MHz (4.2MHz video), 29.97 fpsPAL - 6-8MHz (4.2-6MHz video), 25 fps
ComponentMaintain separate signals for color
Color spacesRGB, YUV, YCRCB, YIQ
CS 414 - Spring 2011
Color Coding: YUV PAL video standard
Y is luminance UV are chrominance
YUV from RGB Y = .299R + .587G + .114B
U = 0.492 (B - Y)V = 0.877 (R - Y)
U-V plane at Y=0.5
CS 414 - Spring 2011Source: wikipedia
Y
U
V
YCrCb Subset of YUV that scales
and shifts the chrominance values into range 0..1
Y = 0.299R + 0.587G + .114BCr = ((B-Y)/2) + 0.5Cb = ((R-Y)/1.6) + 0.5
CS 414 - Spring 2011
YIQ NTSC standard
YIQ from RGB Y = .299R + .587G + .114B
I = .74 (R - Y) - .27 (B - Y)Q = 0.48 (R - Y) + 0.41 (B - Y)
CS 414 - Spring 2011Source: wikipedia
YIQ with Y=0.5
NTSC Video
525 scan lines per frame; 29.97 fps33.37 msec/frame (1 second / 29.97 frames)scan line lasts 63.6 usec (33.37 msec / 525)aspect ratio of 4/3, gives 700 horizontal pixels
20 lines reserved for control information at the beginning of each fieldso only 485 lines of visible data
CS 414 - Spring 2011
NTSC Video
Interlaced scan lines divide each frame into 2 fields, each of which is 262.5 linesphosphors in early TVs did not maintain
luminance long enough (caused flicker)scanning also interlaced; can cause visual
artifacts for high motion scenes
CS 414 - Spring 2011
HDTV Digital Television Broadcast (DTB) System Twice as many horizontal and vertical
columns and lines as traditional TV Resolutions:
1920x1080 (1080p) – Standard HDTV Frame rate: options 50 or 60 frames per
second
CS 414 - Spring 2011
Pixel Aspect Ratio
CS 414 - Spring 2011Source: wikipedia
• Computer Graphics parameter• Mathematical ratio describing horizontal length of a pixel to its vertical height• Used mainly in digital video editing software to properly scale and render videoInto a new format
CS 414 - Spring 2011
Resolution (WxH)
Pixels Aspect RatioPixel aspect ratio
(Standard 16:9)
Video Format Description
1024×768 786,432 16:9 4:3 720p/XGAUsed on PDP HDTV
1280×720 921,600 16:9 1:1 (square) 720p/WXGAUsed on Digital television,
1366×768 1,049,088 16:9Approx. 1:1 (square)
720p/WXGA - HDTV standard format
Used on LCD/PDP HDTV displays
1024×1080 1,105,920 16:9 15:8 1080pUsed on PDP displays HDTV
1280×1080 1,382,400 16:9 3:2 1080pUsed on PDP HDTV displays
1920×1080 2,073,600 16:9 1:1 (square)1080p - HDTV standard format
Used on all types of HDTV technologies
3840x2160 8,294,400 16:9 1:1 (square) 2160p Quad HDTV,
HDTV
Interlaced and/or progressive formatsConventional TVs – use interlaced formats Computer displays (LCDs) – use progressive
scanning MPEG-2 compressed streams In Europe (Germany) – MPEG-4
compressed streams
CS 414 - Spring 2011
Aspect Ratio and Refresh Rate Aspect ratio
Conventional TV is 4:3 (1.33) HDTV is 16:9 (2.11) Cinema uses 1.85:1 or 2.35:1
Frame Rate NTSC is 60Hz interlaced (actually 59.94Hz) PAL/SECAM is 50Hz interlaced Cinema is 24Hz non-interlaced
CS 414 - Spring 2011Source: wikipedia
SMPTE Time Codes
Society of Motion Picture and Television Engineers defines time codes for video HH:MM:SS:FF 01:12:59:16 represents number of pictures corresponding to
1hour, 12 minutes, 59 seconds, 16 frames If we consider 30 fps, then 59 seconds represent 59*30
frames, 12 minutes represent 12*60*30 frames and 1 hour represents 1*60*60*30 frames.
For NTSC, SMPTE uses a 30 drop frame code increment as if using 30 fps, when really NTSC has only 29.97fps defines rules to remove the difference error
CS 414 - Spring 2011
Take Home Exercise
Given a SMPTE time stamp, convert it back to the original frame numbere.g., 00:01:00:10
CS 414 - Spring 2011
Summary
Digitization of Video SignalsComposite CodingComponent Coding
Digital Television (DTV)DVB (Digital Video Broadcast)
Satellite connections, CATV networks – best suited for DTV
DVB-S – for satellites (also DVB-S2) DVB-C – for CATV
CS 414 - Spring 2011
Top Related