Digital Media

Dr. Jim Rowan

ITEC 2110

Video

Video

• Works because of persistence of vision• http://en.wikipedia.org/wiki/Frame_rate• Fusion frequency

– ~ 40 frames per second– depends on the brightness of the image relative to

the viewing environment

• Less than that– flickering– individual images appear losing the illusion of

motion

Video

• Video vs Animation...– Video - capture of frames and then

playback– Animation - create frames individually and

then playback

Video... Computationally demanding

– Capture must be fast enough to capture sufficient frames to produce the illusion of motion

– Transport (if across the web) must be fast enough to carry those captured frames at a rate fast enough to produce the illusion of motion

– Playback must be fast enough to play those captured frames at a rate fast enough to produce the illusion of motion

Video

• If the transport and playback is not fast enough, frames will be dropped

• Video players (like quicktime) make compromises differently– Trying to “degrade gracefully”

• Some drop frames holding the last image– effectively losing the illusion of motion but continuing the

story as a slide show

• Some play lower resolution images• Some continue to play audio

Video Standards

• NTSC– America and Japan– http://en.wikipedia.org/wiki/Ntsc– North America and Japan– 24 frames per second– framesize different than PAL

Video Standards

• PAL– Western Europe and Australia– http://en.wikipedia.org/wiki/PAL– Western Europe and Australia– 25 frames per second– framesize different than NTSC

Video gets big fast

• At a 640 X 480 framesize

• Using 3 byte color (24 bits, one byte per color) each frame ~ 2 megabytes

• One second of video (uncompresssed) is 26 Megabytes

• One minute is 1.85 gigabytes

The effects of large size...

• Uncompressed this exceeds most home computer interface standards

• Strains the internal speed of the home computer

• Strains the storage capability of home computer

• WAY exceeds what can be carried by the net

What to do? Apply compression!

• On the capture side– digitization & compression needs to be

carried out by the hardware to be fast enough

– Can be done in the camera (VTR) – Can be done in the computer (iSight cam)

Compression in the VTR

• Within the camera– 3 (at least) different formats internally

• Differing error correction and compatibility

– Recording on different media• CD• tape

• Mini DV or DV format– Connected to computer using firewire– All 3 formats present the same stream of bits to the

computer

• Artifacts that interfere with processing and recompression are created

Compression in the computer

• Analog is presented to the computer through a video capture card

• Compression is done (usually) in the video capture card

• Allows for a really small camera because the work is done elsewhere

Analog vs Digital

• An analog signal to the computer is susceptible to noise corruption

• Digital signal is not• What’s the big deal?• Consider compressing a video of a wall

painted a solid color– Analog noise will cause small fluctuations from

pixel to pixel– RLE can’t compress it because each pixel is a bit

different

iMovie

video capture card

computer

miniDV

compression

compression

iSightCamera

analog signal

digital signal

640 x 480 = 307,200307,200 can be represented by < 24 bits, call it 3 bytesRLE: 307,200 (3bytes) + RGB (3 bytes) = 6 bytes

Consider compressingthis using RLE

the scene

640 x 480 = 307,200 bytesNoise makes each pixel a little differentRLE: 307,200 bytes x RGB (3bytes) = 921600 bytes

!!!NOISE!!!

hardware vs software compression

• Hardware conversion... user has no control over it... it is hardwired– It is in the camera– It can be in the video card

• Software conversion... is computationally expensive... it’s a slow process– Provides for the most flexibility– Can use different software coder-decoders

(codec), picking and choosing what fits your needs better

Streaming Video

http://en.wikipedia.org/wiki/Streaming_media

Video is transported across the webplayed as it arrives

Similar to broadcast TVCan be part of video conferencingNetwork bandwidth is the enemy

3 Methods of Video delivery

• True Streaming video

• Embedded video

• Progressive download or HTTP streaming

True Streaming videoNetwork programs are streamed

• never stored on disk• can be open-ended• you don’t get a copy to play later

– solves copyright issue!

• can collect viewer stats without Nielsen ratings• advertisers know how many viewers there are• on some, you can advance ahead of where the

download is complete and the play will start again at the new spot

Embedded video

• entire file is transferred before play begins• stored on disk• may require more disk than is available

Progressive download or

HTTP streaming• downloads some and then starts playing• predicts when the download will complete and starts

playing when there is enough downloaded to safely play without interruption

• you get a copy to play later• takes up disk storage• quicktime does this• on some, you can advance ahead of where the

download is complete and the download will start again at the new spot

The TV legacy

• Raster scan - 525 lines in US and Japan• Not enough bandwidth to transmit the

whole frame at a rate of 40+ frames per second

• Resulted in Interlacing fields• Play half the Frame (and half the data)

then play the other half of the Frame

525 lines total

525 lines total

525 lines total

The problem...

• The fields were played one after the other to avoid flicker

• BUT...

• The fields were also captured one after the other...

• there was a time difference between when they were captured

Why is this a problem?

• When you play them on a computer, the computer can refresh much faster and can display the entire frame at the same time

• To play it can put both fields in a frame buffer and displays them at the same time

• If the object is moving fast, the second frame shows the object in a different place

• Results in a “comb effect”

Can we solve the problem?

• You can average the two frames and construct a single frame

• You can toss out one of the fields and interpolate between them

• Neither is very good...

Film to video?

• Problematic– video is 30 frames per second– film is 24 frames per second

• How do you make 30 frames from 24?• The 3-2 pull down

– http://en.wikipedia.org/wiki/3-2_pulldown– 3 film frames occupy the first 3 fields– 2 film frames occupy the next 2 fields

Next

• DV and MPEG

Questions?