The Computer Continuum9-1 Chapter 9: Multimedia Most of the outside information entering our brains,...

The Computer Continuum 9-1

Chapter 9: Multimedia

Most of the outside information entering our brains, enters through our eyes or our ears. They

are, indeed, input devices through which our world communicates with us.


Multimedia

In this chapter:

• What is multimedia?

• How are pictures stored within the computer?

• What can be done with digitized pictures?

• How can pictures be created using a computer and software?

• What is the importance of vocal communication?

• How is speech stored and played back by the computer?

• What are some problems the computer has when analyzing speech?

• What areas are computers used in audio communication?


What is Multimedia?

Multimedia has become interactive

• Interactive multimedia: The use of media such as text, graphics, animation, video, and audio in an interactive way that allows a participant to control it.

• To understand the value of interactive multimedia, we will examine visual and audio concepts in detail.


Visual Media:Manipulating Images

Previously created images (Modifying or processing images that already exist)• Three types of techniques used to manipulate images:

– Minor Processing Techniques

– Enhancement

– Restoration



Common example of digitized images: Satellite weather maps.• GEOS-7 satellite picture

taken on Aug. 24, 1992.

• Outlines created by a computer program.

• Program color coded the picture to give information about the hurricane’s wind speed.



Landsat 4 Satellite:• Uses 3 of the 7 frequencies

available from the Thematic Mapper of the satellite.

• Like having a camera with different filters, each letting a different frequency through (including infrared).



Image enhancement• A type of digital image

processing whose goal is to highlight or enhance particular aspects of an image or change an image’s structure.

• Common example of image enhancement: False-coloring: An image enhancement technique that consists of changing the colors of an image or assigning colors to various parts of an image.



Image restoration• A type of digital image

processing whose goal is to eliminate known, but unwanted image flaws or degradations.



Image enhancement example:• It makes you wonder if

photographs can be trusted.


Visual Media:Creating Images

Artists follow rules (or intentionally don’t follow rules) to create original images:• Hiding parts of things that wouldn’t normally be visible.

• Sources of light that illuminate the objects.

Creating images using the computer• Line images.

• Solid forms to 3D.

• Animation and video.


Visual Media: Creating Images

Creating Line Images• What are some uses of line drawings?

– Map making.

– Architectural drawings.

– Business graphs.

– Plans for building machinery, aircraft, furniture-in-a-box.

– City population density maps.

• How can line images be created?

– Using drawing programs such as Adobe Illustrator.

• Have drawing related features including geometric shapes, free-hand tools, fills, patterns, text.



Bit-mapped or Raster graphical images

• Constructed of individual pixels.

• Only crude manipulation can be done (modification of each pixel).

• Retains “bit-mapped” appearance.

Object-oriented or Vector graphic images

• Stored in the computer as lines, curves, or geometric shapes.

• Formulas are used to draw (circles, twice the size, are still circles!)

• Objects can be moved or modified easily.



Designing 3-dimensional pictures drawn in 2-dimensions

• Curved surfaces

• Color

• Texture

• Shading



One of the classic problems in computer graphics:

• Hidden-line problem: Concerns itself with how to hide the outlines or surfaces of a solid object that shouldn’t be seen from the direction of the observation.

• Example: Studies of an airline pilot in action.



Perspective: The quality that allows 3-dimensional images to be drawn on 2-dimensional surfaces and yet retain the look of a 3-dimensional image.



Shading: A technique used to give the appearance of illumination by some combination of light sources.

The architecture program used to draw this building has a 3D descriptionof the building It can produce pictures from any angle.



Shading flat surfaces of the head (as in b) then blending the surfaces to give a more natural look (as in c).

a b c



Texture: A property of a surface. It is observed and identified through the reflection of the light off the surface.


Visual Media:Animation and Video

Animation:• Early animation started with pictures flipping hand-cranked

players.• Animation 30 years ago:

– Cartoons had to have the figures drawn by artists. – The animator would make them move while taking pictures,

frame by frame.

• Animation now:– Computers are used to create full length feature cartoons.

• Toy Story• Individual frames stored on the computer.• Special player programs can be used to view them on the

computer screen or recorded on videotape using special hardware and software.



Jurassic Park dinosaurs• Wire frames were created in the computer to depict dinosaurs.

• Each wire frame is moved by the computer which calculates positions of each part viewed from a certain angle and distance.

• The computer covered the surfaces of the wire frames with a texture layer.

– Computer takes into account sources of light illuminating the figure.



Morphing: A graphics technique used in animation• A beginning image

will distort and change in a predetermined number of frames into a final image.



Digitizing and Manipulating Video• Digital video hardware systems: Systems capable of capturing

and manipulating video.

• Analog video uses standards developed over 50 years ago.

– Each differs in the number of horizontal lines that make up a single video frame and in the number of frames per second that are displayed on the TV set.

• European broadcast standards (PAL).

• United States broadcast standards (NTSC).

» 30 frames per second.

» VHS.



Video capture: The process of digitizing an analog TV signal. • Done using hardware that produces a digital form of the

analog TV signal.

– Uses the VCR to feed the video signal into a digital hardware card that plugs into the computer.

– Each frame is converted into a bitmapped image.

– The digital video signal is compressed.

• Eliminates data that our eye doesn’t see or that is hardly noticed.

• Codec: A scheme for coding & decoding large amounts of data. (The data is economical and uncomplicated to store.)



Common codecs:• Motion Picture Experts Group (MPEG)

– MPEG-1• Developed to handle slower media such as videoCDs.• Deteriorates when high-speed action occurs.• Roughly the quality of VHS.• 70 minutes with sound fits on a CD-ROM disc.

– MPEG-2• Full-screen codec standard that brings twice the resolution

of VHS.• Four times the resolution of MPEG-1.• Optimized for higher demands of broadcast, satellite, and

DVD.



Common codecs: (continued)• Motion Picture Experts Group (MPEG)

– M-JPEG• Requires more memory space than MPEG-2• Each frame is digitized separately using JPEG

compression.» Makes it easier to edit.

• As we move from analog to digital broadcast, coding and decoding will become unnecessary. (Everything will already be digital!)



DVD: The Digital Versatile Disc• DVD: An optical disk storage technology that looks like the

standard CD, but is faster and has a much larger storage capacity.

– Two forms:

• DVD-Video Discs: Intended as a replacement of VHS.

• DVD-ROM: Intended as a computer storage medium.

– Digital Video Players:

• Software products are available that will play back video coded with one of these codec systems.

» QuickTime, ActiveMovie



Streaming media: Information in one of many different visual or audio forms that is sent from a server in packets to the requesting computer.

• Contains parts of the medium that may be a recording or an actual live event.

• The information is played back in the order sent, starting soon after a few packets arrive, but before all of them have been sent.


Audio Media:Human Speech

The importance of vocal communication:• It is the fastest method of communication.

• Most communication is vocal.

• Researchers believe that human intelligence is closely linked to the development of vocal communication and language.



Creating Speech: Recorded Speech• Recording and digitizing individual words and phrases.

– Common uses:

• Talking cash registers.

• Automated national telephone information service.

– Problems with recorded speech:

• Creating whole sentences of human speech would take a huge amount of computer memory.

• Several versions of each word would need to be digitized to account for pitch and volume levels (for punctuation and variations of the same word).



Creating Speech: Speech Synthesis• Speech synthesis: The electronic production of sounds and

sound patterns that closely resemble human speech.

– Uses phonemes: The fundamental sounds of any given human language.

– Three additional factors that have an affect on how a word or phrase sounds:

• Inflection - The rising and falling pattern of pitch.• Duration - The time spent on or the time between

individual phonemes.• Elision - Splicing phonemes together so that when one

ends, the other begins. The connection will sound natural.



Creating Speech: Speech Synthesis• The sounds produced are electronically generated.

– The sounds simulate those produced by the human vocal tract.

• Can produce both major types of human sounds:

» Voiced sounds: Sounds produced by the vibration of vocal cords in conjunction with specific positioning of teeth, tongue, and lips. Include all vowels and some consonants.(d, g, b)

» Voiceless sounds: Characterized by the lack of vibration of the vocal cords. (s, k, t)



Recognizing Spoken Words• More difficult for the computer than speech synthesis.

• Voiceprint: A visual plot of frequency versus time of sound produced by a human speaker.

– No two voiceprints are completely identical.

• Human voice is determined at least by:» Pitch: The number of cycles per second of a particular

sound’s vibration.» Resonance: The reverberation or amplification of the

voice in the cavities of the vocal tract.

• Problem: Because each individual has uniquely shaped vocal cords and resonating chambers, each has a uniquely individual voice.



Recognizing Speech• Disjointed speech: Words spoken one at a time with silence

between words.

– Software programs are available for home use.

– Each person goes through a recording session to train it.

– Total vocabulary to fewer than 1000 words spoken by an individual.

• Problems:

– Speaking in a disjointed manner is unnatural.

– Any changes in the person’s voice (head cold, sinus infection) may make the speech unrecognizable.



Recognizing Speech• Continuous speech: Words spoken in a continuous stream of

sounds, usually with no pauses between words.

– More difficult for the computer to understand. Computer has difficulty when one word ends and next begins.


Audio Media:Music

The Computer and Recorded Music• Compact Disc Technology

– CDs don’t wear out like records and magnetic tape.

• After numbers are recorded, they don’t change.

• Scratching the surface can affect quality of sound.

– Compact discs can be produced from a master disc very inexpensively.


Audio Media:Music

The Computer and Recorded Music• DAT (Digital Audio Tape) technology

– Advantages of DAT:

• Binary information is recorded onto magnetic tape.

• DAT can easily record sound information.

• Perfect duplication of CDs can be made onto tape.


Audio Media:Music

Recorded Audio Files and Formats• Individual standards have been developed for special

purposes or for specific computers.

– Sound files can be huge. Sound quality affects size.

– The four most common sound file formats:

WAV: Originally developed with Windows 3.1

AU: Developed by SunAudio for UNIX systems but now supported by Windows and Macintosh.

AIFF: Originated on the Macintosh computer.

MP3: (MPEG - level 3) Can be played on Windows, Macintosh, and UNIX systems.


Audio Media:Music

Hardware and Software Needs• Playing back Web sound

– Need a sound card.

• Newer “multimedia” computers have sound cards included.

– Need external speakers.

• Receiving sound files from the Web.

– Must be a type of file your browser recognizes.

• Most new browsers handle Web sound.


Audio Media:Music

MIDI: A Revolution in Music• MIDI (Musical Instrument Digital Interface): Musical

standard created in 1982 that makes possible communication of audio information

– Between keyboards, drum pads, etc.

– To a computer-controlled mixer.

– Each instrument connected to a MIDI bus usually has a small special-purpose computer with some memory.

• MIDI bus: The main communications channel for MIDI information that can consist of signals sent to as many as 16 different instruments or groups of instruments.


Audio Media:Music

All MIDI instruments have three-pronged connectors that are used to connect other MIDI instruments or computers.• Thru port: Sends MIDI signals obtained from the IN port on to other

MIDI instruments.

• OUT port: Sends out MIDI signals.

• IN port: Receives MIDI signals.

• The three connectors necessary for a MIDI interface:


Audio Media:Music

Several ways to use a MIDI instrument:• By itself.

– A MIDI synthesizer can connect to an amplifier with speakers.

• Using one MIDI instrument as a master to a second, acting as a slave.

– Can not only play its own notes, but can also control the slave’s notes.

• In conjunction with a MIDI sequencer.– MIDI sequencer: A device, usually a computer, that

records and plays back MIDI signals.– Can be either a computer with sequencing software or a

dedicated sequencer.


Internet Music and Audio Net

Three techniques for both downloading sound to your computer and listening to live audio over the WWW.1. Streaming audio: A request to hear music is made by clicking

on a button. The Website sends back the sound which you hear immediately. You don’t receive a copy of the sound. (You may need to add a plug-in to hear the sound.)

– Plug-in: A piece of software that is added to your browser to give it desired capability.

2. Sound file is sent over the WWW. It allows you to store the sound file on your computer.

3. MIDI files: Created by a performer on a MIDI instrument. Needs a MIDI player (available on the Internet.)

The Computer Continuum9-1 Chapter 9: Multimedia Most of the outside information entering our brains,...

Documents

Transcript of The Computer Continuum9-1 Chapter 9: Multimedia Most of the outside information entering our brains,...