Introduction to Multimedia. 1- What is Multimedia? 2- Multimedia and Hypermedia 3- World Wide Web 4-...

Introduction to MultimediaIntroduction to Multimedia

Introduction to MultimediaIntroduction to Multimedia

1- What is Multimedia?2- Multimedia and Hypermedia3- World Wide Web4- Overview of Multimedia Software Tools

1- What is Multimedia?2- Multimedia and Hypermedia3- World Wide Web4- Overview of Multimedia Software Tools

What Is Multimedia ?What Is Multimedia ? What Is Multimedia ?What Is Multimedia ?

Multi Multi refer to many or multiplerefer to many or multiple

Media Media Vehicle or agent to convey for somethingVehicle or agent to convey for something Example: air for sound, writing for wordsExample: air for sound, writing for words Computer is a medium for ideas and information Computer is a medium for ideas and information

- Express and communicate these ideas- Express and communicate these ideas

- Store and manipulate information in many different forms- Store and manipulate information in many different forms

MultimediaMultimediaMultiMultiMultiMulti MediaMediaMediaMedia

The big picture The big picture

ImagImagee

TextText

SpeeSpeechch

AudiAudioo

VideVideoo

MultimediaMultimedia

Virtual worldsVirtual worldsStreaming videoStreaming video

Web pagesWeb pagesStreaming audioStreaming audio Client-serverClient-server

MultimediaMultimediaSystemsSystems

AuthoringAuthoring

Media asset Media asset managementmanagement

ToolsTools

FrameworksFrameworks

NetworkNetwork

DistributedDistributedMultimediaMultimediaSystemsSystems

GLOBAL STRUCTURE OF MultimediaGLOBAL STRUCTURE OF Multimedia

GLOBAL STRUCTURE OF MultimediaGLOBAL STRUCTURE OF MultimediaApplication domain : provides functions to the user to develop and present multimedia projects. This includes Software tools, andmultimedia projects development methodology.

System domain : including all supports for using the functions of the device domain, e.g., operating systems, communicationsystems (networking) and database systems.

Device domain : basic concepts and skill for processing various multimedia elements and for handling physical device.

Application domain : provides functions to the user to develop and present multimedia projects. This includes Software tools, andmultimedia projects development methodology.

System domain : including all supports for using the functions of the device domain, e.g., operating systems, communicationsystems (networking) and database systems.

Device domain : basic concepts and skill for processing various multimedia elements and for handling physical device.

What Is Multimedia ? What Is Multimedia ?

MultimediaMultimedia Presentation of a computer application Presentation of a computer application Incorporating media elements such as text, graphics, video, Incorporating media elements such as text, graphics, video,

animation and sound on a computer.animation and sound on a computer.

Multimedia SystemsMultimedia Systems the computer and software systems that are used in the computer and software systems that are used in

multimedia environments multimedia environments Interactive control Interactive control

Distributed multimedia systemsDistributed multimedia systems operate over some form of network infrastructureoperate over some form of network infrastructure

When different people mention the term multimedia, theyWhen different people mention the term multimedia, they

often have quite different, or even opposing, viewpoints.often have quite different, or even opposing, viewpoints.

• A PC vendor: a PC that has sound capability, a DVD-ROM A PC vendor: a PC that has sound capability, a DVD-ROM drive , and perhaps the superiority of multimedia-enabled drive , and perhaps the superiority of multimedia-enabled microprocessors that understand additional multimedia microprocessors that understand additional multimedia instructions.instructions.

• A consumer entertainment vendor: interactive cable TV A consumer entertainment vendor: interactive cable TV with hundreds of digital channels available, or a cable TV-with hundreds of digital channels available, or a cable TV-like service delivered over a high-speed Internet like service delivered over a high-speed Internet connection.connection.

• A Computer Science (CS) student: applications that use A Computer Science (CS) student: applications that use multiple modalities, including text, images, drawings multiple modalities, including text, images, drawings (graphics), animation, video, sound including speech, and (graphics), animation, video, sound including speech, and interactivity.interactivity.

What Is Multimedia ? What Is Multimedia ?

Multimedia and Computer Science:Multimedia and Computer Science: Graphics, visualization, computer vision, data Graphics, visualization, computer vision, data

compression, graph theory, networking, database systems.compression, graph theory, networking, database systems.

• Components(elements) of MultimediaComponents(elements) of Multimedia

Multimedia involves multiple modalities of text, audio, Multimedia involves multiple modalities of text, audio,

ImImages, drawings, animation, and videoages, drawings, animation, and video

A brief history of multimedia A brief history of multimedia

1970-an

1980

1990

Future

8 bit processor

CD-ROM, Local and Wide Area Networking16 bit processorsDesktop Computer

32 bit

World Wide Web

Digital Audio and Video

Graphical Operating Systems

Modern multimedia PCs Modern multimedia PCs 3D graphics cards with 64 MB of graphics memory3D graphics cards with 64 MB of graphics memory

64 note sound cards with wavetable synthesis64 note sound cards with wavetable synthesis

256 MB of main memory256 MB of main memory

60+ GB hard disk space60+ GB hard disk space

50+ speed CD readers50+ speed CD readers

56 Kb/s modem or ISDN connection56 Kb/s modem or ISDN connection

Content(Media)

Applications (Software)

Platforms(Hardware)

Networking

Level 1Level 1

Level 2Level 2

Level 3Level 3

Level 4Level 4

Multimedia Industry Structure

Level 1Level 1 - Consists of MM content providers, for example the

news industry, the television industry and the entertainment industry.

- Level 2Level 2- Consists of multimedia application developers.- Applications include distance learning, desktop

videoconferencing, multimedia kiosks, entertainment and imaging.

- Level 3 Level 3 - Consists of MM platform builders.- These builders include Silicon Graphics, Sun, Intel,

Apple, and other hardware vendors.


Level 4Level 4 - The network infrastructure. - There are two very different networking

environments that will use MM applications: - business networks and - the emerging public networks.


Why Apply Multimedia TechnologyWhy Apply Multimedia Technology

Ease of useEase of use Intuitive InterfaceIntuitive Interface Immersive ExperienceImmersive Experience Self-Paced Interaction and Better RetentionSelf-Paced Interaction and Better Retention Better Understanding of the ContentBetter Understanding of the Content Cost EffectivenessCost Effectiveness More Fun = Greater EfficiencyMore Fun = Greater Efficiency

Multimedia DevelopmentMultimedia Development

ProcessProcess

Capture from device

Read from a file

Receive over the network

Apply effect filters and transforms

Compress or decompress

Convert between formats

Present

Save to a file

Send across the network

InputInput OutputOutput

Examples of multimedia systems Examples of multimedia systems

Content delivery system examples: Television/(radio?) (old, basic system) Web/Internet type Mobile (radio networks)Content distribution type examples: Music Movies (audio/visual content Games Virtual worlds (Check Second life on the Web) eBooks? eMagazines? eNewspapers?

Multimedia Application

A Multimedia Application is an application which uses a collection of multiple media sources e.g. text, graphics, images, sound/audio, animation and/or video.

Multimedia System Desirable Features

Speci Very High Processing Power al Hardware/Software needed Efficient and High I/O Large Storage and Memory High Speed Network Support

Components of a Multimedia System

Capture devices

• e.g. Video Camera, Microphone, Digitising/Sampling Hardware, etc.

Storage Devices

• e.g. Hard disks, CD-ROMs, DVD, etc Communication Networks

• e.g. Ethernet, Internets, etc. Computer Systems

• e.g. Multimedia Desktop machines, Workstations Display Devices

• e.g. CD-quality speakers, HDTV, Hi-Res monitors, Color printers etc.

Multimedia Data: Text

Text • Input: keyboard• Stored and input character by character.• Storage of text is 1 or 2 bytes per character.• Other forms of data (e.g. Spreadsheet files,

XML) may store format as text (with formatting).

• Format: Raw text or formatted text e.g HTML, Rich Text Format (RTF), Word or a program language source

Multimedia Data: Text

Text (cont.)• Not temporal — BUT may have natural implied

sequence e.g. HTML format sequence, Sequence of Java program statements.

• Size Not significant compared with other multimedia data.

Formatted Text

Multimedia Data: Images

Images• Still pictures which (uncompressed) are

represented as a bitmap (a grid of pixels).• Input: Digital camera, scanner or generated by

graphics editor programs (e.g. Paint)• Analog sources will require digitising.• Stored at 1 bit per pixel (Black and White), 8

Bits per pixel (Grey Scale, Color Map) or 24 Bits per pixel (True Color)

Multimedia Data: Images

Images (Cont.)• Size: a 512x512 Grey scale image takes up 1/4

Mb, a 512x512 24 bit image takes 3/4 Mb with no compression.

• Storage increases with image size• Compression is commonly applied

Image : PNG File Format

Multimedia Data: Vector Graphics

Vector Graphics• Format: constructed by the composition of

primitive objects such as lines, polygons, circles, curves and arcs.

• Input: usually generated by a graphics editor program (e.g. Freehand)

• Vector Graphics are usually editable or revisable (unlike Images).

• Standards : SVG, OpenGL, WMF, etc.• Do not take up a very high storage overhead.

Multimedia Data: Vector Graphics

Vector Graphics (Cont.)• Advantages: reduction of data size, meaningful

representation, easier manipulation, resizable without losing quality .

• Disadvantages: not suitable for reproducing photographs or paintings, take a long time to display complex image.

Vector Graphics vs Images

Graphic (WMF) : 10262 bytes

Image (BMP) : 51054 bytes

enlarge

enlarge

Vector Graphics Examples

Multimedia Data: Audio

Audio• Audio signals are continuous analog signals.• Input: microphones and then digitised and

stored • usually compressed.• CD Quality Audio requires 16-bit sampling at

44.1 KHz• 1 Minute of Mono CD quality audio requires 5

Mb.

Multimedia Data: Video

Video• Input: usually captured by a video camera.• There are a variety of video (analog and digital)

formats• Raw video can be regarded as being a series

of single images.• There are typically 25, 30 or 50 frames per

second.• a 512x512 size monochrome video images take

25*0.25 = 6.25Mb for a minute to store uncompressed.

• Digital video clearly needs to be compressed.

Multimedia Delivery

Offline • Use removable storage as medium delivery• Examples : CD-ROM, PC, DVD

Online• Use a network to send information• Examples : WWW• Enables delivery live multimedia content. • Examples : Video Conferencing, broadcast

Multimedia

Data Transfer over the Internet

kbits/s 6KB Text File

100KB Image

4MB Movie

GSM Data 9.6 4.5 84 Seconds 56 Minutes

Slow Modem 14.4 3 Seconds 56 Seconds 37 Minutes

GPRS 36.6 1.25 Seconds 22 Seconds 15 Minutes

Fast Modem 56 1 Second 14 Seconds 9 Minutes

ISDN 128 <1 Second 6 Seconds 4.3 Minutes

T1 Leased Line

1544 <1 Second 1 Second 21 Seconds

ADSL (Typical)

6000 <1 Second <1 Second 5 Seconds

22 Multimedia and HypermediaMultimedia and Hypermedia

A hypertext system: A hypertext system:

meant to be read nonlinearly, by followingmeant to be read nonlinearly, by following links that point links that point to other parts of the document, or to other documents.to other parts of the document, or to other documents.

HyperMediaHyperMedia::

not constrained to be text-based, can include not constrained to be text-based, can include other media, other media, e.g., graphics, images, and especially the continuous e.g., graphics, images, and especially the continuous media | sound and video.media | sound and video.

- The World Wide Web (WWW) | the best example of - The World Wide Web (WWW) | the best example of ahypermedia application.ahypermedia application.

Multimedia Multimedia

means that computer information can be repremeans that computer information can be represented sented through audio, graphics, images, video, and animationthrough audio, graphics, images, video, and animation

in addition to traditional media.in addition to traditional media.

• Typical present multimedia applicationsTypical present multimedia applications

include(Examples of )include(Examples of ):: - Digital video editing and production systems.- Digital video editing and production systems.

- Electronic newspapers/magazines.- Electronic newspapers/magazines.

- World Wide Web.- World Wide Web.

- On-line reference works: e.g. encyclopedias, games, etc.- On-line reference works: e.g. encyclopedias, games, etc.

- Home shopping.- Home shopping.

- Interactive TV.- Interactive TV.

- Multimedia courseware.- Multimedia courseware.

- Video conferencing.- Video conferencing.

- Video-on-demand.- Video-on-demand.

- Interactive movies.- Interactive movies.

3 World Wide Web3 World Wide Web

The W3C has listed the following goals for the WWW:The W3C has listed the following goals for the WWW: 1. Universal access of web resources (by everyone every-1. Universal access of web resources (by everyone every-

where).where).

2. Eeffctiveness of navigating available information.2. Eeffctiveness of navigating available information.

3. Responsible use of posted material.3. Responsible use of posted material.

History of the WWWHistory of the WWW1960 - Charles Goldfarb et al. developed the Generalized1960 - Charles Goldfarb et al. developed the Generalized

Markup Language (GML) for IBM.Markup Language (GML) for IBM.

1986 - The ISO released a nal version of the Standard 1986 - The ISO released a nal version of the Standard Generalized Markup Language (SGML).Generalized Markup Language (SGML).

1990 - Tim Berners-Lee invented the HyperText Markup1990 - Tim Berners-Lee invented the HyperText Markup

Language (HTML), and the HyperText Transfer Language (HTML), and the HyperText Transfer

Protocol (HTTP).Protocol (HTTP).

1993 - NCSA released an alpha version of Mosaic based on1993 - NCSA released an alpha version of Mosaic based on

the version by Marc Andreessen for X-Windows |the version by Marc Andreessen for X-Windows |

the first popular browser.the first popular browser.

1994 - Marc Andreessen et al. formed Mosaic Communica1994 - Marc Andreessen et al. formed Mosaic Communications tions Corporation | later the Netscape Communications Corporation | later the Netscape Communications Corporation.Corporation.

1998 - The W3C accepted XML version 1.0 specifications1998 - The W3C accepted XML version 1.0 specifications

as a Recommendation | the main focus of the W3Cas a Recommendation | the main focus of the W3C

and supersedes HTML.and supersedes HTML.

HTTP (HyperText Transfer Protocol)HTTP (HyperText Transfer Protocol)

HTTP:HTTP: a protocol that was originally designed for transmit- a protocol that was originally designed for transmit-

ting hypermedia, but can also support the transmission ofting hypermedia, but can also support the transmission of

any le type.any le type.

HTTP is a stateless request/response protocol: HTTP is a stateless request/response protocol: no inforno information mation carried over for the next request.carried over for the next request.

The basic request format:The basic request format:

Method URI VersionMethod URI Version

Additional-Headers:Additional-Headers:

Message-bodyMessage-body

The URI (Uniform Resource Identifier): an identifier for theThe URI (Uniform Resource Identifier): an identifier for the

resource accessed, e.g. the host name, always preceded byresource accessed, e.g. the host name, always preceded by

the token \http://".the token \http://".

Two popular methodsTwo popular methods: GET and POST.: GET and POST.

HTML (Hyper Text Markup Language)HTML (Hyper Text Markup Language)

HTML:HTML: a language for publishing Hypermedia on the World a language for publishing Hypermedia on the World

Wide Web | defined using SGML:Wide Web | defined using SGML:

1. HTML uses ASCII, it is portable to all dierent (possibly1. HTML uses ASCII, it is portable to all dierent (possibly

binary incompatible) computer hardware.binary incompatible) computer hardware.

2. The current version of HTML is version 4.01.2. The current version of HTML is version 4.01.

3. The next generation of HTML is XHTML | a reformulation 3. The next generation of HTML is XHTML | a reformulation of HTML using XML.of HTML using XML.

HTML uses tags to describe document elements:HTML uses tags to describe document elements:

- <token params> | defining a starting point,- <token params> | defining a starting point,

- </token> | the ending point of the element.- </token> | the ending point of the element.

- Some elements have no ending tags.- Some elements have no ending tags.

Naturally:Naturally:

HTML has more complex structures and can be HTML has more complex structures and can be mixed in with other standards.mixed in with other standards.

XML (Extensible Markup Language)XML (Extensible Markup Language)XML:XML: a markup language for the WWW in which there is a markup language for the WWW in which there is

modularity of data, structure and view so that user or modularity of data, structure and view so that user or application can be able to define the tags (structure).application can be able to define the tags (structure).

Example of using XML to retrieve stock information from aExample of using XML to retrieve stock information from a

database according to a user query:database according to a user query:

1. First use a global Document Type Definition (DTD) that1. First use a global Document Type Definition (DTD) that

is already defined.is already defined.

2. The server side script will abide by the DTD rules to2. The server side script will abide by the DTD rules to

generate an XML document according to the query usinggenerate an XML document according to the query using

data from your database.data from your database.

3. Finally send user the 3. Finally send user the XML Style Sheet (XSL) dependingXML Style Sheet (XSL) depending

on the type of device used to display the information.on the type of device used to display the information.

The following XML related specications are also standardized:The following XML related specications are also standardized:

- XML Protocol: XML Protocol: used to exchange XML information between used to exchange XML information between processes.processes.

- - XML Schema: XML Schema: a more structured and powerful languagea more structured and powerful language

for defining XML data types (tags).for defining XML data types (tags).

- - XSL:XSL: basically CSS for XML. basically CSS for XML.

- - SMIL:SMIL: synchronized Multimedia Integration Language, synchronized Multimedia Integration Language, pronounced \smile"|a particular application of XML (glob-pronounced \smile"|a particular application of XML (glob-

ally predefined DTD) that allows for specification of ally predefined DTD) that allows for specification of interaction among any media types and user input, in ainteraction among any media types and user input, in a

temporally scripted manner.temporally scripted manner.

SMIL (Synchronized Multimedia SMIL (Synchronized Multimedia IntegrationLanguageIntegrationLanguage))

Purpose of SMIL: Purpose of SMIL: it is also desirable to be able to publishit is also desirable to be able to publish

multimedia presentations using a markup language.multimedia presentations using a markup language.

- A multimedia markup language needs to enable scheduling- A multimedia markup language needs to enable scheduling

and synchronization of different multimedia elements, andand synchronization of different multimedia elements, and

define their interactivity with the user.define their interactivity with the user.

- The W3C established a Working Group in 1997 to come up- The W3C established a Working Group in 1997 to come up

with specifications for a multimedia synchronization with specifications for a multimedia synchronization language | SMIL 2.0 was accepted in August 2001.language | SMIL 2.0 was accepted in August 2001.

SMIL 2.0 is specified in XML using a SMIL 2.0 is specified in XML using a modularization modularization apapproach similar to the one used in xhtml:proach similar to the one used in xhtml:

1. All SMIL elements are divided into modules | sets of1. All SMIL elements are divided into modules | sets of

XML elements, attributes and values that define one XML elements, attributes and values that define one conceptual functionality.conceptual functionality.

2. In the interest of modularization, not all available modules2. In the interest of modularization, not all available modules

need to be included for all applications.need to be included for all applications.

3. Language Profiles: species a particular grouping of3. Language Profiles: species a particular grouping of

modules, and particular modules may have integration modules, and particular modules may have integration requirements that a profile must follow.requirements that a profile must follow.

- SMIL 2.0 has a main language profile that includes almost all - SMIL 2.0 has a main language profile that includes almost all SMIL modules.SMIL modules.

44 Overview of Multimedia Software Overview of Multimedia Software ToolsTools

The categories of software tools briefy The categories of software tools briefy examined here are:examined here are:

1. Music Sequencing and Notation1. Music Sequencing and Notation

2. Digital Audio2. Digital Audio

3. Graphics and Image Editing3. Graphics and Image Editing

4. Video Editing4. Video Editing

5. Animation5. Animation

6. Multimedia Authoring6. Multimedia Authoring

Music Sequencing and NotationMusic Sequencing and Notation

Cakewalk: Cakewalk: now called Pro Audio.now called Pro Audio.

- The term sequencer comes from older devices that stored- The term sequencer comes from older devices that stored

sequences of notes (\events", in MIDI).sequences of notes (\events", in MIDI).

- It is also possible to insert WAV les and Windows MCI- It is also possible to insert WAV les and Windows MCI

commands (for animation and video) into music trackscommands (for animation and video) into music tracks

(MCI is a ubiquitous component of the Windows API.)(MCI is a ubiquitous component of the Windows API.)

CubaseCubase: another sequencing/editing program, with capabil: another sequencing/editing program, with capabil ities ities similar to those of Cakewalk. It includes some digitalsimilar to those of Cakewalk. It includes some digital

audio editing tools.audio editing tools.

MacromediaMacromedia SoundeditSoundedit: mature program for creating au: mature program for creating audio for dio for multimedia projects and the web that integrates wellmultimedia projects and the web that integrates well

with other Macromedia products such as Flash and Director.with other Macromedia products such as Flash and Director.

DigitalDigital AudioAudio

DigitalDigital AudioAudio toolstools deal with accessing and editing the actual deal with accessing and editing the actual sampled sounds that make up audio:sampled sounds that make up audio:

- - CoolCool EditEdit: a very powerful and popular digital audio: a very powerful and popular digital audio

toolkit; emulates a professional audio studio |multi tracktoolkit; emulates a professional audio studio |multi track

productions and sound le editing including digital signalproductions and sound le editing including digital signal

processing effects.processing effects.

- - Sound ForgeSound Forge: a sophisticated PC-based program for: a sophisticated PC-based program for

editing audio WAV files.editing audio WAV files.

- - Pro ToolsPro Tools: a high-end integrated audio production and: a high-end integrated audio production and

editing environment | MIDI creation and manipulation;editing environment | MIDI creation and manipulation;

powerful audio mixing, recording, and editing software.powerful audio mixing, recording, and editing software.

Graphics and Image EditingGraphics and Image Editing

Adobe IllustratorAdobe Illustrator: a powerful publishing tool from Adobe.: a powerful publishing tool from Adobe.

Uses vector graphics; graphics can be exported to Web.Uses vector graphics; graphics can be exported to Web.

Adobe PhotoshopAdobe Photoshop: the standard in a graphics, image : the standard in a graphics, image proprocessing and manipulation tool.cessing and manipulation tool.

- Allows layers of images, graphics, and text that can be - Allows layers of images, graphics, and text that can be separatelyseparately

manipulated for maximum flexibility.manipulated for maximum flexibility.

- Filter factory permits creation of sophisticated lighting-- Filter factory permits creation of sophisticated lighting-effects file-ters.effects file-ters.

Macromedia FireworksMacromedia Fireworks: software for making graphics : software for making graphics specifspecifically for the web.ically for the web.

Macromedia FreehandMacromedia Freehand: a text and web graphics editing: a text and web graphics editing

tool that supports many bitmap formats such as GIF, PNG,tool that supports many bitmap formats such as GIF, PNG,

and JPEG.and JPEG.

Video EditingVideo Editing

Adobe PremiereAdobe Premiere: an intuitive, simple video editing tool for: an intuitive, simple video editing tool for

nonlinear editing, i.e., putting video clips into any order:nonlinear editing, i.e., putting video clips into any order:

- Video and audio are arranged in \tracks".- Video and audio are arranged in \tracks".

- Provides a large number of video and audio tracks, super-- Provides a large number of video and audio tracks, super-

impositions and virtual clips.impositions and virtual clips.

- A large library of built-in transitions, filters and motions- A large library of built-in transitions, filters and motions

for clips for clips ) effective multimedia productions with little) effective multimedia productions with little

effort.effort.

Adobe After EectsAdobe After Eects: a powerful video editing tool that: a powerful video editing tool that

enables users to add and change existing movies. Can addenables users to add and change existing movies. Can add

many effects: lighting, shadows, motion blurring; layers.many effects: lighting, shadows, motion blurring; layers.

Final Cut ProFinal Cut Pro: a video editing tool by Apple; Macintosh: a video editing tool by Apple; Macintosh

only.only.

AnimationAnimationMultimedia APIsMultimedia APIs::

Java3D:Java3D: API used by Java to construct and render 3D API used by Java to construct and render 3D

graphics, similar to the way in which the Java Mediagraphics, similar to the way in which the Java Media

Framework is used for handling media files.Framework is used for handling media files.

1. Provides a basic set of object primitives (cube, splines,1. Provides a basic set of object primitives (cube, splines,

etc.) for building scenes.etc.) for building scenes.

2. It is an abstraction layer built on top of OpenGL or2. It is an abstraction layer built on top of OpenGL or

DirectX (the user can select which).DirectX (the user can select which).

DirectX : DirectX : Windows API that supports video, images, audio and Windows API that supports video, images, audio and 3-D animation3-D animation

OpenGL:OpenGL: the highly portable, most popular 3-D API. the highly portable, most popular 3-D API.

3D Studio Max: 3D Studio Max: rendering tool that includes a number ofrendering tool that includes a number of

very high-end professional tools for character animation,very high-end professional tools for character animation,

game development, and visual effects production.game development, and visual effects production.

Multimedia AuthoringMultimedia Authoring

Macromedia FlashMacromedia Flash: allows users to create interactive movies: allows users to create interactive movies

by using the score metaphor, i.e., a timeline arranged in by using the score metaphor, i.e., a timeline arranged in parallel event sequences.parallel event sequences.

Macromedia DirectorMacromedia Director: uses a movie metaphor to create : uses a movie metaphor to create ininteractive presentations | very powerful and includes a teractive presentations | very powerful and includes a built in scripting language, Lingo, that allows creation of built in scripting language, Lingo, that allows creation of complex interactive movies.complex interactive movies.

AuthorwareAuthorware: a mature, well-supported authoring product: a mature, well-supported authoring product

based on the Iconic/Flow-control metaphor.based on the Iconic/Flow-control metaphor.

Quest:Quest: similar to Authorware in many ways, uses a type similar to Authorware in many ways, uses a type

of flowcharting metaphor. However, the flowchart nodesof flowcharting metaphor. However, the flowchart nodes

can encapsulate information in a more abstract way (calledcan encapsulate information in a more abstract way (called

frames) than simply subroutine levels.frames) than simply subroutine levels.

SET OF TOOLSSET OF TOOLS

QuestionsQuestions

Q1) When designing multimedia systems, what two Q1) When designing multimedia systems, what two levels of functionality need to be considered? levels of functionality need to be considered? Briefly define these levels.Briefly define these levels.

There are two levels of description for an application: There are two levels of description for an application:

** Functional levelFunctional level -- what the application does and source and -- what the application does and source and display data formats. display data formats. E.g.E.g. An application may deal with the An application may deal with the retrieval and display of movies. It retrieves movies and retrieval and display of movies. It retrieves movies and displays them in a window. displays them in a window.

** System LevelSystem Level -- The entities that the application deals with and -- The entities that the application deals with and how it deals with them. In the above example we deal with how it deals with them. In the above example we deal with video frames and voice samples and need to process these video frames and voice samples and need to process these accordingly. accordingly.

Q2) For each common multimedia data type discuss what common Q2) For each common multimedia data type discuss what common functionalities should be supported by a multimedia systemfunctionalities should be supported by a multimedia system..

The following functionality should be provided by a multimedia system: The following functionality should be provided by a multimedia system:

Digital Representation of MediaDigital Representation of Media -- --

Many standardised formats for many media Many standardised formats for many media

Capture: Digitisation of Media Capture: Digitisation of Media -- --

special Hardware/Software special Hardware/Software

Creation and editing Creation and editing ––

assemble media and alter it assemble media and alter it

Storage Requirements Storage Requirements ––

significant for multimedia significant for multimedia

CompressionCompression -- --

related to above and below, ie can save on storage but can hinder retrieval related to above and below, ie can save on storage but can hinder retrieval

Structuring and retrieval methods of media Structuring and retrieval methods of media – –

simple to advanced DataBase Storage simple to advanced DataBase Storage

Display or Playback methods Display or Playback methods -- --

effect of retrieval must view data effect of retrieval must view data

Media SynchronisationMedia Synchronisation -- --

display multimedia as it is intendeddisplay multimedia as it is intended

Typical Range of operations required for common media include:Typical Range of operations required for common media include:

Text Text

Editing Formatting Sorting Indexing Searching ncrypting Editing Formatting Sorting Indexing Searching ncrypting

The above operations involve basic processing such as: The above operations involve basic processing such as:

Character Manipulation Character Manipulation

String Manipulation String Manipulation

AudioAudio

Audio Editing Synchronisation Conversion/Translation Audio Editing Synchronisation Conversion/Translation

Filtering/ Sound Enhancing Operators Compression Searching Filtering/ Sound Enhancing Operators Compression Searching Indexing Indexing


Sample Manipulation Sample Manipulation

Waveform ManipulationWaveform Manipulation

Graphics Graphics

Graphic primitive Editing Shading Mapping lighting Viewing Graphic primitive Editing Shading Mapping lighting Viewing

Rendering Searching Indexing Rendering Searching Indexing


Primitive Manipulation Primitive Manipulation

Structural/Group Manipulation Structural/Group Manipulation

Image Image

Pixel operations Geometric Operations Filtering Conversion Pixel operations Geometric Operations Filtering Conversion

Indexing Compression Searching Indexing Compression Searching


Pixel Manipulation Pixel Manipulation

Area Manipulation, copy image areas, delete area, process areas of Area Manipulation, copy image areas, delete area, process areas of interest interest

Animation Animation

-- --

Primitive/Group Editing Structural Editing Image Overlaying Primitive/Group Editing Structural Editing Image Overlaying

Rendering Synchronisation Searching Indexing Rendering Synchronisation Searching Indexing



Area Manipulation, copy image areas, delete area, process areas of Area Manipulation, copy image areas, delete area, process areas of interest interest

Frame Manipulation, Copy frame(s) to frame(s), Process frame(s) Frame Manipulation, Copy frame(s) to frame(s), Process frame(s)

Video Video

-- --

Pixel Operations Frame Operations Editing Synchronisation Pixel Operations Frame Operations Editing Synchronisation

Conversion Mixing Indexing Searching Video Effects/Filtering Conversion Mixing Indexing Searching Video Effects/Filtering



Area Manipulation, copy image areas, delete area, process areas of Area Manipulation, copy image areas, delete area, process areas of interest binterest b

Frame Manipulation, Copy frame(s) to frame(s), Process frame(s)Frame Manipulation, Copy frame(s) to frame(s), Process frame(s)

q3) q3) You have been commissioned to produce a Multimedia mail You have been commissioned to produce a Multimedia mail system. What media should be supported in such a mail system. What media should be supported in such a mail system and how should an application facilitate assembly, system and how should an application facilitate assembly, delivery and reading of the mail?delivery and reading of the mail?

SKECTH OF SOULTIOIN REALISATION OF BANDWIDTH/MESSAGE SKECTH OF SOULTIOIN REALISATION OF BANDWIDTH/MESSAGE SIZE and how deal with it important. Also how the application SIZE and how deal with it important. Also how the application deals with media is important.deals with media is important.

Multimedia mail an extension of test-only mail:Multimedia mail an extension of test-only mail:

Basic content still testBasic content still test

Annotated with richer mediaAnnotated with richer media::

ImageImage

Speech/AudioSpeech/Audio

Video ClipVideo Clip

Possible problemsPossible problems

Send all data at once ---- large mail messageSend all data at once ---- large mail message

High BandwidthHigh Bandwidth

High Server OverheadsHigh Server Overheads

Mailboxes need large storageMailboxes need large storage

Mail apps need large memoryMail apps need large memory

Low quality audio and images not a problem if small in Low quality audio and images not a problem if small in number?number?

Terminal being used the read mail supports all media?Terminal being used the read mail supports all media?

Printing etc may lose their MessagePrinting etc may lose their Message

Solution Annotations (tags/links) in main bodySolution Annotations (tags/links) in main body

Annotations can be sent with main message (Explicit Media Annotations can be sent with main message (Explicit Media Inclusion)ORInclusion)OR

Get requested specifically by recipient when message is read or Get requested specifically by recipient when message is read or indeed if media is required.(Implicit Media Inclusion)indeed if media is required.(Implicit Media Inclusion)

Latter method allows reading of mail on more platforms etcLatter method allows reading of mail on more platforms etc

Could send previews of data (i.e. lower resolution audio, images, Could send previews of data (i.e. lower resolution audio, images, image snapshots of video, thumbprints of larger media?)image snapshots of video, thumbprints of larger media?)

Application RequirementsApplication Requirements

Multimedia enabledMultimedia enabled

Good user friendly InterfaceGood user friendly Interface

Good network connectionGood network connection

Good media support of formatsGood media support of formats

Input ---- support for Multimedia AuthoringInput ---- support for Multimedia Authoring

Input ---- support for Graphics/Image/Audio/Video input devicesInput ---- support for Graphics/Image/Audio/Video input devices

Q4) Q4) What is meant by the terms Multimedia and Hypermedia? What is meant by the terms Multimedia and Hypermedia? Distinguish between these two concepts.Distinguish between these two concepts.

Multimedia Multimedia ---- An Application which uses a collection of ---- An Application which uses a collection of multiple media sources e.g. text, graphics, images, multiple media sources e.g. text, graphics, images, sound/audio, animation and/or video.sound/audio, animation and/or video.

HypermediaHypermedia --- An application which uses associative --- An application which uses associative relationships among information contained within multiple relationships among information contained within multiple media data for the purpose of facilitating access to, and media data for the purpose of facilitating access to, and manipulation of, the information encapsulated by the data.manipulation of, the information encapsulated by the data.

Q5) What is meant by the terms static media and dynamic Q5) What is meant by the terms static media and dynamic media? Give two examples of each type of media.media? Give two examples of each type of media.

Static Media – does not change over time, e.g. text, graphicsStatic Media – does not change over time, e.g. text, graphics

Dynamic Media --- Time dependent (Temporal), e.g. Video, Dynamic Media --- Time dependent (Temporal), e.g. Video, sound, animation.sound, animation.

q6) q6) Give a definition of multimedia and a multimedia system?Give a definition of multimedia and a multimedia system?

Multimedia Multimedia

is the field concerned with the computer-controlled is the field concerned with the computer-controlled integration of text, graphics, drawings, still and moving images integration of text, graphics, drawings, still and moving images (Video), animation, audio, and any other media where every (Video), animation, audio, and any other media where every type of information can be represented, stored, transmitted type of information can be represented, stored, transmitted and processed digitally.and processed digitally.

A Multimedia System A Multimedia System

is a system capable of processing multimedia data and is a system capable of processing multimedia data and applications.applications.

Q7) What are the key distinctions between multimedia data and Q7) What are the key distinctions between multimedia data and more conventional types of media?more conventional types of media?

Multimedia systems deal with the generation, manipulation, Multimedia systems deal with the generation, manipulation, storage, presentation, and communication of information in storage, presentation, and communication of information in digital form. digital form.

- The data may be in a variety of formats: text, graphics, images, - The data may be in a variety of formats: text, graphics, images, audio, video. audio, video.

- A majority of this data is large and the different media may need - A majority of this data is large and the different media may need synchronisation -- the data may have temporal relationships synchronisation -- the data may have temporal relationships as an integral property. as an integral property.

- Some media is time independent or static or discrete media: - Some media is time independent or static or discrete media: normal data, text, single images, graphics are examples. normal data, text, single images, graphics are examples.

- Video, animation and audio are examples of continuous media- Video, animation and audio are examples of continuous media

Q8) What key issues or problems does a multimedia system have Q8) What key issues or problems does a multimedia system have deal with when handling multimedia data?deal with when handling multimedia data?

A Multimedia system has four basic characteristics: A Multimedia system has four basic characteristics:

Multimedia systems must be computer controlled. Multimedia systems must be computer controlled.

Multimedia systems are integrated. Multimedia systems are integrated.

The information they handle must be represented digitally. The information they handle must be represented digitally.

The interface to the final presentation of media is usually The interface to the final presentation of media is usually interactive. interactive.

Multimedia systems may have to render a variety of media at the Multimedia systems may have to render a variety of media at the same instant -- a distinction from normal applications. There is same instant -- a distinction from normal applications. There is a temporal relationship between many forms of media (e.g. a temporal relationship between many forms of media (e.g. Video and Audio. There 2 are forms of problems here Video and Audio. There 2 are forms of problems here

Sequencing within the media -- playing frames in correct order/time Sequencing within the media -- playing frames in correct order/time frame in video frame in video

Synchronisation -- inter-media scheduling (e.g. Video and Audio). Synchronisation -- inter-media scheduling (e.g. Video and Audio). Lip synchronisation is clearly important for humans to watch Lip synchronisation is clearly important for humans to watch playback of video and audio and even animation and audio. playback of video and audio and even animation and audio. Ever tried watching an out of (lip) sync film for a long time? Ever tried watching an out of (lip) sync film for a long time?

The key issues multimedia systems need to deal with here are: The key issues multimedia systems need to deal with here are:

How to represent and store temporal information. How to represent and store temporal information.

How to strictly maintain the temporal relationships on play How to strictly maintain the temporal relationships on play back/retrieval back/retrieval

What process are involved in the above. What process are involved in the above.

Data has to represented digitally so many initial source of data Data has to represented digitally so many initial source of data needs to be digitise -- translated from analog source to digital needs to be digitise -- translated from analog source to digital representation. The will involve scanning (graphics, still representation. The will involve scanning (graphics, still images), sampling (audio/video) although digital cameras now images), sampling (audio/video) although digital cameras now exist for direct scene to digital capture of images and video. exist for direct scene to digital capture of images and video.

The data is large several Mb easily for audio and video -- therefore The data is large several Mb easily for audio and video -- therefore storage, transfer (bandwidth) and processing overheads are storage, transfer (bandwidth) and processing overheads are high. Data compression techniques very common.high. Data compression techniques very common.

Q9) Briefly describe eight hardware or software features that a Q9) Briefly describe eight hardware or software features that a Multimedia System should possess.Multimedia System should possess.

The following features should be present:The following features should be present:

o o Very High Processing PowerVery High Processing Power

needed to deal with large data processing and real time delivery needed to deal with large data processing and real time delivery of media. of media.

Special hardware commonplace.Special hardware commonplace.

o o Multimedia Capable File SystemMultimedia Capable File System

needed to deliver real-time media -- needed to deliver real-time media -- e.g. Video/Audio Streaming.e.g. Video/Audio Streaming.

Special Hardware/Software needed Special Hardware/Software needed e.g RAID technology.e.g RAID technology.

o o Data Representations/File Formats that support multimediaData Representations/File Formats that support multimedia

Data representations/file formats should be easy to handle yet Data representations/file formats should be easy to handle yet allowallow

for compression/decompression in real-time.for compression/decompression in real-time.

Efficient and High I/OEfficient and High I/O

input and output to the file subsystem needs to be efficient and input and output to the file subsystem needs to be efficient and fast.fast.

Needs to allow for real-time recording as well as playback of data.Needs to allow for real-time recording as well as playback of data.

e.g. Direct to Disk recording systems.e.g. Direct to Disk recording systems.

o o Special Operating SystemSpecial Operating System

to allow access to file system and process data efficiently andto allow access to file system and process data efficiently and

quickly. Needs to support direct transfers to disk, real-timequickly. Needs to support direct transfers to disk, real-time

scheduling, fast interrupt processing, I/O streaming scheduling, fast interrupt processing, I/O streaming etc.etc.

o o Storage and MemoryStorage and Memory

large storage units (of the order of 50 -100 Gb or more) and largelarge storage units (of the order of 50 -100 Gb or more) and large

memory (50 -100 Mb or more). Large Caches also required andmemory (50 -100 Mb or more). Large Caches also required and

frequently of Level 2 and 3 hierarchy for efficient management.frequently of Level 2 and 3 hierarchy for efficient management.

o o Network SupportNetwork Support

Client-server systems common as distributed systems common.Client-server systems common as distributed systems common.

o o Software ToolsSoftware Tools

user friendly tools needed to handle media, design and developuser friendly tools needed to handle media, design and develop

applications, deliver media.applications, deliver media.

q10) The main types of multimedia data are: graphics, images, q10) The main types of multimedia data are: graphics, images, audio,audio,

s and video. What technical issues are associateds and video. What technical issues are associated

when these data types are integrated in a Multimedia System?when these data types are integrated in a Multimedia System?

For each media type briefly relate to the issues involved in For each media type briefly relate to the issues involved in generating,generating,

capturing, storing and transmitting the respective media capturing, storing and transmitting the respective media components.components.

Multimedia issues :Multimedia issues :

o Sequencing within the media o Sequencing within the media -- playing frames in correct -- playing frames in correct order/time frame in videoorder/time frame in video

o Synchronisation -- inter-media schedulingo Synchronisation -- inter-media scheduling

o How to represent and store temporal information.o How to represent and store temporal information.

Data Types:Data Types:

GraphicsGraphics

Source: Graphics input devices include: keyboard (for text and Source: Graphics input devices include: keyboard (for text and cursor control), mouse, cursor control), mouse,

trackball or graphics tablet.trackball or graphics tablet.

Storage: Graphics are usually constructed by the composition of Storage: Graphics are usually constructed by the composition of primitive objects such asprimitive objects such as

lines, polygons, circles, curves and arcs. Graphics are usually lines, polygons, circles, curves and arcs. Graphics are usually generated by a graphicsgenerated by a graphics

editor program ( editor program ( e.g. Freehand) or automatically by a program ( e.g. e.g. Freehand) or automatically by a program ( e.g. Postscript usuallyPostscript usually

generated this way). Graphics are usually editable or revisable generated this way). Graphics are usually editable or revisable (unlike Images).(unlike Images).

Transmission: Graphics files usually store the primitive assembly Transmission: Graphics files usually store the primitive assembly and do not take up aand do not take up a

very high overhead for bandwidth (e.g. Flash Vector Graphics)very high overhead for bandwidth (e.g. Flash Vector Graphics)

ImagesImages

Source: Images may be generated by programs similar to graphics Source: Images may be generated by programs similar to graphics or animationor animation

programs. But images may be scanned for photographs or pictures programs. But images may be scanned for photographs or pictures using a digital scannerusing a digital scanner

or from a digital camera. Some Video cameras allow for still image or from a digital camera. Some Video cameras allow for still image capture also. Analogcapture also. Analog

sources will require digitising.sources will require digitising.

Storage: Images may be stored at 1 bit per pixel (Black and White), Storage: Images may be stored at 1 bit per pixel (Black and White), 8 Bits per pixel8 Bits per pixel

(Grey Scale, Colour Map) or 24 Bits per pixel (True Colour). Thus a (Grey Scale, Colour Map) or 24 Bits per pixel (True Colour). Thus a 512x512 Grey scale512x512 Grey scale

image takes up 1/4 Mb, a 512x512 24 bit image takes 3/4 Mb with no image takes up 1/4 Mb, a 512x512 24 bit image takes 3/4 Mb with no compression.compression.

Transmission: The storage overhead increases with image size so Transmission: The storage overhead increases with image size so compression iscompression is

CMP632 Multimedia Systems SOLUTIONSCMP632 Multimedia Systems SOLUTIONS

commonly applied at source for bandwidth/storage e.g. JPEGcommonly applied at source for bandwidth/storage e.g. JPEG

AudioAudio

Source: They are first captured by a microphones and then Source: They are first captured by a microphones and then digitised and stored ondigitised and stored on

diskdisk

Storage: usually compressed as CD quality audio requires 16-bit Storage: usually compressed as CD quality audio requires 16-bit sampling at 44.1sampling at 44.1

KHz. So 1 Minute of Mono CD quality audio requires 60*44100*2 KHz. So 1 Minute of Mono CD quality audio requires 60*44100*2 BytesBytes

which is approximately 5 Mb.which is approximately 5 Mb.

Transmission/Bandwidth: Maybe be compressed (MPEG) perhapsTransmission/Bandwidth: Maybe be compressed (MPEG) perhaps

streamed/copressed (Realaudio). Wav files may not be compressedstreamed/copressed (Realaudio). Wav files may not be compressed

VideoVideo

Source:Analog Video is usually captured by a video camera and Source:Analog Video is usually captured by a video camera and then digitised.then digitised.

May be digitised at source (Digital Video Camera)May be digitised at source (Digital Video Camera)

Storage: Raw video can be regarded as being a series of single Storage: Raw video can be regarded as being a series of single images. There areimages. There are

typically 25, 30 or 50 frames per second. Therefore a 512x512 sizetypically 25, 30 or 50 frames per second. Therefore a 512x512 size

monochrome video images take 25*0.25 = 6.25Mb for a minute to monochrome video images take 25*0.25 = 6.25Mb for a minute to storestore

uncompressed. Digital video clearly needs to be compressed.uncompressed. Digital video clearly needs to be compressed.

Transmission/Bandwidth: Maybe be compressed (MPEG) perhapsTransmission/Bandwidth: Maybe be compressed (MPEG) perhaps

streamed/copressed (Realvideo/Quicktime).streamed/copressed (Realvideo/Quicktime).

14 Marks Total: Unseen --- Assimilation/Extended reasoning of a 14 Marks Total: Unseen --- Assimilation/Extended reasoning of a few parts offew parts of

the coursethe course

Q11) Briefly describe five ways in which content can be Q11) Briefly describe five ways in which content can be formatted and delivered in a Multimedia Authoring formatted and delivered in a Multimedia Authoring System.System.

1. Scripting (writing)1. Scripting (writing)

Standard Text --- say what you want with wordStandard Text --- say what you want with word

2. Graphics (illustrating)2. Graphics (illustrating)

“ “A picture is worth a thousand words” say what you want with a A picture is worth a thousand words” say what you want with a graphic illustrationsgraphic illustrations

3. Animation (wiggling)3. Animation (wiggling)

Now we approach multimedia --- say what you want with a Now we approach multimedia --- say what you want with a graphic animation or videographic animation or video

4. Audio (hearing)4. Audio (hearing)

Sounds can convey alerts, ambience and contents say what Sounds can convey alerts, ambience and contents say what you want with a narrationyou want with a narration

5. Interactivity (interacting)5. Interactivity (interacting)

True mulitedia immerse yourself in am interactive True mulitedia immerse yourself in am interactive presentation, possibly more instructive. Interactive actions presentation, possibly more instructive. Interactive actions can start animations, audio, move to new parts of can start animations, audio, move to new parts of presentation, control simulations etc.presentation, control simulations etc.

Introduction to Multimedia. 1- What is Multimedia? 2- Multimedia and Hypermedia 3- World Wide Web 4-...

Documents

Transcript of Introduction to Multimedia. 1- What is Multimedia? 2- Multimedia and Hypermedia 3- World Wide Web 4-...