Clarke, R. J (2001) L213-08: 1 Multimedia in Organisations BUSS 213 Lecture 8 Modelling Multimedia...

Clarke, R. J (2001) L213-08: 1

Multimedia in Organisations

BUSS 213

Lecture 8Modelling Multimedia Applications

Clarke, R. J (2001) L213-08: 2

NoticeAssignment 1 Results

Pick up your assignment 1 if you haven’t already done so:

check the marks sheet and make sure that marks have been included for all categories (where relevant) and that it is added up correctly

make sure that the total shown on the cover is the same as that shown on the results list (on my office door)

the final grade for the assignment will be rounded to an integer

Clarke, R. J (2001) L213-08: 3

Agenda (1)

in previous lectures we have considered the different types of media used in multimedia systems and the types of transformations performed on them

we have also considered multimedia systems from the point of view of the kind of design considerations used when building the interface

this lecture we consider what is behind the interface, the software architecture (abstractions) for multimedia implementation…

Clarke, R. J (2001) L213-08: 4

Agenda (2)

we consider the design of multimedia applications from an object-oriented approach (OOHDM)

then consider some of the types of building blocks or classes used in multimedia frameworks

these frameworks consist of media types and their properties and operations are also viewed using object-oriented programming

Clarke, R. J (2001) L213-08: 5

Object-Orientation

Clarke, R. J (2001) L213-08: 6

Object-orientation

In this lecture we consider the use of so-called ‘object-oriented’ development techniques as the basis for:Design of multimedia applicationsImplementation of Multimedia

Applications we describe both the design and

implementation aspects of multimedia as ‘modelling’ for our purposes

in this lecture we emphasis design

Clarke, R. J (2001) L213-08: 7

Object-orientation

thankfully however the implementation details of multimedia systems are hidden by the high-level tools we use during development (Director etc)

in this section we describe the general characteristics of object-orientation- these are broadly applicable in other development contexts as well

object-oriented approaches are a natural fit with multimedia systems- there are several reasons for this

Clarke, R. J (2001) L213-08: 8

Object-orientationMain Properties (after Borland 1991)

there are three main properties which characterise object-oriented development and languages which support object-orientation: Encapsulation: combining a record with the procedures

and functions that manipulate it to form a new data type- an object

Inheritance: defining an object and then using it to build a hierarchy of descendant objects, with each descendant inheriting access to all its ancestors’ code and data

Polymorphism: given an action one name that is shared up and down an object hierarchy, with each object in the hierarchy implementing the action in a way appropriate to itself

Clarke, R. J (2001) L213-08: 9

Object-orientationInheritance (after Borland 1991)…

as with other sciences and technologies a great deal of analysis and development is about classification- the creation of distinctions and there organisation into taxonomies

consider the category of ‘insect’, which has two divisions: Insects with visible wings, and Insects with hidden or no wings

Clarke, R. J (2001) L213-08: 10

Object-orientation… Inheritance (after Borland 1991)…

‘winged insects’ contain a large number of categories including moths, butterflies, flies etc. and each category has many subcategories

a scientist asks several questions when trying to classify a new animal or object How is it similar to the others of its general

class? How is it difficult?

Clarke, R. J (2001) L213-08: 11

Object-orientation… Inheritance (after Borland 1991) …

each different class has a set of behaviours and characteristics that define it

a scientist begins at the top of the tree and asks questions along the way eg. ‘wings’ or ‘no wings’ from the most general to the most specific

once a characteristic is defined all the categories beneath that definition include that characteristic…

Clarke, R. J (2001) L213-08: 12

Object-orientation… Inheritance (after Borland 1991)

once a new insect has been identified as a member of ‘flies’ you don’t need to identify your fly as having wings!

the new insect is said to inherit that characteristic from ‘flies’

insects

wingless winged

butterflies fliesmoths

new insect

Clarke, R. J (2001) L213-08: 13

Object-orientationPolymorphism (after Borland 1991) …

polymorphism is based on the Greek meaning ‘many shapes’- a way of giving an action one name that is shared up and down an object hierarchy with each object in the hierarchy implementing the action in a way appropriate to itself

the winged insects taxonomy is a good example of polymorphism in action- we would like each insect to be drawn on the screen for example- in fact this is a desirable property for all of our electronic insects…

Clarke, R. J (2001) L213-08: 14

Object-orientation… Polymorphism (after Borland 1991)

…but what is different for each object type is the way it must show itself on the screen for example bees have stripes on their body, but dragonflies don’t

part of the basic definition of insect would also implement so called virtual method called ‘Show’- which is resolved at program execution to display on the screen a bee rather than a dragonfly

Clarke, R. J (2001) L213-08: 15

Multimedia Design

Clarke, R. J (2001) L213-08: 16

Multimedia DesignObject Oriented Hypermedia Design Method

an Object Oriented Hypermedia Design Method (OOHDM) has evolved over recent years which applies a set of object-oriented design primitives to hypermedia design

Comprises four different activities: Conceptual Design Navigation Design Abstract Interface Design, and Implementation

Clarke, R. J (2001) L213-08: 17

Multimedia DesignOOHDM Activities …

each activity is performed as a mix of incremental, iterative and prototype-based development styles

during each activity a set of object-oriented models describing particular design concerns are built from previous iterations

Conceptual, navigation and interface design are separated in order to allow developers to concentrate on each one at a time

Clarke, R. J (2001) L213-08: 18

Multimedia Design… OOHDM Activities

the result of this separation is modular, reusable designs and produces a framework for reasoning about the design process while encapsulating design experience specific to each activity

design primitives can be mapped directly into non-object-oriented implementation languages or environments (eg. HTML) or into pure object-oriented environments

we describe each OOHDM’s activity in detail (we exclude implementation- considered latter)

Clarke, R. J (2001) L213-08: 19

Multimedia DesignOOHDM Conceptual Modelling (1)

Goal is to build a model of the application domain using object-oriented modelling principles and notations(like UML)

Output is a class schema where conceptual classes are built using aggregation and hierarchies like the taxonomy shown previously

Clarke, R. J (2001) L213-08: 20

Multimedia DesignOOHDM Conceptual Modelling (2)

Modelling conceptual entities as objects allows behaviour to be encapsulated inside them and therefore improves the design of complex applications

Conceptual models don’t just model the ‘domain’ of users and tasks but also can include complex algorithms and transactions applied to computational entities- permits a broad range of applications to be modelled

Clarke, R. J (2001) L213-08: 21

Multimedia DesignOOHDM Navigation Design

OOHDM does not prescribe any particular procedure to create navigation designs- but it does provide a method to iteratively develop them

Uses scenarios as a means to do this…

1. Determine user profiles (types of users), identify user tasks

2. Scenario collection3. Analysis of scenarios- create a simple

representation of the navigation path described by the scenario (the designer will have to fill in incomplete information, such as implied indexes, missing orderings, exception handling, etc..)

4. Synthesis of a partial context diagram, specifying navigation in contexts that support the task described in the scenario

5. Synthesis of final context diagrams through a process of union and amalgamation of partial schemas, produced in the analysis of each scenario (in Step 4)

Clarke, R. J (2001) L213-08: 22

Multimedia DesignOOHDM Abstract Interface Design

in the stage the interface objects that will be perceived by the user are defined

there is a distinction between navigation operations and interface operations- not everything in an interface is navigation oriented

it it useful to design interfaces in an abstract way- to achieve for example independence in the implementation environment

Clarke, R. J (2001) L213-08: 23

Multimedia DesignOOHDM Design Methods (Rossi et al 1999)

Conceptual ModelNavigation Views

Object Oriented Views of Conceptual Model

Interface Model(Presentation)

Clarke, R. J (2001) L213-08: 24

Multimedia Framework

Clarke, R. J (2001) L213-08: 25

Multimedia Framework

multimedia represents a challenging type of programming with respect to detailed technical implementation because of: the variety of different types of media that have to be

handled and the fact that the characteristics of the hardware change all

the time (non-standardisation as well as technical development)

an advantage of OOP is that it can encapsulate information that is hide details of the particulars of a given media and the specifics of particular hardware

programmers only need to know what is referred to confusingly as the interface (object NOT user interface) rather than the details of the object implementation

Clarke, R. J (2001) L213-08: 26

Multimedia Framework Digital Content Workflows and Modularity

the workflows used in developing multimedia content are increasingly becoming digital (as we have seen in previous lectures for example when we discussed VR)

modern digital content workflows now include what used to be standalone and isolated devices- these devices are becoming virtually integrated together at the desk top or are now available as software rather than hardware

OOP can handle the complex interfacing needed to support interconnectivity and modularity between functions in digital content workflows- the interface between for example an audio effects processor (Sound Forge) and a digital sequencer (CakeWalk)

Clarke, R. J (2001) L213-08: 27

Multimedia Framework Extensibility & Portability

the requirements of multimedia systems are constantly evolving and relatively non-standard- this means that there will always be a need to extend applications in ways which are not always predictable

OOP supports this so called extensibility by providing standard ways in which programs can be enhanced

multimedia systems should be cross-platform as much as this is possible- in OOP the platform dependencies are made explicit in order for the code to be made portable - this means that the application need not be completely rewritten

as demonstrated in the following slides…

Clarke, R. J (2001) L213-08: 28

Object-Oriented MultimediaSpecialising a Framework

Abstract classes

Concrete classes

Platform 2

Abstract classes

Concrete classes

Platform 3

Abstract classes

Concrete classes

Platform 1

Abstract classes

Specialisation

Environments

Framework

Systems programming interfaces

Application programming interface (API)

Clarke, R. J (2001) L213-08: 29


Abstract classes

Concrete classes

Platform 2

Abstract classes

Concrete classes

Platform 3

Abstract classes

Concrete classes

Platform 1

Clarke, R. J (2001) L213-08: 30


Abstract classes

Concrete classes

Abstract classes

Concrete classes

Abstract classes

Concrete classes

Platform 2 Platform 3Platform 1

Clarke, R. J (2001) L213-08: 31


Abstract classes

Concrete classes

Abstract classes

Concrete classes

Abstract classes

Concrete classes


Clarke, R. J (2001) L213-08: 32


Abstract classes

Concrete classes

Abstract classes

Concrete classes

Abstract classes

Concrete classes


Clarke, R. J (2001) L213-08: 33


Abstract classes

Concrete classes

Abstract classes

Concrete classes

Abstract classes

Concrete classes


Clarke, R. J (2001) L213-08: 34


Abstract classes

Concrete classes

Abstract classes

Concrete classes

Abstract classes

Concrete classes


Clarke, R. J (2001) L213-08: 35


Abstract classes

Concrete classes

Abstract classes

Concrete classes

Abstract classes

Concrete classes


Clarke, R. J (2001) L213-08: 36


Abstract classes

Concrete classes

Abstract classes

Concrete classes

Abstract classes

Concrete classes


Clarke, R. J (2001) L213-08: 37


Classes ClassesClasses


Clarke, R. J (2001) L213-08: 38




Clarke, R. J (2001) L213-08: 39




Clarke, R. J (2001) L213-08: 40




Clarke, R. J (2001) L213-08: 41




Clarke, R. J (2001) L213-08: 42




Clarke, R. J (2001) L213-08: 43

Application Application Application




Clarke, R. J (2001) L213-08: 44

Media Classes

Clarke, R. J (2001) L213-08: 45

Media Classes (1)

the low-level implementation of distinct types of media- that is the computer-based representation of media- can also described in object-oriented terms

in this section we consider how to model different media types and there properties and operations using object-oriented programming

the resulting building blocks are referred to as classes which can then be directly included into programs or further extended

Clarke, R. J (2001) L213-08: 46

Media Classes (2)

Gibbs and Tsichritzis (1995) in a now ‘classic work’ showed how media could be considered in detail using a multimedia framework which includes: Media classes Transform Classes Format Classes Component Classes

here we only briefly describe media classes in object oriented terms (in the following slides)

however the student with a technical background and interest is referred to Gibbs and Tsichritzis (1995) for further discussion

Clarke, R. J (2001) L213-08: 47

Media Classes (3)Objects, Instance Variables, Methods …

Objects are a programming constructs that encapsulate or contain its state- a collection of data items- and its behaviour- collection of operations

the object’s data items are called instance variables and its operations are called its methodsObject: myMp3PlayerInstance Variable: currentTrackMethod: PlayTrack(trackNumber)

Clarke, R. J (2001) L213-08: 48

Media Classes (4)… Objects, Instance Variables, Methods …

invoking or using a method can be thought of as sending a message to an object

this message contains both the method name and any arguments needed by the method

the syntax used to invoke the method method differs from language to language (and the stuff of vigorous debate) but the following form of object name followed by method- is commonly used:

myMp3Player.PlayTrack(4)

Clarke, R. J (2001) L213-08: 49

Media Classes (5)… Objects, Instance Variables, Methods …

the result of invoking a method often leads to a change of state of the object for example currentTrack instance variable is now 4

in fact in many OOP languages, the methods of objects are often the only interface by which objects are manipulated- you cannot modify an objects instance variables directly

Clarke, R. J (2001) L213-08: 50

Media Classes (6)Class Hierarchy Diagrams

following the work of Gibbs and Tsichritzis (1995) classes can be shown as class hierarchy diagrams ovals are classes lines depict superclass/subclass relationships the line points from superclass to the subclass classes (ovals) in dark blue are abstract classes

while those in pale pink are concrete classes

media are divided into types- each type is represented by a class…

Clarke, R. J (2001) L213-08: 51

Media Classes (7)…Class Hierarchy Diagrams

Media

Text

Graphic

ImageTemporalMedia

Audio

VideoMusic

Animation

Clarke, R. J (2001) L213-08: 52

Media Classes (8)Media Objects…

instances of media classes are called media objects

a media object consists of a descriptor and and a media value

descriptors bundle together attributes of media objects such as their values correspond to the actual data used to represent artifacts

Clarke, R. J (2001) L213-08: 53

Media Classes (9)… Media Objects…

010010010110010001

Media value(media data)

Mediaartifact

Mediadescriptor

represents

describes

MediaObject

Clarke, R. J (2001) L213-08: 54

Media Classes (10)General Categories of Methods

most general media classes have two general categories of methods:querying- allows media to determine the

media types that are present in an environment

editing- cut, copy and paste operations for any general media type

Clarke, R. J (2001) L213-08: 55

Media Classes (11)Video Classes & Sequence Elements

RawVideo

Video

CompressedVideo

MpegVideo

Frame

VideoElement

CompressedFrame

MpegFrame

Instances of Classes Corresponding Classes

Clarke, R. J (2001) L213-08: 56

Media Classes (12)Audio Classes & Sequence Elements

RcmAudio

Audio

CompressedAudio

AdpcmAudio

Sample

AudioElement

CompressedSample

AdpcmSample


Adpcm and AdpcmSample are abstract classes

Clarke, R. J (2001) L213-08: 57

Media Classes (13)Animation Classes & Sequence Elements


Adpcm and AdpcmSample are abstract classes

Animation

EventBasedAnimation SceneBasedAnimation

AnimationElement

AnimationEvent Scene

Clarke, R. J (2001) L213-08: 58

Media Classes (14)Music Classes & Sequence Elements

EventBasedMusic

MidiMusic

ScoreElement

MusicElement

MidiEvent


ScoreBasedMusic

Music

MusicEvent

Clarke, R. J (2001) L213-08: 59

References

Gibbs, S. J. and D. C.Tsichritzis (1995) Multimedia Programming: Objects, Environments and Frameworks Chapter 2: Media Types, ACM Press Books/Addison-Wesley Publishing Company, 15-78

Rossi, G.; Schwabe, D. and F. Lyardet (1999) “Integrating Patterns into the hypermedia development process” The New Review of Hypermedia and Multimedia 1999 (3), 59-80

Borland, Inc. (1991) Turbo Pascal for Windows: User Guide Borland International

Clarke, R. J (2001) L213-08: 1 Multimedia in Organisations BUSS 213 Lecture 8 Modelling Multimedia...

Documents

Transcript of Clarke, R. J (2001) L213-08: 1 Multimedia in Organisations BUSS 213 Lecture 8 Modelling Multimedia...