Making Wealthcare Easier David B. Loeper, CIMA, CIMC Chairman & CEO.
Rutgers - CIMC Automatic Manifestation of Composite Multimedia Objects.
-
date post
15-Jan-2016 -
Category
Documents
-
view
220 -
download
5
Transcript of Rutgers - CIMC Automatic Manifestation of Composite Multimedia Objects.
Rutgers - CIMC
Automatic Manifestation of Composite Multimedia Objects
Rutgers - CIMC 2
Outline• Motivating Example
• Introduction to CMO
• Types of constraints
• Universal Access
• Architecture
• Framework– CMO specification
– Petri net layer
• MMCTPN
– SMIL layer
• Demo
• User Interaction
• Related work
• Future Work
Rutgers - CIMC 3
Current Situation
• What to do to select your business location?
Rutgers - CIMC 4
Current Situation
• Easy to get lost between the different links.
•Inserts the profile several times because dealing with different web sites ( state /realtors).
Rutgers - CIMC 5
Composite multimedia Object (CMO)
Rutgers - CIMC 6
Composite multimedia Object (CMO)
– Comprised of different media component such as text, image, video, audio, shape files,...
– Variety of relationships among components.
– Relationships represent different types of constraints (e.g., sync.), that must be adhered to when rendering it.
– Associated with each component is a set of parameters, such as playback duration.
Rutgers - CIMC 7
Types of Constraints
• Synchronization constraints– Map must appears in sync.with
Text 1
• Fidelity Constraints– Map must be displayed at a
resolution of at least 640*480
• Spatial Constraints– Fly by is above Text
• Security Constraints– Sales representative audio and video
are accessed by subscribers only.
Rutgers - CIMC 8
CMO
Over Any Network
To Any One with varying expertise, capabilities, and preferences
To Any Device
Universal Access
• When a subject requests a CMO, he may not be able to view the entire CMO.– For example, due to the limitations of his
appliances, or due to lack of his credentials to satisfy the security requirements.
• Need to facilitate access to desired CMO according to the various user’s:
• Capabilities ( e.g, devices)
• Characteristics (e.g, expertise)
• Credentials (e.g, subscribers)
Rutgers - CIMC 9
Architecture
• The server comprises of an object manager that handles the requests from the clients.
• It may also provide tools to author multimedia objects.
• We represent the security, synchronization,fidelity and spatial constraints along with the playback duration of each component in the object plan.
Rutgers - CIMC 10
Architecture
• Oblet A small piece of software installed on the client side, to support objects with ability to decide on their own renderings (manifestations)
• The object manifestation is comprised of object plan modification,object delivery and object rendition.
• Minimize the responsibilities of the server in order to avoid congestion and overloading at the server.
Rutgers - CIMC 11
Framework• CMO layer comprises of the formal
specification of the CMO, such that specifying each components in the CMO along with its associated parameter and the relationships between different components.
• Petri Net layer is a conceptual tool for automatic manifestation, description, analyzing, visualization and validating of the CMO.
• SMIL layer is for implementation and automatic rendering purpose.
Rutgers - CIMC 12
CMO specification
• CMO = ({c 1 ...,c n}, S ):– Each c i is component of CMO
– S is a set of synchronization,spatial,fidelity,security and author constraints.
• Each component c i in turn,is a tuple c =(ml,fd,pd):– ml represents the modality– fd represents the fidelity– pd represents the playback duration
Rutgers - CIMC 13
Constraints• Synchronization Constraints
– We only need 3 from the 7 temporal
relations
– Meets: video meets image
– Sync: video sync text
– Before:
video imagevideo image
video
text
video1 video2
Rutgers - CIMC 14
Constraints
• Fidelity Constraints
– A set of fidelity constraints for modality ml .
– FD (ml )is set of fidelity ranges fd
– for example >= 640 * 480 is to specify that the resolution
be at least 640 *480 to view the object.
• Spatial Constraints– Where to place the components on the screen
Rutgers - CIMC 15
Constraints
• Security Constraints
• Let CR = {cr 1, cr 2 ... }denote the set of all distinct
credentials relevant to the multimedia object.
• Each subject S possesses set of credentials
denoted as s CR
• A security constraint can be:
– (1)Each component c i of the multimedia object is
associated with set of credentials,denoted c i CR .
– (2)A set of components (c i c j ,...)is associated with a
set of credentials,denoted (c i c j ,...)CR .
• E.g., video2 ( v2) need subscription credentials so,
C i CR = V 2 subscription
C i CR = V 2
subscription
Rutgers - CIMC 16
Petri net layer
• Ensure the satisfaction of different types of constraints.
• Conflict identification and resolution.
• Visualization.
Rutgers - CIMC 17
Place/Transition Net
• Place
• Transition
• Token
t1
t0
p0
t2
p1 p2
p3 p4
Rutgers - CIMC 18
Colored Petri Nets (CPN)
Before firing:
After firing with substitution {a|x, b|y, c|z}:
When firing:
1. p1 loses 2 tokens of x;2. p2 loses <x,y> and <y,z>;3. p3 gets <x, z>;4. p4 get e (constant).
Initial marking:1. p1 has 4 tokens, 2 a’s
and 2 d’s;2. p2 has 6 tokens, <a,b>,
<b,c>, <d,a>;3. p3 and p4 has no
tokens.
Rutgers - CIMC 19
CMO specification to Petri Net
• To model a multimedia object plan
• Called MMCTPN (Multimedia color-time Petri-Net)
Rutgers - CIMC 20
MMCTPN Model for a CMO plan
• MMCTPN (Multimedia color-time Petri-Net) consists of– Color-time Petri-Net
– Tokens: Color sets that represent:• Types of Fidelity (e.g.,the resolution of the MM component)
– Absence of fidelity capabilities is represented as Holes• Types of credentials (e.g., credentials for researchers)
– Absence of credentials of the subject requesting the object is represented as Holes
• Types of Modality (e.g., image, video,text, audio)– Absence of modality capabilities is represented as Holes
– Places:• Includes the length of time the multimedia component is played.• A color set that is needed to activate/ play the component.• Null place in each subMMCTPN (A part of the net between two subsequent
transitions)
Rutgers - CIMC 21
MMCTPN
• Credentials and capabilities are represented as tokens and absence of them are represented as holes.
• Author rules are represented as doted holes.
• The initial place is marked with these tokens to start with.
• Tokens are said to be available if it remains in a place for their specified duration.
• Transitions are enabled once any token or hole are available in all input places.
tt
Before FiringPa , dur:10Pa , dur:10 Pa , dur:15Pa , dur:15
After Firing
Pa , dur:10Pa , dur:10 Pa , dur:15Pa , dur:15
t = 0t = 0
Rutgers - CIMC 22
MMCTPN – Object Manifestation
Dur = 10 S
AA
CC
Dur = 0 S
BB
Rutgers - CIMC 23
P2
P1
Ps Pf
P5
P3
null
P9
P6
null
P4
P8
P7
null
<or,gy,br> dur :40s
<all> dur :40s
<bl,gy,br> dur :40s
<re,bc,br> dur :15s <ye,bc,br> dur :10s
<bl,wh,pp> dur :15s
<all> dur :15s
<bl,wh,pp> dur :15s
<gr,br,aq> dur :15s
<re,gy,br,aq> dur :15s
<bl,wh,br> dur :15s
<all> dur :15s
CMO Plan
Rutgers - CIMC 24
MMCTPN execution
<or,gy,br> dur :40s
P2
P1
Ps Pf
P5
P3
null
P9
P6
null
P4
P8
P7
null
<all> dur :40s
<bl,gy,br> dur :40s
<re,bc,br> dur :15s <ye,bc,br> dur :10s
<bl,wh,pp> dur :15s
<all> dur :15s
<bl,wh,pp> dur :15s
<gr,br,aq> dur :15s
<re,gy,br,aq> dur :15s
<bl,wh,br> dur :15s
<all> dur :15s
Rutgers - CIMC 25
Modified CMO Plan
P2
Ps Pf
P5
null
P9
P6
null
P4
null
<all> dur :40s
<bl,gy,br> dur :40s
<ye,bc,br> dur :10s
<bl,wh,pp> dur :15s
<all> dur :15s
<bl,wh,pp> dur :15s
<gr,br,aq> dur :15s
<bl,wh,br> dur :15s
<all> dur :15s
Rutgers - CIMC 26
Petri Net To SMIL
Equals ( SYNC):
<par dur="30s">
<img id="foo" src="a.jpg"/>
<text src="text.html" />
<audio src="audio.au" />
</par>
|------------| image
|------------| text
|------------| Audio
30s
After ( Before):
<seq>
<img src="a.gif" dur="6s" />
<img src="b.gif" dur="4s" begin="1s" />
</seq>
|----------|
6 sec |---------|
4sec
|--------------------------|
11 seconds
Rutgers - CIMC 27
SMIL layer
• SMIL-based rich multimedia presentations integrate several types of media.
• SMIL is XML-based, thus flexible and extensible.
• SMIL is W3C supported, thus enjoys cross industry support.
• SMIL is supported by readily available tools and parsers.
Rutgers - CIMC 28
Demo
http://cimic.rutgers.edu/~ahgomaa/ua/presentations/demo3/egov.smil
http://cimic.rutgers.edu/~ahgomaa/ua/presentations/demo3/egov-adj.smil
Rutgers - CIMC 29
SMIL 2.0 Implementation
we have adopted the security model and the language (XACL) in specifying the security constraints on the multimedia object. We have used the XACL visual tool,tool from IBM XML Security Suite.
Rutgers - CIMC 30
Incorporating User interactivity
• Identify and model the different types of user interactions.
• Make sure that different constraints are still valid.
• Visualize and analyze the effect of the user interaction on different constraints.
Rutgers - CIMC 31
User interaction categories• Temporal Interaction:
– continuous component, ( ex. video )• ''play'', '‘fast forward''
– non-continuous component(ex. image) • "begin", "end", "pause".
• Spatial interaction:• "move “, "Zoom" ,” maximize”, "change font".
• Undo – Redo
• The user may want to interact with the CMO as one object, e.g., pause , begin,…
Rutgers - CIMC 32
User Interaction example
• If a user wants to stop the fly by video, what will be the effect on the following text? (rendering)– Appear immediately after stop.
– Appear after a defined amount of time.
• If a user wants to maximize the map, what will be the effect on rendering the following image? ( conflict)– Rendered behind the map.
– Rendered after another action.
• The CMO author need to:– Visualize
– Analyze
Rutgers - CIMC 33
User Interaction Problems
How to model different types of user interaction on the composite multimedia object plan?
How can the author visualize and analyze the impact of potential user interaction on the rendering of the CMO?
How to detect potential conflicts between different types of constraints?
How to resolve different conflicts between constraints?
Rutgers - CIMC 34
CMO constraints Uniform Model
Conflict resolution
PETRI-NET Modeling for validation
and analysis
Implementation with
user/web friendly with flexibility
and interoperability (SMIL 2.0)
Rutgers - CIMC 35
Related Work
• Multimedia object rendering
• Security
• User interaction
Rutgers - CIMC 36
Related Work
Multimedia Object Rendering:• Adam and Atluri et al.(2001) Presented the Universal Access
problem and the Oblet approach. – Uses Petri Net to present temporal, spatial, modality and fidelity
constraints.
• Bertino et al.(2000) present a system called MPGS - Multimedia Presentation Generator System.– Enables specification of synchronization and spatial constraints
– Capable of analyzing the consistency among these two types of constraints.
None of those papers addressed the security constraints in a CMO
Rutgers - CIMC 37
Related Work
Security:• Damiani et al.(2000),Bertino et Al.(2000) and Kudo et al (2000) addressed
access control models in XML.
• We adopted Kudo et al. approach in presenting a new Petri Net model that incorporate the security constraints in the CMO.
Rutgers - CIMC 38
Related WorkUser interaction:• User interaction research within CMO may be divided into three categories:
– Formal modeling
– Programming approach
– Statistical techniques
• Formal modeling, using Petri net :– Song et al.(1996) presented Timed Petri Net (TPN).
– Guan et al. (1998) presented a distributed object composition Petri net (DOCPN).
– Prabhakaran et al.(1993) presented a dynamic timed Petri nets (DTPN) model.
– All models are restrictive in types of interactions.
– They only deal with the synchronization constraints.
Rutgers - CIMC 39
Related Work• Programming approach
– Partially support user interactions for online CMO.
– LimSee(2002), Grins (2002), Yang (2001).
– All of which based on time line events
– which leads to dead times if the CMO is adaptable.
• Statistical techniques – Predict online user interactions
– Hollfelder et al. (2000) where they model the user behavior as a Continuous Time Markov chain (CTMC).
– Boll et al.(2001) implementing adaptive streaming of MPEG videos for interactive internet applications to support jumping to bookmarks within the MPEG
Rutgers - CIMC 40
Future Work
• Enforcing Security Constraints.• Secure the plan on the client side ( certificates, relational transducer).
• Allowing User Interaction.
• Conflict Identification and Resolution Strategy.