March 9, 2008

Vermelding onderdeel organisatie

1

SEARIS'08 Workshop

IEEEVR 2008, Reno NV, USA

Gerwin de Haan

G. de Haan,F.H. Post

Data Visualization Group & VRLabhttp://visualization.tudelft.nl/VRVis

Flexible Architecture for the Development of Real-Time Interaction Behavior (p.71)

March 9, 2008 2

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

Real-Time Interaction Behavior

(VR) application behavior is interaction/event-based

Interaction Trends:• Multi – Object• Multi – User• Multi – Cursor• Multi – System

Interaction Trends:• Multi – Object• Multi – User• Multi – Cursor• Multi – System

Development Cycle:

• Implementation• Architecture• Interaction Model• Front-End• (Storyboard)

Unexpected complexity,

behavior, structure

Prototype, Experiment,

Analyse

March 9, 2008 3

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

Work in Progress

Prototyping Interaction Software:

• Base Architecture: VRMeer

• Interaction Model: StateStream

• Run-time/Development Environment

-> Unifying through a dynamic language

March 9, 2008 4

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

Architecture: VRMeer

[2] G.de Haan et. al. “Flexible Abstraction Layers for VR application development”

IEEE VR '07

March 9, 2008 5

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

StateStream Model

Low-level design of techniques, relations

• Actors• Conceptual containers• e.g. VR Objects, Interactors, (Algorithms)

• Dual aproach for Interaction Modeling• Statemachine: Discrete behavior• Streammachine: Continuous exchange of data

[6] R.Jacob et. al. “A Software Model and Specification Language for Non-WIMP user interfaces”, ACM TOCHI '99

March 9, 2008 6

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

StateStream Model: StateMachines

• StateCharts: Hierarchical, Concurrent Statemachines

• Functionality in callbacks of Events, Transitions

March 9, 2008 7

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

StateStream Model: StreamMachines

• Data Flow Graph: Nodes, Connections, Ports• Functionality in callbacks of ports

March 9, 2008 8

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

StateStream Model: Relations

March 9, 2008 9

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

Integration with Python

• StateStream components are Python Classes• Methods: expressive functionality• Dynamic Typing: Flexible with restructuring• Inheritance: Derive and override• Introspection: run-time decisions, accessible• Mix-ins: Quick integration with base arch.• Instantiation: Create new components on-the-fly

• Python in control: Intercept and introspect “anywhere, anytime”

March 9, 2008 10

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

Iterated Development• Challenges: containment, hierarchy, timing, order

• Gradual maturing• Model Extension• Transfer to Model• e.g. Templated sub-statemachines

• Complexity explosion unevitable• Supporting analysis tools (e.g. Visualisation,

logging)• Run-Time Environment• Towards end-user environment

March 9, 2008 11

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

Run-Time Environment

• Python glue• ~ concurrent w. VR• GUI components• Introspection• Scripting• Mix event models

(d.tools)

[4] B. Hartmann, “Reflective Pysichal Prototyping through integrated design, test, and analysis”, UIST 2006

March 9, 2008 12

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

Conclusions, Future Directions

• StateStream model and architecture• Work in progress• Visibility, flexibility, expressiveness• Speed up prototyping, development

• Future work• Extensible prototyping environment• (Semi-) Formal model properties• Serialization, Optimization, Distribution

• Multiple Python instances

March 9, 2008 13

G. de Haan, F.H. Post“Flexible Architecture for the Development

of Real-Time Interaction Behavior”SEARIS'08 Workshop

Questions ? Remarks !

g.dehaan@ewi.tudelft.nl - http://visualisation.tudelft.nl/VRVis

!