What is Wrong with Digital Documents? A Conceptual Model for Structural Cross-Media Content...

ER 2010, November 4, 2010

What is Wrong with Digital Documents?

Beat Signer, http://www.beatsigner.com

Web & Information Systems Engineering Lab (WISE)

Department of Computer Science

Vrije Universiteit Brussel

A Conceptual Model for Structural Cross-Media Content Composition and Reuse

http://www.beatsigner.com/

ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]

PARC User Interface (Mid 70s)

Desktop metaphor

WIMP interaction

"filing cabinets" with

hierarchical folders

Paper paradigm

"What You See Is What

You Get" (WYSIWYG)

what about richer

document formats that

existed at that time?

Xerox Star 8010 [http://www.digibarn.com/collections/systems/xerox-8010/]


Digital Documents as a Paper Simulator?

Vannevar Bush

Ted Nelson

Most people don't understand the logic of the concept: "What You See Is What You Get" is based on printing the document out ("get" means "get WHEN YOU PRINT IT OUT"). And that means a metaphysical shift: a document can only consist of what can be printed! [...] No overlays [...] – PAPER UNDER GLASS.

When data of any sort are placed in storage, they are filed alphabetically or numerically, and information is found (when it is) by tracing it down from subclass to subclass. It can be in only one place, unless duplicates are used [...] The human mind does not work that way. It operates by association.

As We May Think, Atlantic Monthly, July 1945

Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009


Paper-based Work Practices

Marcel Proust, À la recherche du temps perdu

Du côté de chez Swann [http://expositions.bnf.fr/brouillons/grand/45.htm]

Marginal

annotations

versioning

Cut and paste

#


oN-Line System (NLS)

Collaborative System

by D. Engelbart et al.

demonstrated in 1968

addressing and linking

parts of files

flexible file views

overlay file structures

cross-file editing

...


010010001001001 0101001100010010100010010 0100100010010001000100001001000 100010001100001001000110010101010110 1100010010100010010010010100010101011001 1010011000011001 010101010100 11000100101000100 110101010101 010000100100 01010101000 01001001001 1111111111 111 0000000010 0 11111111 0101001011 010101010 111010101010101 00101001011010

Fluid Cross-media Information Spaces


Challenges

Representation of cross-media relationships

navigational relationships

structural relationships

- within document structure - external document structure

Content reuse across media boundaries

cross-media transclusion

Distribution of data and metadata across

multiple devices and media formats


Structure Metamodel (RSL)

entity

selector resource

(1,*)(1,*)

(1,1) (0,*)

(0,*) (0,*)

RefersTo

HasTargetHasSource

partition

HasProperties

parameter

Properties(0,*)

(0,*)

HasResolvercontextResolver

Context

Resolvers

Entities(0,*)

(0,*)

user

Users

parameter

Preferences

group

Groups

(0,*)

(0,*)

(0,*)

(0,*)

HasMembers

AccessibleTo

individual

Individuals

(0,*)

(0,*)

(1,1)

partition

HasPreferences

(0,*)

layer

LayersOnLayer |HasLayers|

(1,1)

(0,*) (0,*)

(0,*)

(1,*)

(0,*)

link

Structural

Links

partition

CreatedBy

ResourcesSelectors

|HasChild|

link

Navigational

Links

link

Links

structure

Structures HasElements(0,*) (1,1)

B. Signer and M.C. Norrie, As We May Link: A General Metamodel for Hypermedia Systems. In Proc. of ER 2007, Nov 2007

OM notation

(developed at ETH Zurich)


Associative File System (RBAF)

signer

My Pictures

Vancouver ER 2010

Granville Island

Vancouver Harbour

Public Market

Keynote J. Thorp

Olga and Lamia

Dinner ER 2010

multiple classification

structural link

navigational link

My Videos

Vancouver Aquarium

ER 2010 Keynote

cross-media association

My Presentations

ER 2010 WISE 2009

cross-media

transclusion

single slide

transclusion

(via OOXML

RSL selector)


Associative File System (RBAF) ...


Structure Metamodel Features

Multiple overlay structures can be defined on top

of a set of entities (views)

Structural relationships between parts of

resources (fragments) via selector concept

Structural relationships as first class entities

tagging via properties (key/value pairs)

navigational links to other resources

context-dependent adaptive structures via context

resolvers


Cross-media Information Browser & Editor

How to build a cross-media structure browser & editor?


Conclusions

Existing document formats are very much

based on the simulation of paper affordances

on static desktop computers

Structure metamodel for the representation of

open and fluid cross-media information spaces

associative file system example

cross-media information browser & editor

Remediation of the "paper simulation" approach

WYSIWYG is only one out of many options!


Acknowledgments

Colleagues at ETH Zurich for collaborating on

the RSL link metamodel; in particular

Prof. Dr. Moira C. Norrie

Gregory Cardone for implementing the initial

prototype of the RSL-based associative file

system (RBAF)


References

B. Signer and M.C. Norrie, As We May Link: A General

Metamodel for Hypermedia Systems, Proc. of ER 2007,

Auckland, New Zealand, November 2007, 359–374

V. Bush, As We May Think, Atlantic Monthly 176(1), 1945

T. Nelson, Geeks Bearing Gifts: How the Computer World

Got This Way, Mindful Press, 2009

B. Signer, Fundamental Concepts for Interactive

Paper and Cross-Media Information Spaces,

ISBN 978-3-8370-2713-6, May 2008

What is Wrong with Digital Documents? A Conceptual Model for Structural Cross-Media Content...

Education

Transcript of What is Wrong with Digital Documents? A Conceptual Model for Structural Cross-Media Content...