A Generic Approach for Multi-device User Interface

Rendering with UIML

Kris Luyten, Kristof Thys, Jo Vermeulen and Karin Coninx

{kris.luyten, kristof.thys, jo.vermeulen, karin.coninx}@uhasselt.be

Expertise Centre for Digital MediaAffiliated with the Institute for BroadBand Technology

Hasselt University - transnationale Universiteit LimburgBelgium

A Generic Approach for Multi-device User Interface

Rendering with UIML

A Generic Approach for Multi-device User Interface

Rendering with UIML

What is UIML?

description of auser interface

• high level

• generic

• no specific tags (e.g. <button>)

• allows to build a custom user interface language

What does UIML look like?

Structure

Style

Content

Behavior

Logic

Presentation

InterfacePeers

Vocabulary UIML document

<uiml> <interface> <structure> <part>...</part> </structure> <style> <property>...</property> </style> <behavior> <rule>...</rule> </behavior> </interface> <peers>...</peers></uiml>

A Generic Approach for Multi-device User Interface

Rendering with UIML

Renderer vs Compiler

Compiler

Executable codeUIML

Renderer

• runtime environment

• allows for rapid prototyping

• dynamic changes at runtime

A Generic Approach for Multi-device User Interface

Rendering with UIML

Multiple renderers

• Mono/MS .NET/MS Compact .NET (Uiml.net)

• Java MHP

• Java MIDP

Multiple devices

• Desktop PC

• PDA

• iDTV

• Smartphone

• ...

A Generic Approach for Multi-device User Interface

Rendering with UIML

Common renderer architecture

• requires a VM

• one rendering core

• multiple rendering back-ends

Rendering core

• process UIML

• build internal representation

• mappings

• dynamically at runtime; or

• ad-hoc (when no reflection is available)

Rendering back-ends

• limited responsibility

• process parts that require widget-set specific knowledge

The rendering process

• three stages

• pre-processing

• main processing

• post-processing

Pre-processing

• transform one UIML document into another

• examples

• adaptation by user profile

• applying layout constraints

Main processing

• interpret the UIML file

• build a concrete instantiation using the available widget set

Main processing details

1. look up the required back-end

2. build an internal representation

3. for every part, attach its style and behavior

4. render the generated tree to a concrete instantiation

<structure> <part id=”all” class=”Container”> <part id=”rubens” class=”Button”/> </part></structure>

<style> <property part-name=”rubens” name=”label”> Rubens Barichello </property></style>

rubens

all

label

Post-processing

• manipulate the concrete instantiation

• examples

• bind the UI with the application logic

• adaptable navigation

The demo

Overview

• industry case

• participate in TV show by using different devices

• functional interactive quiz

• with scorekeeping and an evaluation

• adapted to the user’s profile

• on different devices

UIML allows us to reuse the quiz logic

Shared logic

PdaQuiz.Browser

mcdp.CanvasClient

Xlet.UimlXlet

<logic> <d-component id=”Quiz“ ...> <d-method id=”Answered“ ...> <d-param ... /> </d-method> <d-method id=”GotoPage“ ...> <d-param ... /> </d-method> <d-method id=”Exit“ ... /> <d-method id=”GetPoints“ ... /> <d-method id=”Question1Correct“ ... /> <d-method id=”Question1Incorrect“ ... /> <d-method id=”Question2Correct“ ... /> <d-method id=”Question2Incorrect“ ... /> <d-method id=”Question3Correct“ ... /> <d-method id=”Question3Incorrect“ ... /> </d-component></logic>

Smartphone

iDTV

PDA

DesktopDesktopQuiz.Browser

Completely independent of the user interface

GTK# desktop demo

SWF desktop demo

Can we reuse the entire interface

description?

Almost

• interactors

• interaction

• application logic

• content

• layout

Layout constraints

• widget set independent layout specification

• plastic

• extension to UIML

• handled in the pre-processing step

(0,0) (50,0)

“rubens” above “giancarlo”

“rubens” left-aligned-with “giancarlo”

(0,0)

(0,10)

<style> ... <property part-name=”rubens” name=”position”>0,0</property> <property part-name=”giancarlo” name=”position”>0,10</property> ...</style>

Add position properties

MHP iDTV demo

PDA demo

Smartphone demo

User adaptation

• Intelligent User Interface Agent

• knows about user’s preferences

• transforms the interface accordingly

• currently MPEG-21 based

• handled in the pre-processing step

Use case

• visually impaired users

• user agent adapts the UIML description

• larger, bolder fonts

Before

After

Conclusions

• functional client-side UIML renderer on different devices

• .NET version is freely available

• http://research.edm.uhasselt.be/uiml.net

Questions?