UNIFORM: Automatically Generating Consistent Remote Control User Interfaces Jeffrey Nichols, Brad A....

UNIFORM: Automatically Generating Consistent Remote Control User Interfaces

Jeffrey Nichols, Brad A. Myers, Brandon RothrockHuman-Computer Interaction InstituteSchool of Computer ScienceCarnegie Mellon University

Conference on Human Factors in Computing Systems (CHI 2006)April 25, 2006

2

Motivation

3

Motivation, cont.

Appliance interfaces could be more usable if their interfaces were consistent

Almost all appliances have some unique features

Question: How might additional functionality be presented to the user while still providing a consistent interface?

4

UNIFORM

Generates personally consistent user interfaces

Personal Mobile Devices

Personal Universal Controller (PUC) system

Appliances

Specifications

Control

State Feedback

5

UNIFORM

Copier A Copier B

Consistency disabled – Copier A used first

6

Consistency disabled – Copier A used first

UNIFORM

Copier A Copier B

Consistency enabled – Copier A used first

Consistent Copier B

7

Overview

• Motivation

• Previous Work on Consistency

• Specification Authoring Studies

• Requirements for Consistency

• Architecture

• Knowledge Base and Finding Similarity

• Generating Consistent Interfaces

• Discussion and Future Work

8

Related Work

Theory of Consistency• 1988 workshop at CHI was unable to define

consistency [Nielsen 88, Grudin 89]

• Loosely defined as “doing similar things in similar ways” [Reisner 90]

• Different dimensions of consistency [Kellogg 87]

• Work on consistency as knowledge transfer (e.g. [Keiras & Polson 85, Polson 88])

9

Related Work, cont.

Systems addressing Consistency

• Some tools can automatically test for consistencyGLEAN uses a GOMS-based approach [Kieras 95]

Sherlock generates metrics for interpretation by designer [Mahajan 97]

• Previous automatic generation systems have ensured consistency within an application or family of applications

ITS [Wiecha 90]

PUC [Nichols 02]

SUPPLE [Gajos 05]

10

Specification Authoring Study

Study of Authoring Appliance Specifications

• How can specifications vary for different appliances with similar functions?

• How can specifications vary for different authors?

Two User Studies• Study #1: An expert user (me!) wrote

specs forthree different VCRs

• Study #2: Three subjects wrote specifications

for the same VCR

All subjects instructed to: • Include all functions• Be faithful to the existing appliance design

Low End: Panasonic VCR

Mid-Range: Samsung DVD-VCR

High-End: Mitsubishi DVCR

11

Results of Both Studies

Functions• Some were identical• Others were similar but had different labels• Others had very different specifications

Structure• Structure for the first study was very similar across

appliances• Top-level structure was the same for the second study, but

many differences were found at lower levels

Results shows that specifications can differ substantially

Analysis of results showed important distinction between functional and structural consistency

12

Consistency in UNIFORM

We define a consistent interface to be one that incorporates elements and organization that are familiar to the user from previous interfaces

We have refined this definition into six requirements, based on previous research and our authoring study

13

Requirements for Consistency

1. Interfaces should manipulate functions in the same way

2. Interfaces should locate similar functions in the same place

3. Interfaces should use similar labels for similar functions

4. Interfaces should maintain a similar visual appearance

14






5. Usability of unique functions is more important than consistency

15






5. Usability of unique functions is more important than consistency

6. Users must be able to choose to which appliance consistency is ensured

16

Overview

• Motivation




• Architecture




17

UNIFORM Architecture

Knowledge Base

UNIFORM Interface Generation

PUC ApplianceSpecification

Personally Consistent User Interface

18


Knowledge Base

Previous Specifications

Functional Mappings Between Specifications

Previously Generated UIs

Mapping Phase

1

Abstract Consistency

Phase

3

Concrete UIBuilding Phase

4

Output:Consistent User Interface

Concrete Consistency Phase

5

2 Abstract UI Building Phase

Input:Appliance Specification

<?xml?>

<spec>

<stuff/>

</spec>


19


Mapping Phase

1


Phase

3


4



5



<?xml?>

<spec>

<stuff/>

</spec>


Knowledge Base




20


Knowledge Base




UNIFORM Interface GenerationMapping

Phase1


Phase

3


4



5



<?xml?>

<spec>

<stuff/>

</spec>

21


Knowledge Base




Mapping Phase

1


Phase

3



5


<?xml?>

<spec>

<stuff/>

</spec>



4


22


Knowledge Base





4




<?xml?>

<spec>

<stuff/>

</spec>

UNIFORM Interface GenerationMapping

Phase1


Phase

3


5

23

Knowledge Base

Functional Mappings• We support six types of mappings,

based on the results of the specification authoring studies

general, state, node, group, list, and template

• Specifies that a function in one specification is similar to a function in another

Mappings are grouped within the knowledge base in mapping graphs

Knowledge Base




24

Mapping GraphsGroup semantically similar functions across appliances

One graph for each function

• Power• Volume• Media Controls• Etc.

Node counts • Store generation history• Used to find basis for

consistency

Edges• Track whether consistency

can be ensured

Choice in consistency supported by manipulating the node counts

Knowledge Base




3

1

DVD Player

State: Mode { Play, Stop, Pause }

Command: Next TrackCommand: Previous Track

Answering Machine

State: Mode { Play, Stop, Pause }

Command: Play New Messages

Cheap VCR

Command: PlayCommand: StopCommand: PauseCommand: RewindCommand: F-FwdCommand: Record

Mitsubishi VCR

State: Mode { Play, Stop, Pause, Rewind, F-Fwd, Record }

0

0

25

Where do mappings come from?Three ways mappings may be found

• Automatically extracted during mapping phase

• Challenging because of limited semantic information in specifications

• We have implemented two mapping algorithms, one based on work in automatic schema mapping from the database community

• Current best algorithm finds about 60% of mappings

• We are also exploring other techniques

• Specified by the user• Manually entered as text• Interaction techniques?

• Downloaded from the Internet• Manufacturers?• Internet communities? (e.g.

www.remotecentral.com)


Mapping Phase

1

Abstract Consistency Phase

3


4

<?xml?>

<spec>

<stuff/>

</spec>



5


26

Abstract UI Building Phase

Rules from previous PUC system

Transforms specification into abstract user interface

Abstract user interface includes:• Tree organization• Control choice


Mapping Phase

1


3


4

<?xml?>

<spec>

<stuff/>

</spec>



5


27


Most of the work to ensure consistency happens in this phase

• Operates on the abstract user interface

• We have found that most consistency can be ensured at the abstract level

• Also enables many of our rules to cross platforms with little or no additional work

This phase is made up of two parts:• Functional sub-phase

Similar functions are assigned similar labelsSimilar functions are converted, if necessary, to a consistent representation

• Abstract Structural sub-phaseSimilar functions are placed in similar groups


Mapping Phase

1


3


4

<?xml?>

<spec>

<stuff/>

</spec>



5


28

Abstract Structural Sub-Phase

This sub-phase has two parts:• Moving

Similar functions are moved to previous groups

• Re-orderingWithin each group, functions are re-ordered to match previous order

29

Moving

Base

Status Setup

Clock

Channel

Base

Samsung DVD-VCR

Base

Mitsubishi DVCR

30

Moving

Base

Status Setup

Clock

Channel

Base

Setup

Samsung DVD-VCR

Base

Setup

Mitsubishi DVCR

31

Moving

Base

Status Setup

Clock

Channel

Base

Setup

Clock

Samsung DVD-VCR

Base

Status

Clock

Mitsubishi DVCR

32

Moving

Base

Status Setup

Base

Setup

Clock

Samsung DVD-VCR

Base

Status

Clock

Mitsubishi DVCR

Clock

Channel

Status

33

Moving

Base

Status Setup

Clock

Channel

Base

Status

Clock

Channel

Samsung DVD-VCR

Base

Setup

Channel

Mitsubishi DVCR

Channel

Setup

Unique functions are moved with their group but never by themselves

34

Re-Ordering

35

Re-Ordering

When

TimeDate

Channel ConflictSpeedSourceType

Start Stop

Length

VCR+ DayConflict Channel Speed

Timed Recordings for the Samsung DVD-VCR

Timed Recordings for the Mitsubishi DVCR

36

Re-Ordering

Conflict VCR+ Channel When Speed

When Type Channel Speed Conflict



When Type Channel Speed Conflict

37

Re-Ordering

Start StopVCR+ DayConflict Channel Speed



TimeDate

Conflict ChannelSource When Length Type Speed

38

Concrete UI Building Phase

Rules from previous PUC system

Transforms abstract user interface into concrete user interface

Concrete user interface includes:• Organization necessary for screen

size• Panel Structure• Instantiations of UI objects• Positions and sizes of controls


Mapping Phase

1


3


4

<?xml?>

<spec>

<stuff/>

</spec>



5


39


Differences in the available labels or number of controls may have created inconsistencies

• For example, a rule may have not added organization in this interface that was in the previous interface

• The side-panel orientation in a previous interface might have been chosen to allow a long label to fit

Rules in this phase address these problems provided that the usability of unique functions is not affected

• For example, side-panel orientation might not change if needed for a large label or control


Mapping Phase

1


3


4

<?xml?>

<spec>

<stuff/>

</spec>



5


40

Overview

• Motivation




• Architecture




41

Discussion and Future Work

Evaluation of Generated Interfaces (in progress)

• Users will perform ~10 tasks with two all-in-one copier interfaces (HP or Canon)

• Three conditionsActual Appliance vs. PUC vs. Uniform

42

Discussion and Future Work

How else might we deal with unique functions?• Better semantic information• More detailed modeling?

Better knowledge of consistency• Elaborate and improve on requirements for consistency• Dimensions of consistency and relative importance• More operational knowledge

More consistency rules• Adding new organization• Support for “consistency ghosts” and other forms of

“transformational consistency” [Richter 2006]

43

AcknowledgementsCo-Authors

• Brad A. Myers• Brandon Rothrock

Thesis Committee• Scott Hudson• John Zimmerman• Dan Olsen Jr.

Funding• National Science

Foundation• Microsoft• General Motors• Intel• Pittsburgh Digital

Greenhouse

Equipment Grants• Mitsubishi (MERL)

PUC Project Members• Duen Horng Chau• Kevin Litwack• Thomas K. Harris• Michael Higgins• Joseph Hughes • Thomas Psik• Roni Rosenfeld• Rajesh Seenichamy• Pegeen Shen• Htet Htet Aung• Mathilde Pignol• Suporn Pongnumkul• Jeffrey Stylos• Stefanie Tomko• Peter Lucas

Thanks for listening!

For more information…http://www.cs.cmu.edu/~jeffreyn/uniform/http://www.pebbles.hcii.cmu.edu/puc/

UNIFORM: Automatically Generating Consistent Remote Control User Interfaces

Jeffrey Nichols, Brad A. Myers, Brandon RothrockHuman-Computer Interaction InstituteSchool of Computer ScienceCarnegie Mellon University

Conference on Human Factors in Computing Systems (CHI 2006)April 25, 2006

46

Questions

• Have you thought about using an approach where, given that you have 4-5 appliances, you generate all of the possible consistent interfaces and then find the right one (either by the user or through some kind of optimization or metric) – Allen Cypher

• What happens when I start with a simple interface and then get more complex over time? Does the consistent structure cause everything to become hidden? – Margaret Burnett

• Can you handle the case where on one appliance the function is a button and on another the function is multiple steps? – Henry Lieberman

47

Generated Phone Examples

Copier A Copier B

Consistency

disabled

48


Copier A Copier B

Consistency

enabled-

Copier A used first

49


Copier B Copier A

Consistency

enabled-

Copier B used first

50

Functional Sub-Phase

Inspects each function in the new specification

• Determines whether there is a previous function with which to ensure consistency (using mapping graph)

• Rules make changes to the abstract interface to ensure consistency

• Eight rules covering a number of potential functional differences

• Always ensure that previous labels are used

• May change the controls

Two sorting functions from different copier interfaces

51

Copier A Copier B

UNIFORMConsistency disabled – Copier B used firstConsistency disabled – Copier A used first

52

Copier ACopier B

UNIFORMConsistency disabled – Copier B used first

Consistent Copier A

Consistency enabled – Copier B used first

53

Motivation, cont.

54


Mitsubishi DVCR without consistency

Samsung DVD-VCR without consistency

Samsung DVD-VCR consistent with Mitsubishi DVCR

UNIFORM: Automatically Generating Consistent Remote Control User Interfaces Jeffrey Nichols, Brad A....

Documents

Transcript of UNIFORM: Automatically Generating Consistent Remote Control User Interfaces Jeffrey Nichols, Brad A....