novel visualization and interaction for large displays

mary czerwinski

microsoft research

vibe team• Patrick

Baudisch• Mary

Czerwinski• Brian

Meyers• George

Robertson• Greg Smith• Daniel

Robbins• Desney

Tan• Nuria Oliver

large display surfaces are here

Workstation in the real world

home setup

Generic snapshot in real person’s bedroom

overview

• initial large display research–prototypes around usability issues

observed

• visualization and interaction –new user experiences have to scale

the wide continuum of displays

• future directions

harris poll responses (7/02, N=1197)

Mutiple PCs and Displays

0%

10%

20%

30%

40%

50%

60%

70%

80%

None Multiple monitorsattached to

multiplecomputers.

Laptop anddesktop monitor

connectedtogether.

Dualmon or higher

Config

Pe

rce

nt

Re

sp

on

da

nts

All

why a larger display surface?

• productivity benefits 10-30% (despite sw usability issues)

• users prefer more display surface

• prices dropping fast

• footprints getting smaller

Projected LCD Pricing 2002-2005

$437$378 $327 $283

$699$597

$510$436

$1,089

$905

$752$625

$0

$200

$400

$600

$800

$1,000

$1,200

2002 2003 2004 2005

$U

S

15" -13.5%

17" -14.6%

18" -16.9%

1st prototype--dSharp display

• triple projection

• matrox parhelia card

• 3028 x764resolution

• 42 in. across• slightly curved• 120 degree

FOV

task times – significant

Effects of Display Size on Task Times

0

20

40

60

80

100

120

140

160

DISPLAY

Ave

rage

Task

Tim

e (S

econ

ds)

Small

Large

user satisfaction - significantthe tasks were easy to perform

0

1

2

3

4

5

Small Large

Display Size

Ave

rage

Rating

(1=D

isag

ree,

5=Agr

ee)

windows layout - significantI was satisfied with the ease of windows

layout

012345

Display Size

Ave

rage

Rat

ing

(1=D

isag

ree,

5=Agr

ee)

Small

Large

but…usability issues• why click to bring a

clearly visible window into focus? caused many errors

• where is my cursor?• where is my start

button?• where is my taskbar?• where are my dialogs?• the software doesn’t

know where the bezel is…

input: high density cursor

fill-in cursors

current framefill-in cursors

previous frame

mouse

motion

mouse

motion

solution• high-density cursor inserts additional

cursor images between actual cursor positions

• the mouse cursor appear more continuous

problem• at high mouse speeds, the mouse cursor

seems to jump from one position to the next

the windows mouse trail…• makes mouse trail last longer• drawback: cursor images lag behind

...is not high-density cursor• hd cursor makes mouse trail denser• lag-free: mouse stops=>cursor stops

regular mouse cursor

high-density cursor

Windows mouse trail

high-density cursor

input: drag-and-popproblem• large displays create long distance

mouse movement• touch & pen input has problems moving

between screen units

solution• drag-and-pop brings proxies of targets

to the user from across display surfaces• the user can complete drag interactions

locally—no need to deal with distances or to cross display borders

vibelog: 2 research paths

• 1st activity repository for studying windows usage in aggregate– can profile users based on

display size– can be extended to visualize

workflow and capture context

• single user: capture task contexts to surface pertinent ui or provide reminders

windows and task management issues emerge

• larger displays = more open windows

• multimon users arrange windows spatially

• taskbar does not scale:– aggregation model not

task-based– users can’t operate on

groups of related windows

Relationship between # of Monitors and # of Windows Left Open

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

16.00

18.00

No. of Monitors

Avg

. #

of

Win

do

ws

Lef

t O

pen

Single Monitor

DualMon

TripleMon

changes in window access patterns

0

10

20

30

40

50

60

70

80

90

100

1 2 3

Number of Monitors

Per

cen

tag

e o

f A

cces

s T

ech

niq

ue

Win

Taskbar

multitasking support

•vibelog

•projectbar, layoutbar, groupbar

•scalable fabric

task management: groupbar

• taskbar for lightweight grouping of windows

• allows for multiple bars, spatial placement of bars

• ~400 internal downloads

• desktop snapshotting; task snapshots

• licensing to external companies

task management: scalable fabric

• configurable central focus + peripheral context• easy task switch from periphery to focus area• leverages human spatial memory• over 600 downloads, licensing externally

table cloth

• problem:– user wants to

access content physically far away

• solution:– pan the desktop

to user– compress

content to the right of focus

– grab content you need and snap back

meeting support: wincuts

• microsoft supports:– individual user productivity (eg. word, excel, …)– disseminating information (eg. powerpoint, sharepoint, …)– communication (eg. outlook, MSN messenger, …)– distance collaboration (eg. Netmeeting, Livemeeting, …)

• but what about co-located collaborative work?– people bring different expertise and information on personal

devices to meetings and need to share• today’s model is broken—only one person gets to display entire

desktop at a time (for better or worse)• or, must share applications and files with others

– what about private information?

• what about ui that should be scaled for the task at hand?

scalable ui research

• datelens with Ben Bederson

• facetmap—visualizing your digital memories

datelens with Ben Bederson• fisheye

representation of dates

• compact overviews • user control over

the view• integrated search

(keyword)  • enables overviews,

fluid navigation to discover patterns and outliers

• integrated with outlook

facetmap

• a way to visualize all of your digital memories– soon, everything will be recorded

• heart rate, blood oxygen levels, conversations, where you go, who you talk to, what you touch electronically, etc.

– need ways to browse all these streams of media

• most studying text lists of search results• can we do something purely visually that scales?

future vibe directions

• novel interaction and visualization techniques that scale from small to very large displays

• continued evaluation and iteration of designs from a user-centered perspective

• more focus on collaboration and group awareness with large displays

• more information: http://research.microsoft.com/research/vibe

thank you!