Constraints On TreesAnd A Forest Of Other Problems

Ian SmithGeorgia Tech

May 21, 1998

Story

Talk Terrain Map

Introduction Approach

State Preservation

Flexible UIs Media Spaces

“Mumbling”

Relationships

JOB INTERVIEW

Text Width

Inner Width

Inner X

High-Order Bit

C B

D DD...

A

F

B’

E

D D C’...

This is not a constraint.

• This is automatic relationship maintenance.

• Not numerical constraints.

• Not logic programming.

• Propagation-based systems – Not solved “from scratch”

Related Work

• Sutherland’s Sketchpad• Borning’s DeltaBlue, SkyBlue, Indigo• Vander Zanden’s QuickPlan + studies• Hudson’s Eval/Vite• Myers and Guise’s KR solver• Patterson & Hill : AVL, etc.

• Smalltalk’s active values

UI Applications

• User interfaces are trees

• Roughly: one tree is one “window”

Scenario: Org Hacker

Steve McManAccounting

A A& B Software Org Chart

Alice JonesPresident, CEO

Bob SmithCTO

Chris HopkinsCFO

Tom KattDirector Of Eng.

Ian SmithDistinguished Eng.

Kathy WahlDirector Of Finance

Bad Design

Steve McManAccounting

A& B Software Org Chart

Alice JonesPresident, CEO

Bob SmithCTO

Chris HopkinsCFO

Tom KattDirector Of Eng.

Ian SmithDistinguished Eng.

Kathy WahlDirector Of Finance

Pilot DisplayArea

Improved Design

Org Chart Reports To

CEO, President (A. Jones)

CTO (R. Smith)CFO (C. Tompkins)

The Set-Up

• Two (or more) different “views”

• Each view is created/maintained by a tree

• Views are different, but related– Interactors are not one to one– Overall shapes are different

• Views change over time…

• Declare constraints!

Reports ToOrg Chart

President (John)

VP of Eng. (Jane)

Principal Sci. (Mary)

Staff Eng. (Bill)

VP of Finance (Fred)

View2View 1

Approach

Scroller

Tree Control

Tree Node

Tree Node Tree Node

Pilot App

Palette Scroller

Line RectRect...

Window

Ultraman

Ultraman Design-Time

• Tool for expressing relationships

• Designer uses a pattern language

• Language converted to a grammar

• Grammar (and support code) compiled and linked with your app

Ultraman Run-Time

• After each event…

• Source tree is flattened

• Flattened form is “parsed”

• New result tree “generated” by matching patterns

• Screen updated

“The Magic Finger”

State-Preservation

• “Generating” interface B from A implies no state local to A

• Each time B changes A’s state gets “trampled”

• E.g. State of the “open-close” objects on Pilot interface

Incrementality

• Given a small change in the source tree, make the correct, small change to the target tree

A

B C D

C

C

A

E

Approach

• “Generate” a new target tree each time…

• Identify the nodes that are the same

• Preserve those nodes that haven’t changed

A

B C D

C

C

A

E

C

C

A

E

+ =

C

C

A

E

Sameness?

• What does sameness mean?– Same location in the target tree?– Same type?– Result from the same constraint?– Result from the same object in the source tree?– Because they say so?

Solution

• Precompute where sensible

• Three conditions on sameness:– Produced from the same constraint– Produced from the same portion of the source– Compatible types

• If two nodes are the same, they are merged to preserve state

Value Numbering

• Bottom-up change metric run on the parse tree– Leave open possibility of VN on result tree

• Technique derived from common subexpression elimination

Fun Things To Do

• Good library of constraints

• Grammar analysis

• Toolkit for audio display

• User-controlled error handling

• Lazy toolkits

Flexible UIs

• SubArctic -- Java UI toolkit

• UI Toolkits are not a collection of widgets!

• Building new interactions should be easy!

• Built-in support for lenses (UIST ‘97)

• Downloadable, dynamic styles (UIST ‘97)

• Animation as a first class entity (UIST ‘93)

Style

SubArctic and Me

• ~60K lines of code

• Responsible for design and impl. of major API sections

• Bug fixing

• Documentation

• Release Engineering

• Student management

Media Spaces

• Media spaces : audio/video/textual environments for awareness

• Privacy tradeoff – Cryptography helps...

• Disturbance tradeoff

Mumbling

• “Mumbling” is a privacy tradeoff

• Remove large amounts of information content

• Leave some interesting, but not sensitive, bits in the stream

• Audio: 1 integer per 500ms with an amplitude approximation

Mumbling Video

Thanks

• Special thanks to my hosts:Mike SalisburySteve Cousins

• Thanks to Alex Zhao & Keith for giving me some screen shots.

• Thank you for coming!

• Thanks to Ultraman...