A UNIDRAW-BASED USER INTERFACE BUILDER

John M. VlissidesSteven Tang

Technical Report: CSL-TR-91-485

August 1991

Research supported by a grant from Fujitsu America, Inc., by the SRC under Contract go-MC-1 06,by the U.S.Navy/DARPA under Contract N00014-87-K-0729, by the NASA CASIS project underContract NAGW 419, by the Quantum project through a gift from Digital Equipment Corporation,and by a grant from the Charles Lee Powell Foundation.

A Unidraw-Based User Interface Builder

John M. Vlissides and Steven Tang

Technical Report: CSL-TR-91-485

August 1991

Computer Systems LaboratoryDepartments of Electrical Engineering and Computer Science

Stanford UniversityStanford, CA 94305

Abstract

Ibuild is a user interface builder that lets a user manipulate simulations of toolkit objects ratherthan actual toolkit objects. Ibuild is built with Unidraw, a framework for building graphicaleditors that is part of the Interviews toolkit. Unidraw makes the simulation-based approachattractive. Simulating toolkit objects in Unidraw makes it easier to support editing facilities thatare common in other kinds of graphical editors, and it keeps the builder insulated from a particulartoolkit implementation. Ibuild supports direct manipulation analogs of InterViews’ compositionmechanisms, which simplify the specification of an interface’s layout and resize semantics. Ibuildalso leverages the C++ inheritance mechanism to decouple builder-generated code from the restof the application. And while current user interface builders stop at the widget level, ibuildincorporates Unidraw abstractions to simplify the implementation of graphical editors.

Keywords: user interface builders, user interface toolkits, direct manipulation, graphical con-straints

Copyright @ 1991

bY

John M. Vlissides and Steven Tang

Unidraw-Based User Interface Builder

John M. Vlissides and Steven TangStanford University

Abstract

Ibuild is a user interface builder that lets a user ma-nipulate simulations of toolkit objects rather than ac-tual toolkit objects. Ibuild is built with Unidraw, aframework for building graphical editors that is part ofthe Interviews toolkit. Unidraw makes the simulation-based approach attractive. Simulating toolkit objectsin Unidraw makes it easier to support editing facili-ties that are common in other kinds of graphical ed-itors, and it keeps the builder insulated from a par-ticular toolkit implementation. Ibuild supports di-rect manipulation analogs of Interviews’ compositionmechanisms, which simplify the specification of aninterface’s layout and resize semantics. Ibuild alsoleverages the C++ inheritance mechanism to decouplebuilder-generated code from the rest of the application.And while current user interface builders stop at thewidget level, ibuild incorporates Unidraw abstractionsto simplify the implementation of graphical editors.

Keywords: user interface builders, user interfacetoolkits, direct manipulation, graphical constraints

1 Introduction

Few user interface builders today are written fromscratch. Like other interactive applications, most ben-efit from a toolkit-based implementation. But in addi-tion to using a toolkit in the normal way, most buildersalso use instances of toolkit classes as data objects: auser manipulates these instances directly in the builderto construct a user interface. Such instance-basedbuilders employ mechanisms that keep the toolkit ob-jects passive as the designer assembles the interface.This approach has certain advantages: the builder’sobjects look and feel exactly like the toolkit objectsdo, since they are one in the same, and testing the fi-nal interface involves simply disabling the mechanism

This research has been supported by a grant from Fujitsu Amer-ica, Inc., by the SRC under contract 90-MC-106, by the U.S.Navy/DARPA under contract N00014-87-K-0729, by the NASACASIS project under Contract NAGW 419, by the Quantum projectthrough a gift from Digital Equipment Corporation, and by a grantfrom the Charles Lee Powell foundation.

To appear in the Proceedings of the ACM SIGGRAPH/SIGCHIUser Interface Software and Technologies ‘91 Conference, HiltonHead, South Carolina, November 1991.

that keeps the objects passive during the buihling pro-cess.

But the instance-based approach has several dis-advantages as well. To implement passivation, thebuilder must subvert normal toolkit semantics, whichusually requires knowledge of toolkit internals. Thebuilder and toolkit implementations are thus closelycoupled, making it harder to retarget the builder to anew toolkit. Moreover, an instance-based approachcomplicates the implementation of features that aretaken for granted in graphical editors for other do-mains, features such as scrolling and zooming the in-terface, eliding parts of it, and supporting multipleviews.

Ibuild is a user interface builder for the Interviewstoolkit [6] that lets a user manipulate simulations oftoolkit objects. Ibuild’s implementation of these ob-jects is independent of the toolkit’s implementation.Builder developers have avoided simulation-based ap-proaches because they offer no implementation ad-vantage over an instance-based approach when thebuilder is developed on top of a traditional user in-terface toolkit. Ibuild, however, takes advantage ofUnidraw [ 131, a framework for building direct manipu-lation graphical editors. Unidraw provides abstractionsabove the toolkit level that simplify the construction ofsuch applications. Unidraw makes a simulation-basedapproach attractive, and it has enabled us to explorethe advantages of simulation as an alternative to theinstance-based approach. Moreover, Unidraw providesthe basis for builder abstractions beyond the widgetlevel.

In addition to standard toolkit objects such as scrollbars, buttons, and menus (generally known as wid-gets or, in Interviews terminology, interactors), ibuildprovides direct-manipulation analogs of Interviewscomposition mechanisms for specifying spatial rela-tionships between interactors. Simulation makes itpossible to let the user modify part of the result-ing hierarchical interactor structure without disturb-ing unrelated parts and without compromising thebuilder’s what-you-see-is-what-you-get (WYSIWYG)model. The user can also establish relationships be-tween interactors by direct manipulation and can in-corporate Unidraw abstractions into the interface tosupport graphical editing applications. Ibuild gener-

- 2

ates Interviews code that implements the interface, et al’s FormsVBT dialog builder [l] employs a TEX-and it uses the C++ inheritance mechanism to insu- based boxes-and-glue model for widget layout simi-late ibuild-generated code from application code and lar to Interviews’ (and ibuild’s). While FormsVBT’sto pen-nit extension of built-in interactors. direct-manipulation interface is not WYSIWYG, the

This paper describes ibuild’s design and implemen- system can also display a non-editable WYSIWYGtation on top of Unidraw. view.

2 Background

User interface builders promote visual user interfacedesign and development to minimize the need for con-ventional programming. Current commercial and re-search builders support interface specification on sev-eral levels:

1. Most support the interactive layout of a user in-terface, that is, absolute positioning of widgets bydirect manipulation.

2. A few offer mechanisms for expressing spatialrelationships between widgets, which define theresize semantics of an interface.

3. Some support hierarchical structuring of widgetsto aid in the specification of complex layouts.

4. Some research builders offer demonstration-basedspecification. At least one research builder, Lap-idary [7], supports demonstrational widget speci-fication.

The more elements a user interface has, the moreimportant it is for the builder to support hierarchy.The reason comparatively few builders support hierar-chy stems from their WYSIWYG nature: the widgethierarchy has no visible manifestation in the interfacefrom the user’s perspective. Thus hierarchical builderseither depart from a truly WYSIWYG model or theyintroduce additional, hierarchical views of the inter-face, or both. For example, child widget compositionsin OPUS are elided as gray boxes. The user can ex-pand a box into a window that contains the child’s con-tents with its own children elided. Navigating througha deeply nested hierarchy can be cumbersome, how-ever, since it is not possible to jump more than onelevel at a time. TeleSoft’s TeleUSE [lo] provides atree view of the widget hierarchy and allows limitedediting to take place there. FormsVBT provides a tex-tual view for viewing and specifying the interface inan s-expression-based language. Unfortunately, it isoften difficult for a user to map between widgets in thegraphical view and nodes in a tree or s-expressions ina program.

A common way to support specification of resize se-mantics is through a graphical notation. In Cardelli’sDialogEditor [3], one of the earliest builders, the userdraws attachment points that keep an edge of a wid-get attached to an arbitrary point in the interface. Thecomponent will stretch or shrink if necessary to main-tain the attachment. OPUS [4] extends this model toinclude a variety of notations for aligning widgets andspacing them proportionately. The main drawback ofthese notations, apart from their often-cryptic appear-ance, is that their presence interferes with the interfacebeing built, thereby mitigating the benefits of a WYSI-WYG editor. NeXT’s Interface Builder [14] takesa simpler approach. The user can introduce integralnumbers of springs around widgets that define howreadily the widget deforms when pressed or pulled bythe window it occupies. The springs do not appear inthe interface but are specified instead through dialogboxes. This scheme is simple and non-intrusive butcannot express many common layouts, such as mutu-ally centered or tiled widget arrangements. Avrahami,

Another approach to visual interface specification isthrough demonstrational techniques. These have beenapplied most successfully to solving the layout prob-lem, as in OSU [5], Lapidary, and Druid [8], thougheach of these systems iets the user specify certain in-terface semantics by demonstration. The known prob-lems with demonstrational specification in general ap-ply equally well to its use in interface builders:

Deducing non-trivial interface semantics from alimited number of user actions is difficult.

The inferred behavior is not always predictable.

There is usually no way to edit a demonstration.

The system understands a limited number ofdemonstrations and must have a fallback speci-fication mechanism when these are insufficient toexpress desired behavior.

It is difficult to extend the system to understandnew demonstrations.

There are other issues that current builders fail toaddress adequately., No current builder eliminates the

need for conventional programming in application de-velopment. Therefore a programmer must at somepoint integrate builder-generated code with the restof the application. To carry out the integration, theprogrammer usually needs to access or even modifyobjects in the generated code. Once generated codeis modified, however, the programmer can no longeredit the interface in the builder without merging hischanges into code generated subsequently. Code con-sistency thus becomes a problem. A related problemfor builders is providing a mechanism for extendingthe functionahty of the builder’s built-in widgets. Fi-nally, no current builder has proven practical for speci-fying interfaces beyond the widget level. For example,assembling a non-trivial, production-quality graphicalediting application is beyond the capabilities of currentsystems.

3 Composing an Interfacewith Ibuild

We will introduce ibuild’s interface and capabilities bycomposing a simple text editing interface.

3.1 Basic Composition

Figure 1 shows the interface being built with ibuild.Three objects are instantiated initially (from left toright):

l A TextEditor instance provides a multiline textediting interface but defines no input handling.

l A VBorder defines a vertical border.

l A V&roller supplies an interface for scrolling inthe vertical dimension.

Each object represents an instance of an Interactor sub-class. The user can instantiate any of the interactorsthat Interviews predefines by clicking in the viewingarea with the appropriate creation tool engaged. Thecreation tools installed currently appear along the bot-tom of ibuild’s interface. The manipulation semanticsfor instantiating an object vary with the interactor. Forexample, the TextEditor instance is created by sweep-ing out a rectangular area of the desired size, whilethe VBorder is specified by dragging a vertical line.

All inter-actors have a shape. The shape defines anatural size (width and height) for the interactor aswell as the amount by which the interactor is preparedto stretch or shrink from the natural size. For example,the natural size of both the VBorder and VScroller is

-3-

zero in the vertical dimension, and both can stretch “in-finitely” in this dimension. The TextEditor is infinitelystretchable and shrinkable as well, but its natural sizeis defined implicitly when it is created by direct ma-nipulation.

Each object in the interface is thus far uncomposedand independent. Now we will compose the interactorsin an HBox so that they tile horizontally. In general,composing interactors in ibuild involves selecting theinstances to be composed and then choosing a com-position mechanism from the Composition menu, forexample, “HBox.” Once composed, the interactors areno longer independent; in this case the HBox instancebecomes the only selectable component in this viewof the interface (see Figure 2). Note that the borderand scroller have stretched vertically to accommodatethe TextEditor’s natural height. This follows from theHBox’s shape semantics, which specify that the natu-ral height of an HBox is the maximum of the naturalheights of its children.

We can resize the interface as it now stands withibuild’s Resize tool (Figure 3), which will simulate awindow manager’s r&zing operation. Doing so cangive a designer the feedback needed to ensure a pleas-ing layout in the face of resizes. Note that the re-size semantics are a natural outgrowth of the compo-sition mechanisms-no graphical notation apart fromthe interface itself is required to specify the layoutconstraints, and the builder’s rendition of the interfaceremains WYSIWYG. To ibuild’s user, the objects thatdefine layout and resizing semantics are an integralpart of the interface. The end-user, on the other hand,has no knowledge of their presence.

The HBox object can now participate in a largercomposition that includes pull-down menus. Figure 4shows the text editing interface once it is fully com-posed, and Figure 5 depicts its interactor hierarchyschematically. The top-level interactor in the interfaceis a ShadowFrame, a composition that draws a dropshadow around its (one and only) component. TheShadowFrame’s component is a VBox, which tiles itscomponents vertically. Inside the VBox is the HBoxcomposed earlier, a horizontal border (HBorder), andanother HE3ox containing a MenuBar object and apiece of horizontal glue, or HGlue. HGlue (and itsvertical counterpart VGlue) are interactor subclassesthat define whitespace of user-specified natural size,stretchability, and shrinkability along their major axis(which for HGlue is the horizontal axis); they havezero natural size and infinite stretchability in their mi-nor axis. Glue is composed along with other elementsof the interface to provide both empty space and slack

- 4 -

: Select MOW Resize Exanine Relate Edit Magnify Narrow :::. :..;:_ :::. i.3 m I i I Q z cl

._’ ‘: ‘_._ _: 1.:. :

! File Edit Composition Font Border FgColor BgColor Align View,$i”‘r$:;: ;:::.. . . .. . . . .

: GraphicBlock FileBrowser TextEditor StringBrowser 0 0 6 3 e MaryinFrarne PulldownMenu i

i PullriyhtMenu ParlelControl CommandControl Vieweri. . . . . . . .._. :. .._I ._. .:..:... _~_._~_._._~.~ _._._.. _._..._._._....._.:....,., _._....._._._. ._...,..,. ,., ., .,. ., ._._ ._ _., ,. ., ., ,., ., ,. ., ._ ,. ,. ., ., ,. ,... ., .,.,.,.,. ,._.,.,.,. ._.,.,.,._._. ._._._.,.,.,./,._.,.,.,., I ,.,.,._. .,.,.,.,.,._.,.,.,.,.,.~.,.,.~.,.,.,.~. :.

Figure 1: Uncomposed TextEditor, VBorder, and VScroller instances

: Select Move Resize Examine Relate Edit Mqnify t4arrow3 m I i I Q z cl

: File Edit Composition Font Border FgColor BgColor Align View

i HGlue VGlue:I

Men :Ti:: Panner M e s s a g e StringEditor i,.:..., :. IGraphicBlock FileBrowser TextEditor WinyBrowser 0 r=> G 9 Q e MqinFrmne PulldownMenu

PullrightMenu PanelControl Comm;arldControl Viewer

Figure 2: TextEditor, VBorder, and VScroller instances composed in an HE3ox

- 5 -

[ GrayhicBlock FileBrowser TextEditor &inyBrowter 3 0 G 0 e TGT MaryinFrame PulldownMenu j:

i PullriyhtMenu PanelControl CommandControl Viewer 1::...................................:..........: ._ _._....._..._..._._.,...,...,..... . .._.....,.,. _... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .,. ._.,._._. ._._.,._.,._._._._.,...,...,.,...,...,. .,...,.,. ._...l.....,.,.,.,.,.,,,.,.,.,.,.,,..,.,...... .,._.,. . . . . . . . . . . . . . . . . . . . . ..,.........,.,.,.,.,.,...,.,.....,..., .,.,. ,.,.,.,.,.,.,.,.,.,., ,.,.,., . ,.,.,.,.,.,.,.,.....~. ,.,.........,..... :.

Figure 3: HESox composition after resizing

GraphicBlock FileBrowser TextEditor StringBrowser (5 0 Q 0 ,/a e MarginFrame

PullriyhtMenu PanelControl CommandControl Viewer

Figure 4: Text editing interface, fully composed

- 6 -

ShadowFrame9

Figure 5: Interactor hierarchy for the text editing interface

between interactors. Glue normally deforms soonerthan other, non-trivial interactors when there is moreor less space than the natural amount. In this case, theglue in the HBox effectively left-justifies its MenuBarsibling.

The MenuBar contains two PulldownMenu in-stances, which it tiles horizontally. It also establishesthe dependencies between the menus so that they workin concert The only visible manifestation of the pull-down menus in the MenuBar is the tag that the userclicks on to expose the menu’s body, which containsits choices. A pull-down menu tiles the interactorsin its body vertically. Any interactor can appear in amenu’s body, including other menus, thereby support-ing hierarchical menus. We explain how the body iscomposed in the next section.

Of existing builders, FormsVBT’s compositionmodel is closest to ibuild’s. There are significant dif-ferences, however. FormsVBT interactors cannot de-fine shrinkability, thus limiting the range of resize be-haviors. Its composition objects have visible manifes-tations that do not appear in the final interface. More-over, Ibuild supports a greater variety of compositionmechanisms, including nearly all that Interviews de-fines [6].

3.2 Modifying the Composition

How does one modify part of the composition withoutdisturbing other parts? Stated another way, how doesone refer to and edit composition objects buried deepin the interface? One could start from scratch or coulddissolve the compositions to expose the leaf interac-tors. But both of these alternatives force the user torepeat work already performed, which is undesirableeven for small interfaces.

Like TeleUSE, ibuild uses multiple views to addressthis problem, but ibuild’s views are always WYSI-WYG. Suppose we want to augment our text edit-ing application’s scrolling interface with two buttonsthat support incremental vertical scrolling, either upor down. In Interviews terminology we could placean UpMover above the VScrolIer and a DownMoverbelow it. This calls for a composition in which themovers and the scroller are tiled vertically. We there-fore introduce a VBox into the existing interface.

First we will create a separate view of the inter-face in which to work, keeping the original view.Any changes we make in one view will appear syn-chronously in all other views. Each view can bescrolled and zoomed independently, and interactorsmay be selected in multiple views independently.

Initially both views display the interface in its en-tirety. Now we will narrow the second view intothe HBox composition containing the TextEditor, theVBorder, and the VScroller. In Figure 6 the user hasclicked on the VScroller with ibuild’s Narrow toolengaged Doing so produces a pop-up menu contain-ing entries that reflect the instance hierarchy in thescroller’s subtme. Each entry is the class name of aninstance along the path from the root of the hierarchy(the topmost entry in the menu) to the parent of theinstance clicked upon (the bottommost menu entry, inthis case, the HBox containing the VScroller). Select-ing one of these entries will narrow the view to thechosen node, eliding other parts of the composition.We can then edit that portion of the interface out ofcontext, as if it was the entire interface.

Figure 7 depicts the two views of the interface withthe one on the right narrowed to view the HBox’s con-tents. Note that the original view (on the left) merelyprovides context by displaying the overall interface.

- 7 -

..,.,.,.> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .A .P. .P... . . . . . . . . . . . . .:... . ._. ._. ._._._. .c. .:.:... .., . . . . . .., . . . . . . .P. . . . . . . . . . . ...‘...‘. . . . . ..‘... . . . . . .c..... . . . .‘..... . . 1. , . . . . . . . . . . ._. . . .,.

~~~~~~ ~~~. ., ., ,. . 1

.,..,.......1;8a~~~~ii~~~~~~~~i~:

. . .;+..& . . . . . . . . . ~ .:..::......:::.....::.::: :..:: I ,:....... ~- text_ed

‘,” A;ij$;; jtg&&. .:.I . .%

: L”-:$ File Edit Composition Font Border FyColor BgColor Align View

HGlue VGlue M e n u l t e m 0 radio q check -

i] HGlue VGlue M e n u l t e m Message StringEditor GraphicBlock-,. ..” “” :

FileBrowser TextEditor iI:

FileBrowse

PullriyhtMenu

Figure 6: Narrowing a separate view

Figure 7: Editing inside the HE3ox

Figure 8: Recomposed interface

The user may create as many or as few views as de-sired, and each may be narrowed independently.

The HBox’s siblings and ancestors are elided in thenarrowed view, and its children can be selected indi-vidually. The user has added an UpMover, a Down-Mover, and two HBorders to the interface and is aboutto compose them in a VBox along with the VScroller.These additions appear in the original view as well.Ibuild does not, however, enforce all composition se-mantics constantly when editing a subview. It is im-portant to preserve the user’s freedom to position in-teractors in the most natural way before composingthem. Restricting an interactor’s placement prior to be-ing composed is unnecessary because composition ul-timately determines an interactor’s placement Avoid-ing any such restriction lets the user arrange interac-tors freely, giving him a feel for spatial relationshipsbefore committing them to a particular composition.Ibuild provides a Natural Size command that reposi-tions the selected compositions to reflect their naturalappearance. Figure 8 shows the interface at its naturalsize.

The user employs the same narrowing mechanismto specify the body of a menu. F’igure 9 shows theuser narrowing in on the “Edit” pull-down menu with

the Narrow tool. Inside the pull-down menu the usersees the tag in its highlighted state, suggesting that themenu’s body is exposed. The user may then instantiatethe interactors that make up the body, including othermenus or compositions of interactors (Figure 10). Asin other subviews, the user may place newly instan-tiated interactors anywhere; the menu’s compositionsemantics will define its children’s ultimate positions.In this case, the pull-down menu will tile its body’schildren vertically.

The ability to edit parts of an interactor compositionis an indispensable one in any builder that supportshierarchy. Note that the composition model that Inter-Views defines and ibuild inherits at first glance seemsan inherently bottom-up approach to interface design.Yet with the subview capability, the designer is freeto revisit and expand parts of the composition in atop-down manner. Menu specification in particular istop-down.

4 Interactor Attributes

In addition to a shape, interactors have other attributesof interest to the user. Ibuild provides an Examinetool for inspecting and modifying these attributes.

Figure 9: Narrowing in on a pull-down menu

~~~~ - InI ’ [part ofJ text ed Helvetica-Bold 12

~~Select3 Movem Re3ii-y Examinei RelateI Editq Mwnifyz 1~~~~~~~~~~~

i File Edit Composition Font Border FyColor BgColor Aliyu View ;:;:::f :::: :,:,.~i~$~

Figure 10: Editing a pull-down menu’s body

FileBrowse

PullriuhtMenu

0 0 Q 0 Q MarginFrame PulldownMenu it

PanelControl CommandControl. . . . . . . . ..1........... 8 .._......~...........~.~,.,*.~....,........ + . ..*_..~.~...~._~.~,..~..............:......:.~.. ._~...._........._......~..,...,..............~......... * .*...._.....,..........~. FE: ._......._.

Figure 11: Examining a TextEditor

4.1 Examining an Interactor

Figure 11 shows the two-level pop-up menu pro-duced by the Examine tool when the user clicks uponthe TextEditor instance in our example interface. Inthis case the user has the option of examining ei-ther run-time information about the interactor (via theInfo... choice) or the static customization properties(via Props...). In either case, a submenu analogousto the Narrow tool’s lets the user choose the instanceto examine in the composition, from the leaf instanceclicked upon to the topmost interactor in the hierarchy.

But by changing the class name, the user directs ibuildto declare a new subclass of the base class and to usean instance of that subclass. The generated code willinclude a subclass declaration that the user can modifyto specialize the behavior of the base class. For ex-ample, our text editing interface would probably defineand use a subclass of TextEditor (instead of TextEditoritself) that implements application-specific input han-dling. Specifying subclasses provides a simple way toextend the behavior of ibuild’s predetied interactorsprogrammatically.

If the user chooses to examine the TextEditor’s run-time information, then an Interactor Information di-alog will appear as shown in Figure 12. Had the userchosen to examine the TextEditor’s properties, an In-teractor Properties dialog would have appeared inwhich to type static attribute names and their values.The Interactor Properties dialog is the same for all in-teractors: it has a simple text editor-based interface forspecifying the properties. The Interactor Informationdialog varies from interactor to interactor, though allhave a class name, a base class name, a member name,canvas dimensions, and shape information.

The interactor’s member name identifies the inter-actor to application code should the application needto access it; this issue is discussed further in Section 6.The interactor’s canvas defines the actual screen spacethe interactor occupies, which may be different fromthe desired size specified by its shape. Most interac-tors computer their shape from information they keepinternally; thus their shape information is not editablein the information dialog. However, glue and otherinteractors that parameter& their shape have editableshape information fields.

When the class name and base class name are the TextEditors have a minimum of run-time attributes.same, ibuild will instantiate the corresponding class. Below is a sampling of the information that can be

Move3

Resize :

i HGlue VGlue M e n u l t e m : amer[plil Message StringEditor GrayhicBlock FileBrowser TextEditor

; GraphicBlock FileBrowrseri. StringBrowser Q 0 G 9 CQ 62

Tex?MarginFrame PulldovrnMenu

i PullrightMenu Panelli; _..*.*. . .: PullrightMenu PanelControl CommandControl Viewer

. . *.*_....;, 9 :_._.,...,.........., ,.,...,.,...,... . . . . . .,. . . . . . .._............ ,. .,.....,. .,.:,.,..... .,._. . . . . . .,... ..I....,... . . . .:..,.,... .,._.,.,...,... . ..*.._.... .:..: .::.._. . .._I _.. .,I,._... ._._I ._ ..:. . . . . . . . . . . . . . . . . ..I.. _j:_ ._ _._ ._. _. ._ ._. _. ._ _. _. . . _. ._ . ._. ._. _. . ,. . ., ._. ._. ._, ._._._. ._._ ._.,. ._._._.,._. ._. ._ . ..gymT. ._._ _._. .,._. ._. ._. ._._._. ._,_._. ._. ._. ._._. ._,_._._.,._. ._._._._._. ,_. . ._ . . . . ._ . . . . . , ._ ._ . ._. _,

Figure 12: TextEditor attributes

specified for different interactors through the Examinetool interface:

l ScrollBars and Movers have a field for specifyingthe member name of the interactor they scroll.

l Menus have a field for specifying a member func-tion to call when invoked.

l Buttons provide an interface for specifying But-tonstate information [6] along with a memberfunction to call whenever the ButtonState’s valuechanges.

l FileBrowser (a scrollable list of files in a direc-tory) provides fields for specifying a directory tosearch and regular expressions for filtering files.

l The pop-up menu for GraphicBlock (an interac-tor that displays, scrolls, and zooms structuredgraphics objects [12]) and Viewer (a Unidraw ob-ject derived from GraphicBlock) adds a Graph-ics entry that lets the user specify the graphicsthey display through a drawing editor interface(idraw [9]).

4.2 Relating Interactors,

Some interface semantics establish relationships be-tween interactors. For example, a scroll bar normallyscrolls another interactor, some button choices mightexclude others, and double-clicking in a scrollable listof choices might be equivalent to pressing a button todismiss the dialog. These relationships can be estab-lished through dialog boxes produced by the Examinetool, but such interaction is indirect at best and clumsyin practice.

Ibuild’s Relate tool provides a direct-manipulationinterface for establishing relationships between inter-actors. The Relate tool involves two interactor in-stances, the source and the target. The user identifiesthe source and then the target by clicking on each inturn. As with the Examine tool, the Relate tool popsup a menu in response to each downclick to let theuser select the instance in the hierarchy being related

To demonstrate how the Relate tool works, we willuse it in our text editing example to make the scrollerand movers adjust the TextEditor instance. Figure 13shows the pop-up menu produced by clicking on theVScroller instance. The user specifies the source byselecting an entry in the pop-up menu. In this case theuser could choose the VScroller, but then the movers

/I File Edit Composition Font Bo

TextEditor

rg!Y MarginFrame PulldovrnMenuGraphicBlock FileBrowser Texii

. PulhightMenuPan&i.

Viewer_L *id‘. A..... . . . . ..I... ..- -.,, •,‘*IYL.~ Y.iiV1/.~ . . . . . . . . . . . . . . . . . . . . . ~‘~ L ,,...................,.......,..,.,..,,.,..... . . . . . . . . . . . .

Figure 13: Specifying the source with the Relate tool

must be related separately. Instead, the user can relateboth movers and the scroller at once by making theVBox that composes them the source, thereby speci-fying a recursive relation.

Once the source is specified, the next step is toidentify the target by clicking on the TextEditor in-stance (Figure 14). Again, a menu pops up to let theuser choose the instance in the hierarchy. Ibuild drawsa line from the source to the current mouse positionto remind the user to specify the target. Upclickingthe mouse over the “TextEditor” entry in the menucompletes the relation. If the user now examinesthe scroller or either mover, the TextEditor instance’smember name will appear in the information dialog’sfield that identifies the instance adjusted

The user may use the Relate tool to establish ameaningful relationship between many combinationsof interactors. The semantics of the relationship de-pends on the interactors and sometimes on the orderin which the relation is made, but no more than onesuch relationship exists between two inter-actors-therecan be no ambiguity. For example, relating one radiobutton to another ensures that they mutually excludeone another. Relating a push button to a dialog makesthe push button dismiss the dialog. The Relate tool

prevents the user from making meaningless relations(for example, between a pull-down menu and a bor-der) simply by not popping up the menu (source ortarget) that would permit such a relation.

5 Unidraw Abstractions

In addition to providing widget-level support, ibuildalso incorporates Unidraw abstractions that simplifythe construction of graphical editors. Unidraw is de-signed to span the gap between traditional user inter-face toolkits and the implementation requirements ofthese applications. It partitions their basic functional-ity into four class hierarchies:

1. Components represent the elements in a graphi-cal editing domain, for example, geometric shapesin technical drawing, schematics of electronicparts in circuit layout, and notes in written mu-sic. Components encapsulate the appearance andsemantics of these elements. The user arrangescomponents to convey information in the domainof interest. A component is made up of a sub-ject and zero or more views: the subject definesthe component’s semantics, while views define

:ji Select Move

‘1 File Edit Composition Font Boi

Figure 14: Specifying the target

presentations of the subject. Components can bestructured hierarchically.

Tools support direct manipulation of components.Tools employ animation and other visual effectsfor immediate feedback to reinforce the user’sperception that he is dealing with real objects.Examples include tools for selecting componentsfor subsequent editing, for applying coordinatetransformations such as translation and rotation,and for connecting components.

Commands define operations on components andother objects. Commands are similar to messagesin traditional object-oriented systems in that com-ponents can receive and respond to them. Com-mands can also be executed in isolation to per-form arbitrary computation, and they can reversethe effects of such execution to support undo.Examples include commands for changing the at-tributes of a component, duplicating a component,and grouping several components into a compos-ite component.

External representations convey domain-spe-cific information outside the editor. Each com-ponent can define one or more external represen-

tations of itself. For example, a transistor compo-nent can define both a PostScript representationfor printing and a netlist representation for circuitsimulation; each is generated by a different classof external representation. An external represen-tation object thus defines a one-way mapping be-tween a component and its representation in anoutside format.

The Unidraw library defines the base classes and a setof subclasses for each of these elements. The prede-fined subclasses support basic drawing editing capa-bilities. For example, the library includes:

l components for elementary shapes such as linesand polygons

l tools for creating and selecting components andfor applying coordinate transformations by directmanipulation

l commands for cut, copy, paste, undo, redo,changing component attributes, saving and restor-ing drawings

l external representation objects for generatingPostScript

Figure 15: Composing Unidraw objects

The library also defines special interactors to en-gage and control these abstractions. It defines twosubclasses of Interviews’ Control class, which isthe base class for interactors that execute an action:CommandControl provides an interface for executingcommands from menus, and PanelControl engages aparticular tool. Ibuild includes these interactor classesand lets the user specify via the Examine tool the com-mand or tool subclass they use. A user can specify ei-ther a predef3red command or tool subclass by name;he can also extend the existing classes programmati-tally by subclassing them as he can interactors.

Ibuild currently supports two additional Unidrawclasses:

1. A Viewer displays a graphical component view,most often the root view in a hierarchy. A viewerprovides a framework for displaying the view,supporting such “non-semantic” manipulations asscrolling and zooming. Viewers also take rawwindow system or toolkit events and translatethem into Unidraw protocol requests.

2. An Editor is an interactor subclass that composesthe application’s interface. It associates tools (inPanelControls) and commands (in CommandCon-trols) with one or more viewers and composesthem along with other toolkit objects into a co-

herent user interface. A Unidraw-based applica-tion can create any number of editor objects toprovide a multi-view editing environment.

Figure 15 depicts how one might compose a graph-ical editor with ibuild’s Unidraw abstractions. Panel-Controls for engaging various tools appear composedin a VBox along the left side of the interface. Thetop three PanelControls engage tools for editing exist-ing components, while the bottom three let the userinstantiate components. CommandControls containinguser-executed commands are composed in pull-downmenus along the top. The Viewer in the center displaysthe component view hierarchy that the user edits. Theinterface is composed in an Editor object, which co-ordinates the operation of the other Unidraw-specificobjects.

6 Integrating Interfaceand Application Code

A user interface builder’s objective is to implementan interface from the user’s graphical specification.To this end, most builders generate toolkit code, andibuild is no exception. However, the user interface isnormally just a part of a larger application. We have

- 15 -

core subclassstubs for redeflnlng vlttuals

programmer-added state or protocol

(read-only)

(read-only)

Figure 16: Core class and subclass derivation

designed a novel technique for integrating interfaceand application code and for keeping the two consis-tent amidst changes to one or the other.

Ibuild generates a C++ class called the core classfor each interactor subclass in an ibuild editing session.A user wiIl usually deline a subclass for each top-levelinteractor composition in the interface. The core classimplements the user’s interactor composition and de-clares all exported member names and functions. Theuser exports a name by checking “Export” beside thename in the interactor information dialog. For exam-ple, ibuild generates a protected member variable foreach interactor that has exported its member name, anda public virtual function may exist for each menu itemin the composition. Yet protected members cannot beinvoked by other objects, and the member functionshave no implementation. So how do these supportcode integration?

The programmer could modify the core class towork with other application code, but subsequent edit-ing and regeneration of the interface in ibuild wouldcreate a code merging problem. Instead, ibuild gener-ates a core subclass in addition to the core class. Asthe name suggests, each core subclass is derived from acore class (Figure 16). The programmer modifies thecore subclass, not the core class, to establish a linkto the rest of the application. He can define memberfunctions that provide controlled access to the exportedmember variables, and he can redefine the core classvirtual functions to do useful work.

If the interface’s appearance is later modified, thenthe user can choose to regenerate c&y the core classfiles, which were not modified-the changes to thecore subclass are thus unaffected. The programmercan then recompile the application, and it will reflectthe change in appearance. Only if the designer makesradical changes to the interface (such as deleting in-

application software

4system software

Toolkit (Interviews)

1Window system (X) 1

Figure 17: Layers of software underlying ibuild

teractor instances upon which the application depends)should it become necessary to edit the core subclassagain.

7 Implementation

Ibuild’s implementation relies on Unidraw for itsgraphical editing capabilities and on Interviews’ pre-defined interactors to implement its look and feel. Fig-ure 17 depicts the dependencies between these layersof software.

7.1 Interactor Components

Ibuild defines a component subject subclass called In-teractorComp from which to derive components thatsimulate the predefined Interviews interactors. Inter-actorComp adds two Interactor-inspired methods to thebase graphical component protocol: Reconfig notifiesthe InteractorComp to compute its desired shape basedon its children and whatever other criteria it considers,and Resize signals that the InteractorComp’s canvas(actual screen space) has been allocated.

Though the Interactor base class defines consider-ably more protocol than these two methods, they wereall that were needed for our simulation. We initiallywanted ibuild to simulate input behavior, but later werealized that there was little to be gained from lettingthe user push buttons on and off. Therefore ibuild’sinteractors do not interpret input, and they and have noneed for the equivalent of Interactor’s Handle methodor for objects that specify event interest. Rendering op-erations are already part of Unidraw’s graphical com-ponent semantics. Child traversal, window manager,and other operations are similarly superfluous.

One benefit of these simplifications is that ibuild’sinteractor simulations consume only about 60% of thememory of their interactor counterparts, though thisnumber increases with each additional view. A sig-

nificant cost associated with interactors is the X win-dow each one allocates, for which the simulated in-teractors have no need. Also, the multiple view fea-ture is simply inherited from the Unidraw architec-ture; no special work-arounds were needed to im-plement it. Other features that come for free areview scrolling and zooming, incremental screen up-date, hit detection, Unidraw’s framework for support-ing direct manipulation, standard editing features suchas cut/copy/paste, and arbitrary level undo and redo.Moreover, we have found it easier to simulate an in-teractor in Unidraw than to create the real one fromscratch; thus the simulation-based implementation canserve as a testbed for interactor development.

7.2 Code Generation

CodeView is the base class from which external rep-resentation objects for each interactor component arederived The name “CodeView” reflects the fact thatexternal representation objects in Unidraw are simplyanother kind of component view. The external repre-sentation itself is Interviews and Unidraw code writtenin C++. A CodeView subclass instance is instantiatedfor the top-level interactor components at code gener-ation time. Each subclass is responsible for generatingthe source code representation for the component sub-ject it views.

Code generation is divided into multiple phases,each entailing a traversal of the entire CodeView hier-archy. This hierarchy mirrors the interactor hierarchybeing simulated. Each CodeView instance is responsi-ble for generating the proper code fragment in a givenphase. For example, forward declarations are emittedin a phase in which all participating CodeViews areasked to generate their own forward declarations. Notall CodeViews participate in all phases. For example,only a few CodeView subclasses generate ButtonStatedeclarations, because not every interactor has a But-tonstate.

AI1 the traversal machinery and protocol for emittingcode are inherited from Unidraw’s external represen-tation framework. The CodeView base class initiateseach pass and checks for errors. CodeView subclassesdefine the code fragments appropriate to each interac-tor component.

8 Future Directions

Ibuild has proven to be flexible and powerful enoughto eliminate the need to hand-craft most interactor codeand a significant amount of Unidraw code. Currently

we are focusing on penetrating deeper into Unidrawabstractions. Ibuild supports only the highest-level as-pects of the Unidraw architecture. It lets the user incor-porate existing components, commands, tools, view-ers, and editors into an interface, and the only supportfor extending the piedefined objects’ semantics is pro-grammatically, by subclassing. Yet Unidraw definesobjects at lower levels that help programmers imple-ment graphical connectivity semantics, dataflow anddependency maintenance between components, andthe direct manipulation behavior of tools. Graduallyinteractive analogs of these programming abstractionswill make their way into ibuild’s interface.

Another area we plan to address is support for Inter-Views’ glyph abstraction [2]. Glyphs are lightweightand sharable structured graphics objects that can beused to represent the appearance of any object in agraphical user interface. Glyphs support a super-set ofexisting interactor composition mechanisms, includ-ing support for high-quality text formatting. Tradi-tional Interviews interactors are being replaced byglyph-based versions. This has several implicationsfor Unidraw and ibuild:

Glyphs provide a superset of Unidraw ‘s structuredgraphics capabilities and are much less expen-sive. We therefore plan to retarget Unidraw touse glyphs.

Ibuild should generate glyph code.

Simulating glyph-based interactors would be eveneasier than current interactors. Most of the ef-fort involved in writing a new interactor compo-nent is in expressing the target interactor’s ap-pearance in terms of structured graphics objectsand in simulating the resizing behavior. Glyphsdefine both, so interactor components could usethe glyphs from glyph-based interactors directlyto define their appearance. This also eliminatesthe need to keep the builder consistent with thetoolkit, though this has not proved to be a prob-lem because the library of predefined interactorsrarely changes.

Finally, we plan to extend ibuild to support standardlook-and-feels and perhaps other toolkits. Already theglyph-based interactors implemented so far supportthree interface styles. Once ibuild supports glyphs,then, supporting multiple styles should be straightfor-ward.

- 17

9 Conclusion

Ibuild demonstrates how a user interface builder canleverage the functionality of object-oriented libraries.It offers several features not found in existing builders:

A composition metaphor powerful enough to de-fine interface layout and resizing semantics with-out giving up a WYSIWYG interface.

Viewing facilities at least as good as other kindsof graphics editors.

Abstractions that go beyond the widget level intothe realm of graphical object editing [ 111.

An implementation that is not tied intimately toa particular widget set.

An architecture that can support more than onetoolkit, including unrealized ones.

Extension and code integration mechanisms thatleverage the C++ object model. The program-mer can extend the functionality of any built-in Interviews or Unidraw object. Moreover, theprogrammer can change the semantics of ibuild-generated code without precluding further editingin ibuild and without special environmental sup-port.

We see a trend towards a proliferation of object-oriented libraries that offer domain-specific solutionsto application development problems. Early libraries

. supported basic data structures, input/output manage-ment (e.g., C++ streams), and graphical user interfaces(i.e., widgets). Newer libraries address areas like mul-tithreading (i.e., tasking), debugging, graphical editing(e.g., Unidraw), and signal processing.

Each new library brings with it an opportunity forsome sort of builder, just as widget sets begat user in-terface builders and Unidraw’s abstractions made theirway into ibuild. Builder development on this scale willbe difficult without even higher-level programmingabstractions, because it is unlikely that programmerswill be able to exploit an instance-based implementa-tion in builders for non-graphical libraries. This onlystrengthens the argument for a simulation-based ap-proach. It also points to the need for builder-buildingabstractions that simplify builder development for anylibrary.

References

VI Gideon Avrahami,Marc H. Brown. A

Kennethtwo-view

P. Brooks,approach to

andcon-

PI

[31

r41

PI

WI

[71

WI

PI

WI

WI

WI

strutting user interfaces. In ACM SZGGRAPH ‘89Conference Proceedings, pages 137-146, Boston,MA, July 1989.

Paul R. Calder and Mark A. Linton. Glyphs: Fly-weight objects for user interfaces. In Proceedingsof the ACM SIGGRAPH Third Annual Sympo-sium on User Interface Software and Technology,pages 92-101, Snowbird, UT, October 1990.

Luca Cardelli. Building user interfaces by directmanipulation. In Proceedings of the ACM SIG-GRAPH Symposium on User Intevace Software,pages 152-166, Banff, Alberta, October 1988.

Scott E. Hudson and Shamim P. Mohamed. In-teractive specification of flexible user interfacedisplays. ACM Transactions on Information Sys-tems, 8(3):269-288, July 1990.

T.G. Lewis, Fred T. Handloser IlI, Sharada Bose,and Sherry Yang. Prototypes from standarduser interface management systems. Computer,22(5):5 l-60, May 1989.

Mark A. Linton, John M. Vlissides, and Paul R.Calder. Composing user interfaces with Inter-Views. Computer, 22(2):8-22, February 1989.

Brad A. Myers, Brad Vander Zanden, andRoger B. Dannenberg. Creating graphical inter-active application objects by demonstration. InProceedings of the ACM SIGGRAPH Second An-nual Symposium on User Interface Software andTechnology, pages 95-104, Williamsburg, VA,November 1989.

Gurminder Singh, Chun Hong Kok, and Teng YeNgan. Druid: A system for demonstrational rapiduser interface development. In Proceedings of theACM SIGGRAPH Third Annual Symposium onUser Intelface Software and Technology, pages167-177, Snowbird, UT, October 1990.

Stanford University. InterViews Reference Man-ual, Version 3.0, 1991.

TeleSoft, Inc., San Diego, CA. TeleUSE User’sManual, 1990.

John M. Vlissides. Generalized Graphical ObjectEditing. PhD thesis, Stanford University, 1990.

John M. Vlissides and Mark A. Linton. Applyingobject-oriented design to structured graphics. InProceedings of the 1988 USENIX C++ Confer-ence, pages 81-94, Denver, CO, October 1988.

- 18 -

[13] John M I&sides and Mark A. Linton. Unidraw:A framework for building domain-specific graph-ical editors. ACM Transactions on InformationSystems, 8(3):237-268, July 1990.

[14] Bruce F. Webster. The NeXT Book. Addison-Wesley, Reading, MA, 1989.