Sandboxed Interaction for CasualUsers in Shared Spaces

Andrea Albarelli

e-mail: albarelli@unive.it

Augusto Celentano

e-mail: auce@unive.it

Universita Ca’ Foscari Venezia

Dipartimento di Scienze Ambientali,

Informatica e Statistica

via Torino 155, Mestre VE, Italy

Copyright is held by the authors.

CHItaly 2013, September 16–19, 2013, Trento, Italy.

AbstractA simple and natural interaction is considered the mostimportant feature of an interface when dealing withmultiple impromptu users in public spaces. Intuitivenessand forthright feedback are key factors to enable a steeplearning curve for untrained users that need to grasp theinteraction model in a short time span. While manysystems meet this requirement, some fall short when thenumber of users grows and their behavior is exceedinglyunrestrained. We introduce the idea of sandboxedinteraction, where the user actions are constrainedthrough a specially crafted combination of physical andsoftware-based filters. A case study is discussed related tothe cultural heritage domain.

Author Keywordsnatural interaction, multiuser interface, sandboxing,shared environment

ACM Classification KeywordsH.5.2 [User Interfaces]: Input devices and strategies,Interaction styles.

Casual interaction in shared public spacesTouch based, tangible and gestural interaction are oftenconsidered as the most effective interaction styles fordirect, intuitive and casual human-machine interfaces. Atypical scenario is offered by interactive installations in

public spaces, such as museums, cultural exhibits, airportand station lobbies. Within these scenarios, usersapproach an information delivery system without explicitinstruction and must be able to relate with it in aproductive manner within the few minutes of attentionthat are usually allocated to such situations. This resultcan be achieved through context specific and easy to useinterfaces that take advantage of different natural inputmeans such as touch sensing, eye tracking [4], tangibleinterfaces, gesture recognition [3], up to full-body 3D posereconstruction [2].

While these interfaces are meant to support a well defineddegree of scalability, most of them are limited to very fewusers at the same time and are prone to failure whenspecific critical conditions are met, such as:

• when the number of concurrent users exceeds thephysical capabilities of the underlying hardware(e.g., the amount of simultaneous touches that canbe detected by a surface or the number of peoplethat a Kinect-based sensor can track);

• when the occurrence of many simultaneousinteractions, while correctly handled by thehardware, hinders the causal link between the actionand the result, thus confusing the users (e.g., whentwo users issue contrasting commands);

• when the user abuses the interface itself (e.g., byusing tangible inputs not in the intended way);

• when the user acts too hastily or too slowly withrespect to the meant interaction dynamics.

Usually, the failure mode is not tied to a crash or totallack of usability of the system, but rather to an increasingconfusion in the system response. Unfortunately, thiseffect is exactly the most important to avoid in theaforementioned scenarios, since it interferes with the

ability of the user to quickly get acquainted with theinteraction metaphor.

Most of these conditions can be avoided by adoptingoverload oriented and robust design solutions, or byaugmenting the system with additional artifacts (e.g.,tangible elements) or software fences (e.g., warningfeedbacks). To our knowledge, however, no systematiceffort has been done so far in order to classify thesemeasures and to study their effectiveness under differentassumptions, use cases and metrics. To fill this gap and tostart a discussion about such topics we are proposing theconcept of sandboxed interaction.

Sandboxed InteractionSandboxing is a concept implemented in mobile operatingsystems to separate the working spaces of differentapplications, with the goal to avoid interferences betweenthem and improve security. In computer interfaces, withthis term we mean the separation of the users and theiractions effect to avoid conflicts between them, and also toavoid system overload due to the combinatorial effect ofmultiple user interaction. Traditional WIMP interfaces arenaturally sandboxed, because the interaction possibilitiesare constrained by the widgets of the interface, that arelinked to one window at a time, are visible and wellrecognizable. In advanced forms of interaction provided bytangible interfaces, touch devices and gestural interfaces,the evidence of what user action activates a systemfunction is less evident. Also, the size of the interactionspaces and its intrinsic multi-user nature augments therisks of conflicts between users. Hence some form ofseparation of the interaction spaces is necessary.

Our first study of sandboxed interaction applies to ascenario where a set of users interacts with a sharedinformation delivery system retrieving visual and audible

information. The goal of a sandbox is to limit the numberand nature of actions directed to the interface. Thisdiffers from visual hinting, usage suggestions or otherforms of user persuasion, since the goal of a sandbox is tostrictly enforce a compliant behavior. Such enforcementcan happen in three main ways:

Figure 1: Structure of themultiuser interactive table

Figure 2: People interactingduring an exhibition.

Figure 3: Artworks displayed onthe wall in front of the table.

• physical constraints: this can include bounds to thenumber of users due to the limited availability ofinteraction means (i.e., of physical objects intangible interfaces) or to constraints in the motionand actions available to the interface (i.e. limitationof the movements of a cursor to a section of theinterface, reduced number of degrees of freedom orlimits to the maximum speed);

• feedback cues: visual, audible or haptic feedbackthat informs the user of a non compliant behaviorand contextually stops the response of the interfacein a graceful downgraded way. This differs from softhints and suggestions since the feedback alwaysleads to the halt of the normal response of thesystem;

• information privatization: shifting the interactionfrom the public shared interface to a private one(e.g., a set of personal devices instead of a largeshared surface), which is expected to offer anindependent personal navigation.

Case study: a map-based multiuser art browserThe described principles of sandboxed interaction havebeen applied to a multiuser interactive installation in anart exhibition at Ca’ Foscari in Venice. The installationshows views of Venice painted by famous artists in XVand XVI Centuries; each painting is referred by a locationof a translucent Venice map applied to the surface of alarge multiuser touch table. (Fig. 1-1 and 1-2). A set ofpoints of interest on the map were highlighted by a

computer controlled projector under the table and, inresponse to placing a cursor on a location (Fig. 1-3 and1-4), a painting depicting that location was displayed onthe wall (Fig. 1-5). Four wall projectors, each projectingone or two paintings, accounted for at most eight differentpaintings, corresponding to eight different locations inVenice, to be visible. Figures 2 and 3 show the actualinstallation. The technical solutions adopted in buildingthe table and its interface are described in [1].

A first prototype built on a multitouch table, where userscan select locations by putting a finger on the map spots,showed several severe shortcomings during the early tests.Being the table quite large (3 by 2 meters), many visitorsof the exhibition were able to access it at once, triggeringa very high number of events, which in turn led to a veryfast pace in the change of the displayed artworks with aconfusing result. Furthermore, many users (especiallychildren) were leaning against the table surface toexamine the details of the map, triggering a large andunpredictable number of additional false events due toforearms, elbows and trunks. The negative effect of suchunforeseen events was eventually exacerbated by thelimited number of wall-mounted projectors with respect tothe visitors interested in the installation.

In order to mitigate those problems, we chose to put aphysical constraint to the availability of the system. Weintroduced a cursor, a passive device (Fig. 1-3 and 4) thatthe user must place on the point of interest, instead ofdirectly touching it. A cursor is a little brick with infra-redreflective material on the bottom, detected by the tablehardware suitably tuned. Using special infra-redilluminators and cameras, we were able to filter the cursorsfrom touches and other unwitting contacts. Furthermore,by controlling the number of available cursors, we werealso able to limit the number of active users.

The size of the cursor, and the size and density of themap, were an obstacle to the exact positioning of thecursor over a sensible point. For such reason, we added afeedback cue in the form of light flames flowing from thecursor to the nearest point of interest (Fig. 1-4 and 4).This feedback is, correctly, only a suggestion and not anautomatic completion: the related artwork is not displayeduntil the cursor is actually moved over the interest point.

Figure 4: A passive cursorsuggesting a selection.

Figure 5: Two active cursors.

Figure 6: Using an active cursoras a private device.

As a third solution to the gap between the potentialnumber of users and the system interpretation and displaycapabilities, we also applied a form of informationprivatization by introducing active cursors (Fig. 5) builtaround a smartphone featuring a private touch interface.While working like the passive cursors when placed on thetable, their main function was to let the user, afterselecting an artwork, to “lock” it on the smartphone for apersonal, private navigation through additional paintings,related to nearby points of the map, without committingto the projectors and without further physical interactionwith the table (Fig. 6). This allowed us to raise the totalnumbers of concurrent users.

Final RemarksThe proposed techniques for sandboxed interactionaddress the problems of casual users and shared interfacesin three different ways, with the cumulative effect ofreducing the system overload and avoiding user conflictswithout usability losses. The presence of physicalinteraction objects suggested the possible actions in aclear and easy way. Active cursors allowed users todecouple their navigation from the public space, inducinga sense of personalization and privacy to their experience.

No systematic test and evaluation has been done so far,but the system has been fully operational during theexhibition for two months, behaving properly and without

performance loss even in crowded situations. Theobserved behavior has been consistent with the expectedone: the presence of the passive and active devicesprevented the generation of too many possibly conflictinguser interactions, which would occur with bare fingertouches. Indeed, many users were trying to interact withfingers, a consolidated habit with interactive tables;observing other users helped to understand the interactionlogic and the correct use of the devices.

Future work will face the definition of a kind of sandboxontology for designing and applying the constraints in acontext dependent way, the selection of appropriatemetrics for assessing their effectiveness and, finally, asystematic experimental evaluation.

AcknowledgmentsFilippo Bergamasco, Luca Cosmo and Andrea Torsello aregratefully acknowledged for their contribution to thedesign and implementation of the multitouch table.

References[1] Albarelli, A., Bergamasco, F., Celentano, A., Cosmo,

L., and Torsello, A. Using multiple sensors for reliablemarkerless identification through supervised learning.Machine Vision and Applications (2013), 1–16.

[2] Cafaro, F., Panella, A., Lyons, L., Roberts, J., andRadinsky, J. I see you there!: developing identity-preserving embodied interaction for museum exhibits.In Proc. SIGCHI Conf. on Human Factors inComputing Systems, ACM (2013), 1911–1920.

[3] Malerczyk, C. Interactive Museum Exhibit UsingPointing Gesture Recognition. In WSCG Short papers,(2004), 165–172.

[4] Milekic, S. Gaze-tracking and museums: Currentresearch and implications. In Museums and the Web(2010), 61–70.