Affordances, Media Spaces and Privacy/Awareness

1

Cognitive Psychology

Traditional approach in psychologyDominance of direct sensory inputs: light, sound, etc.

broken down into 'objective' phenomena e.g. wavelengths

Construction of a symbol to classify input stateIf light in a particular range of wavelengths then Red

Fitting that symbol into goal-directed algorithm or planIf I approach a red light then I stop my car

Didn't (doesn't) fit with observations?Real-life classification is far more dynamic and contextualRational plan-based action: too simple a model of most of

everyday life Remember Suchman’s critique of Winograd & Flores?

J.J. Gibson: Ecological Psychology

New (1950s) approach to psychology: still competingKey concept: affordance

A potential use as perceived by an individualBased on both sense data (i.e. physical phenomena) and

observer (i.e interpretation and understanding)

Strong influence on HCI in 1990s: Don Norman, Bill Gaver...

Perception directly based on actionUnderstanding things by doing, acting, working…

Direct, non-rationalised behaviour as the normNo indirect steps of naming, rule selection, etc.

Occasional reliance on abstract symbols and plans

Perceptible and False Affordances

Door handle exampleShape affords action of turning and pullingHandle has a perceptible affordance Designer's intended interpretation happens in useMatch between designer’s and user’s interpretations

If well-designed, user’s interpretation does not involve rational analysis, i.e. the invisibility or transparency that is ubicomp’s design ideal

Door locked? Door turns the other way?Handle has false affordances, i.e. it is misinterpretedMay suggest actions, but not useful/intended onesErrors are common, so extra symbols needed e.g. “Push” sign

Hidden Affordances

Design does allow intended action…but it is not directly perceivableUser may ‘misinterpret’ the design

User interprets the design, but not in the intended way

Reliance on learned behaviour or rational analysisLearning may let one person perceive a symbol where another can’tAnalysis may mean slower use, risk of failure

Reveal affordances by using other symbolsAdd symbols ‘now’: textual labels, tooltips, etc., or non-textual

media such as sound, KARMA’s overlaid graphics…Or rely on past use of symbols: discussion, explanation,

reading, examples the user has seen…

Culture, Experience and Learning

Gibson and GaverSome affordances developed via culture, experience and situation

...but others exist independently of this background?

Problem with “affordances as properties that can be analysed in their own terms” is subjectivity and contextuality

Extreme example of different interpns: blind, disabled or Stone Age man

Is there a line between individual and cultural learning? No

Gaver focuses on low-level perception“the complementarity of handles and the motor system”

He took insufficient account of high-level perceptionPerception is not just the motor system or the eyeIt is also dependent on learning, experience, culture… language

Complex Actions, Complex Affordances

Sequential affordances: symbols distributed over timeEach action leads to new affordancesGrasping door handle reveals ability to turn itKARMA: illustrations adapt to sequences of subtasks

Nested affordances: symbols distributed over spaceA flat network of symbols in a local area

Not necessarily hierarchical

Forms the context, the whole helping interpretation of each partA hidden door becomes apparent with a doormat in frontA web link is hidden but nearby text tells you where to click

Perception as a Dynamic Process

Perception is an active and exploratory processActivity and experience continually resolve ambiguityLoop: each action opens up a new state or context

New combinations of symbols and media, that reveal new potential interpretations and (re)actions

System design has to take this into account (e.g. ubicomp)Continually adapting to the ongoing context & activityA newly sensed event may not be significant in itself

But its role in a later combination of ‘current’ features may be

More parallelism & less control when compared to GUIsNot all the objects people use are made usable via the system (e.g.

furniture, buildings) and the system can’t control user actions on them (e.g. moving towards them, pointing at them)

Stepping in a Muddle

“[The] philosophical muddle of assuming fixed classes of objects, each defined by its common features and then given a name. As Ludwig Wittgenstein knew, you cannot specify the necessary and sufficient features of the class of things to which a name is given. […] You do not have to classify and label things in order to perceive what they afford.” J.J. Gibson

Gibson built ecological psychology on the concept that the meaning of a name or symbol is its patterns of association and use with other symbols in everyday life

To do this, he drew on 20thC philosophers who established that concept: Wittgenstein, Heidegger...

Ontology

Ontology: what is the basis of activity & perception?What ‘types’ are most fundamental to meaning?

Shift from objective analysis to subjective activity1800s positivism: reliant on absolute and objective features that

every mind pictures, models and plans with“Independent, objective and contemporaneous” features

Early/mid 1900s: Heidegger and WittgensteinEveryday non-rationalised activity as the normPlans and rationalised action as exceptional, as tools we use when

more efficient learned patterns of activity are not applicableThis is where Suchman’s critique of Winograd & Flores came from

Positivism: An Approach Born in the 19th Century, but Refusing to Die

19th century linguistics: systems of absolutesClaim that a symbol can be analysed alone as a ‘positive’ entity

Something that can stand by itself, have meaning in itself

Each symbol refers to a unique, absolute, objective, ideal ‘thing in the world’

There is a historically traceable process of words improving their fit with these absolutes

These absolutes offer solid taxonomic benchmarks for classification schemes, and are what meaning stems from

We can find these perfect absolutes and schemes if we just work hard enough

So, the written word “red” refers to an ideal, perfect absolute: Red

Problem: It Doesn’t Work

Action & interpretation via absolutes doesn’t fit with real lifeInconsistency with how people normally speak or act

e.g. fights over tight definitions of words, objects and types

e.g. UBS spent over two years arguing what a bank account was

There ‘should’ be increasing stability over time, but… no“Cattle” now means cows, but 400 years ago meant any property

There ‘should’ be a trend towards 1:1 matches, but… noOften one language uses one word where another language uses two or

more

“River” covers both the French words “riviére” and “fleuve”

Japanese has more than one word for the number one

20th Century Linguistics

F. de Saussure’s ‘Course in General Linguistics’Published posthumously in 1916, based on his lecture notes

Each word is part of a relative systemNo external system of absolutes to rely onThe meaning of each symbol is the patterns of use relative to

other symbols Especially the other symbols used in the same context, and the other

symbols you could replace the symbol with in a given context

Symbols can be in any or all media This means new/digital media and old/analogue media, all interwoven Patterns of co-occurrence in use are not dependent of a symbol’s form or

medium, but on patterns of use in everyday life Mixing types of symbols meant that linguistics became unified with

semiology (or semiotics) to become the general science of symbols

Red

It’s not necessarily about wavelengths of light Not just the form of the wave, or even the medium of light

To understand when you can call something “red”, you have to understand “brown,” “orange,” yellow”…To know when to use “red” or another colour wordYou also have to understand “blood”, “rose”, “communist”…

Note that associations span mediaThe word red on the page is associated with the spoken word “red”... but also a

drop of blood, roses in a garden, a 1950s movie of a military parade in Moscow...

Subjectivity and Adaptation

Each person experiences the world differentlyExperiences a different pattern of associations/interpretationsBut… does that mean randomness? Anything goes? (No.)

Adaptation as a social and evolutionary processInteraction with other people (and with the world) drives people’s

patterns toward consistency and utilityEfficiency gain from meanings and patterns of behaviour that are both

useful and consistentBut... too much consistency means no adaptation to new contextsA problem in workflow systems: too static, too consistent, too limiting

Language needs some variation, to evolve language so as to let us handle new uses, contexts, technologies, etc.

Individuals’ creativity (and misinterpretation of others) continually creates new symbols and patterns ⇒ if useful, they survive and spread

Contemporary Ontology

Language is interpretive activityNot just verbal activity, but all media togetherA process not a state

Perception is influenced by experience, and they both influence activity, which adds to experience

So each person is in an endless loop or circle of interpretation

Understanding and meaning are based on subjective experience of how symbols are combined in activity

Each symbol is interpreted through its patterns/combinations in use, so the patterns are as important as the symbols

i.e. can't be broken down into independent, objective parts

Interaction between people leads to some consistencyPerceiving what they do, and them perceiving what you do, lets people

learn new patterns, resolve problems & inconsistencies

Interpretation at the Centre

‘Revolutions’ have already happened in linguistics, semiology, architecture, psychology, physics… but not CompSci (yet)Interpretation is the common human activity at the core of these advances

No one of their frameworks of interpretation is ‘true’ above othersWhen they talk about human activity, each must in some way be consistent with

the others

Philosophy is the field that tries to make this generalisation or combination of them allTo find what is consistent, solid, useful

Ubicomp and CSCW are the areas of CompSci that have done the most to ‘catch up’ with philosophy (so far)

Media Spaces

A media space usually centres on video technologyEach person has camera, monitor, microphone, speaker… and computer (e.g.

Fig. 1)Now often (narrowly) assumed to be webcams and videophones

Media spaces are the classic CSCW ‘awareness systems’Not primarily about direct, engaged communication

e.g. a phone call or a face-to-face meeting

More about supporting ‘background’ long term communicationWho’s around, who’s busy, what’s going on

Space v. Place

Why call them media ‘spaces’?Is there a geometric coordinate system? No.It’s just a metaphor: drawing on ideas/skills from familiar spatial

media e.g. the affordances of everyday vision, sound etc.

What are we designing? Not just spatial data or geometryInstead, aiming for ‘place’ with a feeling of locality or community

e.g. “not just a house, but a home”

Understanding, expectations and involvement with regard to what is appropriate activity and how to interpret activity

Potentially Invasive

Systems’ affordances for awareness may be uncertain or invasiveOthers are aware of activity you’d prefer them not to be aware of ⇒ invasion

of privacy Also, what others are aware of may be unclear or ambiguousDisembodiment: info about you is in a context you can’t see, so hard to

interpret its setting and effectDissociation: results of an action are visible to you, but the action and actor are

not

Design solutions generally based on control and feedback mechanisms

RAVE: a system based on service-based control & feedbackSometimes asymmetric, e.g. don’t notify me of every glance into my officeCombined several media, e.g. audio notification of glances

The RAVE System

Supporting ‘work over its entire range’Support for transitions between different degrees of

engagement, awareness and planning2x2 matrix of Figure 2

Example: RAVE buttons (Figure 3)Began with simple on/off connection controlEvolved to represent higher level activities and multiple degrees

of engagement

The Privacy/Awareness Tradeoff

Simplest policy: symmetrical or reciprocal awarenessIf I can see you then you can see me…but handling every connection is often distracting and

annoying

Asymmetric privacy/awareness mechanisms can be goodFit with social conventions, e.g. ‘glance in’ useful and not too

distractingStart with explicit reliance on existing/familiar social controls,

then gradually refine appropriate new tech controls

Offering Control and Feedback

Access control handled at service leveldifferent permissions for glance, vphone…not handled at the level of raw connections

User set access based on people as well as servicee.g. glance controls in figure 4

Services also used as basis for feedbackKhronika events gave feedback on use: pop-up text, verbal audio,

non-verbal audio...

e.g. 'open-door' sound just before someone glances in

Non-Verbal Audio

Less intrusive awareness of connectionDoes not require symmetric connectionsLess spatial attention and focus than text

Less distracting than verbal audioEasier to ‘shape’ sound to reduce annoyanceReuse of familiar sounds, e.g. of door hinges

Use a palette of sounds to fit with services

Awareness over Time

Khronika: an event notification serviceEveryday schedules, events & interactionsEvents in old media: meetings, conferences, visitors, talks, going

to the pub…Events in new media: email arriving, active badge movements,

load on servers…Used for feedback on A/V connections: audio, email, pop-up

window, speech…

Past & future events accessed via a calendarEvents could be explicitly entered, or set up via rulesNo past video/audio stored, only a brief calendar entry

Portholes

Part of RAVE’s use both locally and remotelyto overcome the informal interaction lost by people working far apartreaction to studies that showed importance of informal communication

Small, regularly updated images from public areas & officesless bandwidth than live video connections

Links to other tools: easy to escalate degree of engagementClick on an image: get menu of email and other RAVE commands

Written by Paul Dourish in one weekend, by the way

Portholes connected offices in the UK and USA. Image resolution was much deliberately lower than the original video sources. This had technical benefits, e.g. it could fit many images on screen, but also social ones, e.g. people felt the system was not too intrusive.

Evaluation of RAVEA ‘background’ tool can be hard to assess in a controlled 'lab' setting

Long startup time, harder to remember specific experiences of use, effects on work groups not well understood

Use sociological/ethnographic methods instead

Interviews and observations of use, especially of privacy v. awarenessDifferent reactions from different people, partly due to different roles

Long-term social benefits, e.g. a good sense of community, were at least as strong as benefits from short-term use as a tool

Patterns of use specific to the new media as well as specific to the old media of the settingUsing the edge of the field of view as a barrier, saying hello to

connected people when entering a room, pointing a camera out a window…

Affordances of Media Spaces

Designing potential forms of collaborationFirst assess in terms of physical phenomena, but then assess

in terms of social activity

Transmission of visual/auditory informationAffords remote collaboration… but how well?

Conversely, it also affords invasion of privacy

Find constraints on perception & interactionSome limits and constraints can be useful ⇒ privacy features

But others are best removed or changed ⇒ new designs

Affordances for Vision and Movement

Video provides a limited field of viewCameras view less of surroundings: limited peripheral vision and exploration

Suggests wide-angle lenses and/or multiple cameras

However, limited field of view lets people know where they can be ‘off screen’

Resolution of video far less than the eyeSuggests camera movement or zooming, or higher resolution video

However, limited resolution stops people seeing too much of letters, what’s on the desk, what’s on the computer screen…

Cameras/mikes static: can’t be moved by remote userPrevents exploration and inspection, prevents people orienting to each other

e.g. to start up conversation

Suggests remote control or multiple cameras, video mosaics, head tracking…

Careful, though: with static camera/mikes, people know where they can be seen or heard, and where they can’t ⇒ also need new control/feedback tools

Affordances for Predictable Interaction

Media spaces are often anisotropic: can see one way but not backMakes symmetry and prediction of behaviour hard

Example: cameras and monitors not on same axis

Interferes with gesture & gaze awareness, and also affords glances and spying

Led to ‘video tunnels’ and MIT’s ‘ClearBoard’ Cameras and displays seem to be aligned

Also led to behaviour tailored to anisotropic setupe.g. people deliberately looking into camera, not into the monitor

Privacy and Awareness

CompSci’s traditional focus on security is not enough Tech issues: awareness and access to data/resources

Personal issues: subjective preference, current context of activitySocial issues: cultural norms, acceptable behaviourOrganisational issues: company role/job, codes of practice, laws

Privacy and awareness are complementaryThe same in terms of tech resources, but opposites in personal/social terms

These are evolving, dynamic design issuesNot all of them are under the designer’s controlNevertheless, part of system design as much as GUI style or database table format

A Design Approach for Privacy and Awareness

Framework put forward by Bellotti and Sellen at Xerox EuroPARCDesign for Privacy in Ubiquitous Computing Environments paper

Grounded in experience of media spaces & ubicompBut applicable to most types of collaborative system

Useful for assessed exercise!

Use a matrix based on control and feedback of informationControl: empowering people to stipulate what information they project and who

can get hold of it

Feedback: informing people when and what information about them is being captured, and to whom the information is being made available

For each of these two, consider capture, construction, accessibility and purpose

Four Issues for Feedback and for Control

1. Capture: most important in designWhat kind of phenomena/symbols/media are recorded?

2. ConstructionHow is this information processed, or combined with other information?

3. AccessibilityAccessible to the public, particular groups/people, just me, or who?

4. PurposeTo what uses is the information put? How might it be used in the future?

Other features worth considering: trustworthiness, appropriate timing, perceptibility, unobtrusiveness, minimal intrusiveness, fail-safety, flexibility, low effort, meaningfulness, learnability, low cost…

Some Problems with RAVE

Long term vphone connections sometimes forgottenPeople forgot that the audio connection was on, or assumed that the

connection was a ‘background’ video-only link

Feedback of the current state was not easy to perceive, and needed more design work

Cameras in public areasFeedback monitors may not suffice, especially for visitors who have no

facilities for control over the RAVE equipment

Sometimes we highlighted or labelled the monitor, to say what it wasPrinted signs, setting up a mannequin to hold the camera...

The Feedback/Control Matrix in the Design Process

Set out existing privacy/awareness norms, practices and problems in the 2x4 matrixMark cells with weak or no privacy/awareness support

Note possible designs for marked cellsAssess new solution’s effect across matrix: are you reducing old problems

on one place while creating other new problems elsewhere?

Can you design in enough slack to support choice and evolution when put into use?Support the evolution of social patterns and of the technology

(Note: this matrix may be very useful in your assessed exercise.)

Summary

Affordances highlight users’ interpretation of systemsUnderstanding and use are individual, contextual and social Philosophy lets us better understand interpretation and use

Can also be used to analyse and create designs

RAVE as exemplar of a media spaceNew media interwoven with old, to support collaborative work

ranging from background awareness to f2f engagement

Feedback and control framework can help with problems of privacy, disembodiment and dissociation

Papers: Tech Affordances, RAVE and Design for Privacy