IVITCMDtalk

Cognitive Approaches for Studying Information

Visualization and Interaction

Fisher and Dill

IVITCMDApril 22, 2006

Background Molecular Biology, Medical Biophysics,

Hiram/CWRU Cognitive Neuroscience, Experimental Psychology,

UCSC Human-Computer Interaction

Inst. for Robotics & Intelligent Systems, UWOCognitive Science, RutgersSFU School Interactive Arts and TechnologyUBC Media And Graphics Interdisciplinary

Centre


Design Approach Human-centred Theoretically motivated Grounded in practice of design Emphasize design process

Evolve human sciences to interaction scienceFramework for performance analysisTaxonomy of interaction

Integrate with (software) engineering models


Pasteur’s Quadrant (Stokes)

MAGIC


Research approach Decompose task based on cognitive

architecture Identify interaction challenges to cognitive

processing modules Devise “toy world” test of that module’s

robustness to challengeUse-inspired basic research Research interaction, not the mind

Model data for quantitative prediction


Air traffic control research

Free Flight ATC “fishtank” projection

Change camera position for better view

How will global motion affect tracking?


Multiple object tracking (Pylyshyn)

QuickTime™ and aAnimation decompressor

are needed to see this picture.


3-D Projected display

QuickTime™ and a3ivx D4 4.5.1 decompressor





Test performance at different speeds...


Fit human tracking function (Lui)


... Then add display motion




Tracking vs object speed


Tracking vs 2 object speeds


Tracking in warped space


Recap

Pasteur: Basic research on fermentation will inform applied work on disease (and beer)

We think:Basic research on tracking will give us design

guidelines for fishtank ATC displaysShould generalize to other systemsModeling leads to quantitative prediction and

customization for “personal equations” This is not psychology


Design space explosion

Buxton/Kasik


As interactive technology becomes immersive,

multimodal, and ubiquitous

The emphasis in interaction design shifts to facilitating perceptual cognition and

embodied interaction


Interaction science & design

Paul EhrlichKnew chemistry, but not biochemUsed science to reduce design space to 900 Had animal model for testingNumber 606 = SalvarsanAdvance science and treatment

TodayMedical science is distributed cognition Clinicians, physiologists biochem...Many will never be first author


Interaction design future?

Design for key sensory & motor

systems

Assess specific aspects of interaction

Walkthrough or experiment

Implement prototype

Interaction Science


Implementations Perception & attention in dynamic workspaces

Air traffic control (HRL, NSERC), Multimodal driving interfaces (Nissan)

Cognitive & collaborative aspects of tasksCZWeb/CZTalk Laser pointers, cell phones + situated displays (NTT,

Panasonic)CSCL video, attention management/privacy(CANARIE,

EU Kaleidoscope)

Space constancy in VRWayfinding in CAD worlds (Boeing)Display customization for perceptual abilities (GM)


Acknowledgements Profs

Jim Enns (UBC) Sid Fels (UBC) Ron Rensink (UBC)

Grads Colin Swindels Ritchie Argue Erin Austin Geniva Lui Alexander Stevenson

• Postdocs/Staff– Jason Harrison (UBC)– Barry Po

• Grants & partners– Nissan– Boeing– GM– HRL– NSERC– IRIS– MAGIC Endowment Fund


Spatial Cognition in complex environments

Multiple events at different locations Multimodal events (sight, sound, touch) Action in space Tools

FINSTs and indexical cognitionSpace Constancy

Functional space constancy: Ability to interact directly

Apparent space constancy: Ability to make judgments

Sensory integration (FLMP, calibration by pairing)


Communicative Pragmatics & Groupware Psycholinguistic pragmatics theories and

methods extended to interactive environments Support for familiar F2F metachannels

Embodied communication (gesture, facial expression, body language

Prosody

Interaction methods to support underlying processes Acknowledgement and repair mechanismsAdvancementLayering


Personalization Personal Equation: Individual differences

in perceptual, indexical attentive, and cognitive processesTest usersModel dataDescribe individual differences in model

parameters Customize display for PEI Attentive systems adapt PEI

Within a session: fatigue, attentivenessBetween sessions: aging changes


Other work

Combine qualitative and mathematical techniques

Highly skilled interaction

Study control and communication

QuickTime™ and aH.263 decompressor



Grounded theory

Open codingThemes arise from the data, phenomena are

observed Axial coding

Relations are drawn between themes and observations in the data

TriangulationMixed methods to test theories


Add HCI aspects/analyses

Replace flutes with analog synth Allows us to capture

Control: Breath and fingeringCommunication: Gestural motion

Can add HCI variables such as lag, bias, noise etc. to understand how skilled performers compensate

May feedback to music controller design


Mapping to Visual analytics

Build toolset: “Thick Description” + Math models Understand “gold standard” for interaction

Technology to support expertise - “virtuosity” How information from multiple senses is integrated “Tight loop” Perception/action patterns, sequences,

rhythmsDevelopment of coordination between users

IVITCMDtalk

Documents

Transcript of IVITCMDtalk