Coast to Coast March 2013

48

description

General talk telecast to universities across Canada. A bit more of a focus on mixed-initiative systems than some earlier talks

Transcript of Coast to Coast March 2013

Page 1: Coast to Coast March 2013
Page 2: Coast to Coast March 2013

Visual Analytics as a Cognitive Science

Brian Fisher

SFU School of Interactive Arts & Technology and Program in Cognitive Science

UBC Media & Graphics Interdisciplinary Centre (MAGIC)

Page 3: Coast to Coast March 2013

• In 2011, data expected to be about 1.8 zettabytes (1021).

• In 2013, Internet traffic to reach 667 exabytes (1018)/yr.

• Comparison: US Library of Congress is ~10 petabytes (1015).

© 2011 VIVA All rights reserved.

“Big Data: Volume, Velocity & Variety

• “By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.” (McKinsey Global Institute 2011)

Page 4: Coast to Coast March 2013

Challenges for model-based approaches

• “3 Vs” challenge• Relevance, validity, reliability of data uncertain• Insufficient time to reach solution

• Model challenges• Multiple models to chose from• Assumptions may or may not hold

• “Wicked problems” challenge (Rittel)• Lack criteria to evaluate solution• Each problem is unique (no population)• Problem is not understood until solution is found

The Information needed to understand the problem depends upon one’s idea for solving it. -- Rittel & Webber 1973

Page 5: Coast to Coast March 2013

Visualization history• NSF: Visualization in

Scientific Computing: McCormick, DeFanti, & Brown, Computer Graphics, Nov. 1987

• IEEE Visualization conference 1990

“The purpose of [scientific] computing is insight, not numbers.” Richard Hamming

“Visualization is a method of computing.” Authors of report

Page 6: Coast to Coast March 2013

Information visualization • 1990 Conference

on diagrammatic reasoning

• 1995 InfoVis Conference• “Information

Visualization: Wings for the Mind” Keynote by Stuart Card

Page 7: Coast to Coast March 2013

Stuart Card’s view

• Increase the memory & processing resources available to users

•Reduce the search for information by using visual representations to enhance the detection of patterns

•Engage perceptual inference operations•Use perceptual attention mechanisms for

monitoring•Support manipulation of information

Page 8: Coast to Coast March 2013

Human/Computer cognitive system

•Mixed-initiative system composed of•Computer: Math, logic, machine learning etc. •Human: Perception, cognition, & collaboration

•Need an interactive visual interface that can effectively blend the two•Computer: steerable algorithms•Human: complementary actions

* Meyer J., Thomas, J., Diehl, S., Fisher, B., Keim, D., Laidlaw, D. Miksch S., Mueller, K. Ribarsky, W., Preim, B., & Ynnerman, A. (2010) From Visualization to Visually Enabled Reasoning. In “Scientific Visualization: Advanced Concepts”. vol. 1 pp. 227-245. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany. I978-3-939897-19-4

Page 9: Coast to Coast March 2013

http://www.visual-literacy.org/periodic_table/periodic_table.html

Page 10: Coast to Coast March 2013

On the Death of Visualization (Lorensen 2004)

Can It Survive Without Customers?

• Visualization, alone, is not a solution.

• Visualization is a critical part of many applications.

• Visualization, the Community, lacks application domain knowledge.

• Visualization has become a commodity.

• Visualization is not having an impact in applications.

Page 11: Coast to Coast March 2013

Visual analytics origins• National Visualization & Analytics Center (NVAC)

• Battelle/PNNL 2004 R&D Agenda panel• University: Brown, GMU, Georgia Tech, OSU, Penn State, Purdue, SFU ,

Stanford, UC, UI, UM, UNC, UU, WPI

• Industry: Boeing, Microsoft, PARC, Sandia Labs

• Gov: CIA, DHS, FBI, NIST, NSA, unspecified

– Industry Consortium – Regional Visualization Centres – Centre of Excellence

•Ccicada (Rutgers DIMACS)•VACCINE (Purdue et al)

Page 12: Coast to Coast March 2013

“This science must be built on integrated perceptual and cognitive theories that embrace the dynamic interaction between cognition, perception, and action. It must provide insight on fundamental cognitive concepts such as attention and memory. It must build basic knowledge about the psychological foundations of concepts such as ‘meaning,’ ‘flow,’ ‘confidence,’ and ‘abstraction.’ “

“Illuminating the Path” (IEEE Press)

“The science of analytical reasoning facilitated by interactive visual interfaces”

Visual analytics

Page 13: Coast to Coast March 2013

How are VA Information systems different?

• Development based on understanding of expert cognition in situ• Informed by current cognitive & social science• Engagement with community of experts • Emergent cognitive science of expert reasoning

• Obvious support for analytical processes-- collaboration and interaction as well as observation• Graphical analog for analytic processes• Support “Human-information discourse”• Integrated across roles in the community

Page 14: Coast to Coast March 2013
Page 15: Coast to Coast March 2013

Visual analytics disciplines • Statistics, data representation and statistical graphics• Geospatial and Temporal Sciences• Applied Mathematics• Knowledge representation, management and

discovery• Ontology, semantics, Natural Language Processing, extraction,

synthesis, …

• Cognitive and Perceptual Sciences• Communication: Capture, Illustrate and present a

message• Decision sciences• Information and Scientific Visualization

Jim Thomas slide

Page 16: Coast to Coast March 2013
Page 17: Coast to Coast March 2013
Page 18: Coast to Coast March 2013

Cognitive &PerceptualSciences

VisualInformationSystems

Graphic & Interaction Design

Mathematical & Statistical Methods

EffectiveSituated

R&D

Page 19: Coast to Coast March 2013

http://www.vacommunity.org/HomePage

Page 20: Coast to Coast March 2013

My claims

• VA becomes a science when its disciplines develop scientific methods that advance the goals of visually-enabled reasoning

• VA itself must build translational research methods, an application-focused “trade language”

Fisher, B., Green, T.M., Arias-Hernández, R. (2011) "Visual analytics as a translational cognitive science," Topics in Cognitive Science 3,3 609–625.

Page 21: Coast to Coast March 2013

What kind of science?• Natural science approach to reasoning

• Experimental psychology, decision sciences

• Social science approach to coordination • Distributed cognition, social psychology

• Focus both on phenomena associated with rich interaction with visual information-- a “cyberpsychology”

• Add technology bits as needed

Fisher, B., Green, T.M., Arias-Hernández, R. (2011) "Visual analytics as a translational cognitive science," Topics in Cognitive Science 3,3 609–625.

Page 22: Coast to Coast March 2013

"Il n'existe pas une catégorie de sciences auxquelles on puisse donner le nom de sciences appliquées. Il y a la science et les applications de la science, liées entre elles comme le fruit à l'arbre qui l'a porté"

Louis Pasteur

Pure Basic Research(Bohr)

Use-inspired Basic Research (Pasteur)

Sampling,Description,Taxonomy(Audubon)

Pure Applied Research (Edison)

Quest for Fundamental Understanding?

No

Yes

Consideration of Use ?

No

Yes

(1822–95)

Pasteur’s Quadrant

(Stokes)

Page 23: Coast to Coast March 2013

The study of thought, learning, and mental organization, which draws on

aspects of psychology, linguistics, philosophy, and computer modelling.

(OED)

Cognitive Science Society founded in 1979

Cognitive Science

Page 24: Coast to Coast March 2013

• Daniel Bobrow - AI• Eugene Charniak - AI• Allan Collins - Psychology• Edward Feigenbaum - AI• Charles Fillmore - Linguistics• Jerry Fodor - Philosophy• Walter Kintsch - Psychology• Donald Norman - Psychology• Zenon Pylyshyn - Psychology• Raj Reddy - AI• Eleanor Rosch - Psychology• Roger Schank - AI

Cognitive science origins

Page 25: Coast to Coast March 2013

Key figures not at DallasAI Linguistics Neurosci Philosophy Psychology

Chomsky

Miller

Minsky

Newell

Simon

✓ ✓

✓ ✓ ✓ ✓

✓ ✓ ✓

✓ ✓

✓ ✓ ✓

Page 26: Coast to Coast March 2013

Cogsci society logo

Page 27: Coast to Coast March 2013

• A division of labour?•Simon’s“nearly decomposable problems”

• Conceptual & methodological “trading zone”?•Image and Logic: a Material Culture of Microphysics (Galison)

•Trading Zones in Cognitive Science (Thagard)

• A translational science of cognition?

What happens in the maze?

Page 28: Coast to Coast March 2013

Translational cognitive science: Distributed cognition

• Expert perception and projection • Images may be static

• Active engagement with artifact• Objects are responsive, not autonomous• Complementary action (Kirsh)• Soft constraints (Gray)

• Agent-agent interaction• Socially-distributed cognition (Hutchins)• Mixed-initiative interaction

Page 29: Coast to Coast March 2013

Claims• Since analytical reasoning is cognition, a

science of analytical reasoning ought to be a cognitive science

• Current cognitive science (psychology, visions research, social science) research methods and foci are useful but not sufficient for a visual analytics

• The multidisciplinary field approach taken by the Cognitive Science Society is a useful example for visual analytics

Page 30: Coast to Coast March 2013

Core problem for VA science• Visual analytics is not a natural kind

• We design environments and their characteristics• We adapt to those environments

• While psychology can concentrate on typical behaviour, VA will be a moving target-- its capabilities may well change in response to the environments we create• e.g. recalibration by pairing-- statistical regularities

of the natural environment determine how sensory channels recalibrate each other

Page 31: Coast to Coast March 2013

• Cognitive architecture from psychology• Extend to expert human performance

• Cognitive expertise• Visual expertise• Visuomotor expertise• Multimodality & modularity

• Test human capabilities in dynamic display environments

Distributed cognition in air traffic control

Page 32: Coast to Coast March 2013

Air traffic control • NextGen ATC

“fishtank” projection • Change camera

position for better view

• How will global motion affect tracking?

Liu, G. Austen, E. L., Booth, K.S. Fisher, B., Argue, R. Rempel, M.I., & Enns, J. (2005) Multiple Object Tracking Is Based On Scene, Not Retinal, Coordinates. Journal of Experimental Psychology: Human Perception and Performance. 31(2), Apr 2005, 235-247.

http://www.youtube.com/watch?v=tKJVB4id_TY

with John Dill

Page 33: Coast to Coast March 2013

FINST theory of spatial indexing

Page 34: Coast to Coast March 2013

Multiple object tracking (Pylyshyn)

Page 35: Coast to Coast March 2013

3-D Projected display

Page 36: Coast to Coast March 2013

Test at different speeds

Page 37: Coast to Coast March 2013

Fit human tracking function

Page 38: Coast to Coast March 2013

... Then add display motion

Page 39: Coast to Coast March 2013

Tracking vs object speed

Page 40: Coast to Coast March 2013

Tracking in warped space

Page 41: Coast to Coast March 2013

Tracking in warped space

Page 42: Coast to Coast March 2013

Conclusion: Humans track in allocentric space

• Retinal speed of targets does not determine performance

• Motion of targets relative to each other does

• But only if motion preserves good metric characteristics of space

• Explanation is at the level of a human - display cognitive system

Page 43: Coast to Coast March 2013

CognitionPerceptual

ScienceMethods

SocialScienceMethods

Computation and

Visualization Methods

Graphic & Interaction

DesignMethods

Visual analysis

Page 44: Coast to Coast March 2013

• Base on cognitive architecture & D-Cog perspectives

• Focus on specific situations: •Cognition supported by interactive visualization•Socially distributed cognition via technology

• Incorporate these in •Design & evaluation of technologies•Selection, training, organizational change

Visual analytics methods

Page 45: Coast to Coast March 2013

• “Pair analytics” sessions•Student visual analyst &

trained domain expert collaborate on analytic task

•Student “drives”, expert “navigates”

•Video session & capture screen

Socially distributed cognition

Arias-Hernandez, R, Kaastra, L.T., and Fisher, B. (2011) Joint Action Theory and Pair Analytics: In-vivo Studies of Cognition and Social Interaction in Collaborative Visual Analytics. In L. Carlson, C. Hoelscher, and T. Shipley (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 3244-3249). Austin TX: Cognitive Science Society.

Page 46: Coast to Coast March 2013

SCIENCElab research• Emergency Management (NSERC, DHS)

• Mobile analytics / sensor analytics• “Virtual EOC” visual analytic environment

• Aircraft Safety, Reliability (Boeing/MITACS)• “Pair analytics” of complex quant and text data

• Economics and finance (MITACS, NSF)• Behavioural economics (portfolios)

• Healthcare Monitoring & Management (DHS)• Complex data in health research (CFRI)• Public health monitoring & management (BC Injury

Research and Prevention Unit)

Page 47: Coast to Coast March 2013

SCIENCElab• Dr Richard Arias-

Hernández• Dr. Nathalie

Prevost• Dr. Linda Kaastra• Samar Al-Hajj• Nadya Calderón

• Tera Marie Green• Sabrina Hauser• Ali Khalili• Barry Po• Aaron Smith• Andrew Wade

Page 48: Coast to Coast March 2013

Decision Analytics, Mobile Services and Service Science

Visualization and Analytics for Decision Support, Operational Management, and

Scientific Discovery Submissions due June 15, 2013http://www.hicss.hawaii.edu/hicss_47/