Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis
-
Upload
aruna-balakrishnan -
Category
Documents
-
view
280 -
download
0
Transcript of Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis
![Page 1: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/1.jpg)
Pitfalls of Information Access with Visualizationsin Remote Collaborative Analysis
Aruna D. Balakrishnan, Susan R. Fussell, Sara Kiesler & Aniket KitturHuman Computer Interaction InstituteCarnegie Mellon UniversityCarnegie Mellon University
February 10, 2010
![Page 2: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/2.jpg)
![Page 3: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/3.jpg)
Synchronous Remote Collaborations
di ib d dH d fi distributed datainformation overload
How do you figure out what’s important?
3
![Page 4: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/4.jpg)
Visualizations
textual & numerical data
visualizations
4
visualizations
![Page 5: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/5.jpg)
Interactive Social Network Diagrams
Larkin & Simon 1987Sparrow1998Viégas, Wattenberg, & Dave 2004
5
g , g,Balakrishnan, Fussell, & Kiesler 2008
![Page 6: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/6.jpg)
Shared Information
BenefitsEliminates need f r data e chan eEliminates need for data exchangeFocus on understanding & analysis of materialsSh d l d l Shared mental model (Fraidin 2004; Blockeel & Moyle 2002; Kozlowski, Ilgen & Klimoski 2006)
DisadvantagesC fi i bi Confirmation bias (Mojzisch & Schulz-Hardt 2005)
6
![Page 7: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/7.jpg)
Distributed Information
BenefitsIncreased disc ssi n d e t f rced inf e chan eIncreased discussion due to forced info exchangeMultiple perspectives
DisadvantagesForced information exchange
7
![Page 8: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/8.jpg)
Hypotheses
Hypothesis 1Pairs of analysts will perform better when each partner has partial Pairs of analysts will perform better when each partner has partial
data than when each partner has all the data.
Hypothesis 2 Pairs of analysts with a visualization tool will outperform analysts
without a visualization tool.
H h i 3Hypothesis 3Visualizations will benefit collaborative analysis more when each
partner has partial data than when each partner has all the datapartner has partial data than when each partner has all the data.
8
![Page 9: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/9.jpg)
![Page 10: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/10.jpg)
Experiment Design
Detective mystery experimental paradigm Pairs of participants investigated multiple crimes (Scupelli et. al, 2005)p p g p ( p , )
Shared task To find a serial killer responsible for several of these crimesTo find a serial killer responsible for several of these crimes
Distributed evidence E id f i l kill di t ib t d 7 Evidence for serial killer distributed among 7 casesOnly 5 of 7 cases had relevant information to the task
R ll b iRemote collaborationSeated apart to simulate distributed workAccess to Instant MessengerAccess to Instant Messenger
10
![Page 11: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/11.jpg)
Experiment Task
11
![Page 12: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/12.jpg)
Experiment Design
Between-subjects
2 x 3 factorial designPairs randomly assigned to
1 of 2 information access conditions1 of 3 visualization conditions1 of 3 visualization conditions
12
![Page 13: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/13.jpg)
Experimental ConditionsVisualization Conditions
Shared VisUnshared VisNo VisInfo Access Conditions
Split Data
N = 15 N = 15 N = 15
All All Data N = 15N = 15 N = 15
13
![Page 14: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/14.jpg)
![Page 15: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/15.jpg)
No Visualization
Spreadsheet Case file
IM
15
IM
![Page 16: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/16.jpg)
Unshared Visualization
Visualization Case file
IM
16
IM
![Page 17: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/17.jpg)
Shared Visualization
Case fileVisualization
IM
17
IM
![Page 18: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/18.jpg)
Experiment Details
180 ParticipantsAge range from 18 64 yearsAge range from 18 – 64 yearsFluency in English required
Procedure30 minutes
Informed consentTrained in detective work Trained in visualization useTrained in visualization use
60 minutesSerial Killer task
18
![Page 19: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/19.jpg)
Dependent Measures
S l ti d d T k P f Solutions recorded in final reports
Task PerformanceSolve serial killer task
C mm nicati nInstant messenger
logs
CommunicationAmount of communicationTypes of communication gypes o co u cat o(intercoder reliability kappa = .71)
19
![Page 20: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/20.jpg)
![Page 21: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/21.jpg)
Results: Average number of IM words
500
600 Split DataAll Data
400
500 All Data
Average # IM
200
300# IM words
per
0
100
N U h d Sh d
person
None Unshared Shared
( 0 0 )
21
Visualization Condition(p < 0.01)
![Page 22: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/22.jpg)
Results: Split Data Pair Solve Rate
100%Split Data
Percentof Pairs
80%
60%of Pairs Solving
the Case40%
N U h d Sh d0%
20%
None Unshared Shared
22
Visualization Condition
![Page 23: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/23.jpg)
Results: All Data Pair Solve Rate
100%Split DataAll Data
Percentof Pairs
80%
60%
All Data
of Pairs Solving
the Case40%
N U h d Sh d0%
20%
None Unshared Shared
23
Visualization Condition
![Page 24: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/24.jpg)
Results: Communication
# Hypotheses SharedDiscussion of a new hypothesis regarding the serial killer case
I think these four blunt instrument victims areinstrument victims areconnected.
I feel like it is a suspicious man on the b sbus.
24
![Page 25: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/25.jpg)
Results: Hypotheses Discussion
4.0Split DataAll Data
3.0
hese
s se
d rs
on
All Data
1 0
2.0
Hyp
oth
disc
uspe
r pe
r
0.0
1.0
N U h d Sh dNone Unshared Shared
25
Visualization Condition
![Page 26: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/26.jpg)
Results: Correct and Incorrect Solutions
Solved100%
Split All Split All Split All
Incorrect SuspectIndividual
solution
80%
60%
Suspected pattern
solution rates by category
40%
Did not solve
N U h d Sh d0%
20%
None Unshared Shared
26
Visualization Condition
![Page 27: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/27.jpg)
Results: Correct and Incorrect Solutions
Solved100%
Split All Split All Split All
Incorrect SuspectIndividual
solution
80%
60%
Suspected pattern
solution rates by category
40%
Did not solve
N U h d Sh d0%
20%
None Unshared Shared
27
Visualization Condition
![Page 28: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/28.jpg)
Results: Correct and Incorrect Solutions
Solved100%
Split All Split All Split All
confirmation bias
Incorrect SuspectIndividual
solution
80%
60%
Suspected pattern
solution rates by category
40%
Did not solve
N U h d Sh d0%
20%
None Unshared Shared
28
Visualization Condition
![Page 29: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/29.jpg)
Results: Correct and Incorrect Solutions
Solved100%
Split All Split All Split All
Incorrect SuspectIndividual
solution
80%
60%
Suspected pattern
solution rates by category
40%
Did not solve
N U h d Sh d0%
20%
None Unshared Shared
29
Visualization Condition
![Page 30: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/30.jpg)
Results: Summary
Hypothesis 1 : ConfirmedPairs of analysts discussed more and performed better when each Pairs of analysts discussed more and performed better when each
partner had half the data than when each partner had all data.
Hypothesis 2 : Partially ConfirmedPairs of analysts with a visualization tool outperformed analysts y p y
without a visualization tool but only if they had split data.
Hypothesis 3: ConfirmedVisualizations benefited collaborative analysis more when each
h d l d h h h h d ll dpartner had split data than when each partner had all data.
30
![Page 31: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/31.jpg)
![Page 32: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/32.jpg)
Design Implications
32
![Page 33: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/33.jpg)
Sharing Traces
Gutwin 2002
33
Gutwin 2002Cleveland, W.S. & Schmieg, G.M. 1987
![Page 34: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/34.jpg)
How Do We Improve Solve Rates?
100%
Percent
80%
60%of Pairs
Solving the Case
60%
40%60%
Case
0%
20% 27%
Split Data All Data0%
Shared Visualization Condition
34
Shared Visualization Condition
![Page 35: Pitfalls of Information Access with Visualizations in Remote Collaborative Analysis](https://reader034.fdocuments.in/reader034/viewer/2022051015/5565e536d8b42a20638b4839/html5/thumbnails/35.jpg)
Thank You!
National Science Foundation Grant IIS-0325047La ren GieseLauren GieseyMeghan SharmaP l R b iPaul RubritzGail KusbitPeter Scupelli
Image AttributionsChalk body: Johnny Vulcan http://flickr.com/photos/johnnyvulkan/328482660/ Footprint: · ª н м · http://flickr.com/photos/hashir/420913764/Hammer: ZoofyTheJinx http://flickr.com/photos/zoofythejinx/255395476/y J p p y jHandcuffs: pingnews.com http://flickr.com/photos/pingnews/154771251/
35