You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers &...
-
Upload
marianna-hunt -
Category
Documents
-
view
214 -
download
0
Transcript of You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers &...
![Page 1: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/1.jpg)
You Are Not Alone:How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers
Bjoern HartmannStanford HCI Lunch
8/19/2009
![Page 2: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/2.jpg)
The Idea (Not New)
• Record what users are doing while using an authoring tool. (At what level of detail? Privacy? Confidentiality?)
• Extract relevant patterns from these traces. (What patterns? Automatically or with user involvement?)
• Aggregate data from many users. (How? What is the right group boundary?)
• Present useful data back to either the users, or the developers. (What is useful? In what format? Feedback loop or canned answers?)
![Page 3: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/3.jpg)
![Page 4: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/4.jpg)
![Page 5: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/5.jpg)
Algorithms:Recommender
systems,Data mining, PL
Social Perspective:
Crowd sourcing,User communities
Domain:Authoring tools
![Page 6: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/6.jpg)
Potential benefits
• For users:– Gain expertise through tutorials (Grabler SIGGRAPH09) & tool
suggestions (Matejka UIST09)– Understand expert practices (2draw.net)– Improved documentation (Stylos VLHCC09)– Help with debugging
(Kim, SIGSOFT06; Livshits SIGSOFT05)• For tool developers & researchers:
– Understand user practices (Terry CHI08)– Understand program behavior in the wild
(Liblit PLDI05)– Understand usability problems in the wild (Hilbert 2000)
![Page 7: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/7.jpg)
INSTRUMENTING IMAGE MANIPULATION APPLICATIONS
![Page 8: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/8.jpg)
Example: 2draw.net
8
![Page 9: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/9.jpg)
Examining 2draw
• Record: canvas state over time• Extract: snapshots of drawing• Aggregate: no aggregation across users• Present: browse timeline of snapshots
• Benefit: understand technique behind drawings
![Page 10: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/10.jpg)
Terry et al., InGimp (CHI 2008)
http://www.ingimp.org/statsjam/index.php/Main_Page
![Page 11: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/11.jpg)
Examining InGimp
• Record: application state / command use• Extract: • Aggregate: send usage sessions to remote db• Present: usage statistics
• Benefit: understand aggregate user profiles
![Page 12: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/12.jpg)
Own Experiment: Instrumenting Processing
• Use Distributed Version Control System to record a new revision every time the user compiles/runs program
![Page 13: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/13.jpg)
Grabler et al., Photo Manipulation Tutorials (SIGGRAPH 09)
![Page 14: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/14.jpg)
![Page 15: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/15.jpg)
Examining PMT
• Record: application state / command use / screenshots
• Extract: high-level commands• Aggregate: ---• Present: graphical, annotated tutorial
• Benefit: higher quality, lower cost tutorials
![Page 16: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/16.jpg)
CommunityCommands (Matjeka, UIST09)
![Page 17: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/17.jpg)
IMPROVED DOCUMENTATION
![Page 18: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/18.jpg)
Stylos, Jadeite (VL/HCC 2009)
![Page 19: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/19.jpg)
![Page 20: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/20.jpg)
![Page 21: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/21.jpg)
Documentation Algorithm
• For each file in source code corpus of Processing projects (existing documentation, forum posts, web search), calculate # of function calls for all known API functions (use hash table fn_name->count)
• Rescale font size on documentation page by relative frequency of occurrence in corpus
![Page 22: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/22.jpg)
DEBUGGING
![Page 23: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/23.jpg)
Cooperative Bug Isolation (Liblit, UCB)
23
![Page 24: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/24.jpg)
Examining CBI
• Record: sparse sampling of application state• Extract: ---• Aggregate: establish correspondence between
different reports• Present: priority list of runtime bugs to
developer
• Benefit: understand real defects in released software
![Page 25: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/25.jpg)
BugMem (Kim, UCSC)
![Page 26: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/26.jpg)
Examining BugMem
• Record: --- (use existing source code repository)• Extract: bug signature and fixes• Aggregate: ?• Present: list of bugs in repository that match
fixes in same repository
• Benefit: find bugs in existing code that your team has fixed in the past
![Page 27: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/27.jpg)
DynaMine (Livshits @ Stanford)
![Page 28: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/28.jpg)
‘;l’;l
![Page 29: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/29.jpg)
![Page 30: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/30.jpg)
![Page 31: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/31.jpg)
Examining HelpMeOut
• Record: source code at every compilation step• Extract: error messages and code diffs• Aggregate: collect fixes from many users in db;
explanations from experts• Present: list of fixes in db that match user’s error
and code context; explanations when available
• Benefit: find fixes that others have used to correct similar problems in the past
![Page 32: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/32.jpg)
A Design Space for Finding Answers to Questions from Online Data
How many answersare needed?
When are answersavailable?
Immediately(Already published)
Near real-time
With Latency
1 10 100
Who publishesanswers?
Authority Expert Peer
What reporting format?
Individual answers Aggregate data
Can questionerseek clarification/detail?
How many answersare shown / available? 1 10 100
How was answer authored? Explicitly Implicitly
Yes No
Anyone?
![Page 33: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/33.jpg)
HelpMeOut
How many answersare needed?
When are answersavailable?
Immediately(Already published)
Near real-time
With Latency
1 10 100
Who publishesanswers?
Authority Expert Peer
What reporting format?
Individual answers Aggregate data
Can questionerseek clarification/detail?
How many answersare shown / available? 1 10 100
How was answer authored? Explicitly Implicitly
Yes No
Anyone?
![Page 34: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/34.jpg)
Stack Overflow
How many answersare needed?
When are answersavailable?
Immediately(Already published)
Near real-time
With Latency
1 10 100
Who publishesanswers?
Authority Expert Peer
What reporting format?
Individual answers Aggregate data
Can questionerseek clarification/detail?
How many answersare shown / available? 1 10 100
How was answer authored? Explicitly Implicitly
Yes No
Anyone?
![Page 35: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/35.jpg)
Non-Example: Scratch (MIT)
35
Scratch authoring environment with “Share” button
Scratch web site lists shared projects
![Page 36: You Are Not Alone: How Authoring Tools Can Leverage Activity Traces to Help Users, Developers & Researchers Bjoern Hartmann Stanford HCI Lunch 8/19/2009.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f305503460f94c4baf1/html5/thumbnails/36.jpg)