The Impact of Technical Domain Expertise on Search Behavior and Task Outcome
13
The Impact of Technical Domain Expertise on Search Behavior and Task Outcome Julia Kiseleva, Alejandro Montes García, Jaap Kamps, Nikita Spirin Eindhoven University of Technology University of Amsterdam University of Illinois at Urbana-Champaign WSDM Workshops ’16, San Francisco, USA
-
Upload
julia-kiseleva -
Category
Data & Analytics
-
view
334 -
download
0
Transcript of The Impact of Technical Domain Expertise on Search Behavior and Task Outcome
The Impact of Technical Domain Expertiseon Search Behavior and Task Outcome
Julia Kiseleva, Alejandro Montes García, Jaap Kamps, Nikita Spirin
Eindhoven University of TechnologyUniversity of Amsterdam
University of Illinois at Urbana-Champaign
WSDM Workshops ’16, San Francisco, USA
Developer
Motivation: Domain vs. Search Expertise
Not Developer
Developer
Motivation: Domain vs. Search Expertise
Not Developer
Search Expertise
Search Expertise
Developer
Motivation: Domain vs. Search Expertise
Not Developer
Search Expertise
Search Expertise
TechnicalExpertise
TechnicalExpertise
JavaDeveloper
Motivation: Domain vs. Search Expertise
Not Java Developer
JavaExpertise
JavaExpertise
JavaDeveloper
Motivation: Domain vs. Search Expertise
Not Java Developer
JavaExpertise
JavaExpertise
Expert Clicks are less Biased
• Main: How does technical domain expertise influence search behavior?
1. What is the impact of technical domain expertise on the search process?
2. What is the impact of technical domain expertise on search outcome?
Research Objectives
• Online Setup • Technical Expertise: Java and JavaScript• Collected Data:oDemographic data: age, gender education level;oSelf reported expertise in Java and JavaScriptoTest to measure expertise in Java and JavaScript
User Study Design
• 29 participants • Education: High school 8%, Bachelor 12%,
Master 56%, PhD 24%• Age: 18-23 years 16%, 24-29 years 36%, 30-35
years 28%, 36-42, years 16%, 43-48 years 4%
User Study Data
Position Bias vs Technical Expertise
Right Answer vs Technical Expertise
• The technical domain expert's biases was less pronounced and they tended to check the SERP's bottom
• Having a general programming expertise helped to derive the good answers on the SERP, but the experts with high proficiency managed to detect better answers as they dug them from the bottom of the SERP
Conclusions
