The Impact of Technical Domain Expertiseon Search Behavior and Task Outcome

Julia Kiseleva, Alejandro Montes García, Jaap Kamps, Nikita Spirin

Eindhoven University of TechnologyUniversity of Amsterdam

University of Illinois at Urbana-Champaign

WSDM Workshops ’16, San Francisco, USA

Developer

Motivation: Domain vs. Search Expertise

Not Developer

Developer

Motivation: Domain vs. Search Expertise

Not Developer

Search Expertise

Search Expertise

Developer

Motivation: Domain vs. Search Expertise

Not Developer

Search Expertise

Search Expertise

TechnicalExpertise

TechnicalExpertise

JavaDeveloper

Motivation: Domain vs. Search Expertise

Not Java Developer

JavaExpertise

JavaExpertise

JavaDeveloper

Motivation: Domain vs. Search Expertise

Not Java Developer

JavaExpertise

JavaExpertise

Expert Clicks are less Biased

• Main: How does technical domain expertise influence search behavior?

1. What is the impact of technical domain expertise on the search process?

2. What is the impact of technical domain expertise on search outcome?

Research Objectives

• Online Setup • Technical Expertise: Java and JavaScript• Collected Data:oDemographic data: age, gender education level;oSelf reported expertise in Java and JavaScriptoTest to measure expertise in Java and JavaScript

User Study Design

• 29 participants • Education: High school 8%, Bachelor 12%,

Master 56%, PhD 24%• Age: 18-23 years 16%, 24-29 years 36%, 30-35

years 28%, 36-42, years 16%, 43-48 years 4%

User Study Data

Position Bias vs Technical Expertise

Right Answer vs Technical Expertise

• The technical domain expert's biases was less pronounced and they tended to check the SERP's bottom

• Having a general programming expertise helped to derive the good answers on the SERP, but the experts with high proficiency managed to detect better answers as they dug them from the bottom of the SERP

Conclusions