Skills, Reputation, and Search

Pete SkomorochPrincipal Data Scientist, LinkedIn

2

Vision: Create Economic Opportunity for Every Professional

TimeLocation

©2012 LinkedIn Corporation. All Rights Reserved.

3

LinkedIn: The Professional Profile of Record

200+MMembers 200M MemberProfiles

4

LinkedIn Search: Connecting Talent with Opportunity

5

Skills Correlated with the Job Title “Data Scientist”

6

Skills Related to “Big Data”

7

Information Retrieval

8

Soul Retrieval

9

10

Lucene on LinkedIn

11

Lucene Endorsement Graph

12

Solr on LinkedIn

13

Solr Endorsement Graph

14

Reputation: Building the Endorsement Graph

15

Viral Growth: 1 Billion Endorsements in 5 Months

16

How Did We Gather this Data?

1. Desire + Social Proof

2. Viral Loops + Network Effects

3. Data Foundation + Recommendation Algorithms

17

1) Desire & Social Proof

A endorses

B

B notified

B “accepts” endorsement

B endorses

C

B endorses

D

Endorsement recommendations

Email NotificationNews Feed2) Viral Loops & Network Effects

19

3) Data Foundation: Skills & Suggested Skills

20

Data Foundation: LinkedIn Skills

Social Tagging Accelerates Adoption

Suggested endorsements

Skill recommendations

Skill marketing

©2012 LinkedIn Cororation. All Rights Reserved.

Virality only

22

Outline

Skill discovery

Skill tagging

Skill recommendations

Suggested endorsements

23

Skill Discovery: Unsupervised Topics from Profiles

Extract

24

Topic Clustering & Phrase Sense Disambiguation

25

Deduplication Signals from Mechanical Turk

26

Sample Task for Mechanical Turk Workers

27

Skill Phrase Deduplication

28

Outline

Skill discovery

Skill tagging

Skill recommendations

Suggested endorsements

29

Lead designer and engineer for the implementation of a user-centric, fully-configurable UI for data aggregation and reporting.Developed over 20 SaaS custom applications using Python, Javascript and RoR.

Tagging Skill Phrases

Tagging: Extract potential skill phrases from text

Standardize unambiguous phrase variants

JavaScript RoR SaaS Python

ror

rubyonrails

ruby on rails development

ruby rails

ruby on rail

Ruby on Rails

Document (ex: Profile)

Tokenization

Skills Tagger

Phrases

(up to 6 words)

Skills Classifier

Skills

(unordered)

Skills

(ranked by relevance)

30

Outline

Skill discovery

Skill tagging

Skill recommendations

Suggested endorsements

31

Skill Inference

How suggested/inferred skills work:

– The skill likelihood is a conditional model

– Probabilities are combined using a Naïve Bayes Classifier

If you are an engineer at Apple, you probably know about iPhone Development.

Profile

Extract attributes

- Company ID

- Title ID

- Groups ID

- Industry ID

- …

Skills Classifier

Skills

(ranked by likelihood)

Feature

Vectors

37

Skill Recommendations for Your LinkedIn Profile

49% Conversion

4% Conversion

38

Outline

Skill discovery

Skill tagging

Skill recommendations

Suggested endorsements

39

Social Tagging via Skill Endorsements

Social Tagging Accelerates Adoption

Skill endorsements

Skill recommendations

Skill marketing

©2012 LinkedIn Cororation. All Rights Reserved.

41

Data Amplifies Desire

1. Desire + Social Proof

2. Viral Loops + Network Effects

3. Data Catalyst + Recommendation Algorithms

42

Over 58 Million Profiles are now Tagged with Skills

43

All This Data Flows Back Into Our Lucene Index

44

Helping us Connect Talent & Opportunity

TimeLocation

©2012 LinkedIn Corporation. All Rights Reserved.

45

Questions?

We’re hiring: data.linkedin.com

@peteskomoroch

CONTACTPete Skomoroch@peteskomoroch

http://data.linkedin.com