1TIM

Ta Nha Linh

13 March 2009

Harvesting useful information on researchers' home pages

Ta Nha Linh

Supervisor: Asst. Prof. Min-Yen Kan

2TIM

Ta Nha Linh

13 March 2009

Outline

• Motivation

• Challenges

• Researchers Information COllector (RICO)

• Contributions

• Future Works

3TIM

Ta Nha Linh

13 March 2009

Outline

• Motivation

• Challenges

• Researchers Information COllector (RICO)

• Contributions

• Future Works

4TIM

Ta Nha Linh

13 March 2009

Motivation

• Databases dedicated to scientific publications: CiteSeer, Google Scholar, ACM Portal, SpringerLink

• How about the authors of those publications?

• Publication-centric.

5TIM

Ta Nha Linh

13 March 2009

Motivation

• Researcher-centric database?– Singapore Researchers Database: researchers to sign up and input, restricted conditions, in Singapore only

– Resilience Alliance Reseachers Database: manual submission by researchers, in ecological and social sciences

– Some other similar databases: manual update, specific to certain organization

6TIM

Ta Nha Linh

13 March 2009

• Goal: Automated system to build researchers database, for multiple disciplines

• Input: Researchers’ home pages.

– Basic information

– Contact information

– Educational history

– Publications

7TIM

Ta Nha Linh

13 March 2009

Outline

• Motivation

• Challenges

• Researchers Information COllector (RICO)

• Contributions

• Future Works

8TIM

Ta Nha Linh

13 March 2009

Challenges

• Different layouts– Templates

– Personal pages

• Different content– Pages introducing researchers

– CV-like

– Personal pages

• Different content structures– Tables / lists

– Natural language text

9TIM

Ta Nha Linh

13 March 2009

10TIM

Ta Nha Linh

13 March 2009

11TIM

Ta Nha Linh

13 March 2009

12TIM

Ta Nha Linh

13 March 2009

Challenges

• Different data presentations

hangli at microsoft dot com cs.duke.edu, junyang ASJMZheng@ntu.edu.sg erafalin(at)cs.tufts.edu <Image src=’email.jpg’/> Natalio.Krasnogor -replace all this by at symbol- nottingham.ac.uk wmt then the at-sign then uci dot edu

13TIM

Ta Nha Linh

13 March 2009

Outline

• Motivation

• Challenges

• Researchers Information COllector (RICO)

• Contributions

• Future Works

14TIM

Ta Nha Linh

13 March 2009

Researchers Information COllector (RICO)

• Field Identification

• Home page Identification

• Post Processing

15TIM

Ta Nha Linh

13 March 2009

RICO - Architecture

Home page Identification

Field Identification

Post-Processing

16TIM

Ta Nha Linh

13 March 2009

Researchers Information COllector (RICO)

• Field Identification

• Home page Identification

• Post Processing

17TIM

Ta Nha Linh

13 March 2009

Field Identification - Purpose

• To identify data in the page contents to corresponding fields in a pre-defined set of desired information.

• Current set includes:Name – Position – Affiliation

Address – Phone – Fax - Email

BS year – BS major – BS university

MS year – MS major – MS university

PhD year – PhD major – PhD university

Research Interest – Publications

18TIM

Ta Nha Linh

13 March 2009

Field Identification - Related works• Tang et al (2007), (2008) – ArnetMiner

– Prepocessing: tokenize text into 5 categories

– Tagging of tokens by using Conditional Random Field (CRF)

– F1 = 83.37% (~1,000 researchers)

– Set of features used: + Content features (word, morphological, image

features)+ Pattern features (positive word, special token,

reseacher name features)+ Term features (term, dictionary features)

19TIM

Ta Nha Linh

13 March 2009

Field Identification - Related works

• Tang et al (2007), (2008) – ArnetMiner

– Has researcher’s name as input. This is an important information to be made used of when parse other fields. Different from TIM.

– Based only on text of the page. Stylistic information can be of use.

20TIM

Ta Nha Linh

13 March 2009

Field Identification - Methodology

• Input: a researcher home page

• CRF is the learning model

• Features used– Global features

– Lexicon features

– Context features

– Dictionaries features

– Stylistic features

21TIM

Ta Nha Linh

13 March 2009

Field Identification - Methodology

• Global features: apply for current token– Morphological features

– Initials

– Number

– Punctuation

• Lexicon features: apply for current token– Positive words for certain annotation fields: Position, Affiliation, Address, Phone, Fax, Email

22TIM

Ta Nha Linh

13 March 2009

Field Identification - Methodology• Context features: apply for whole line

– Name context– Address context– Phone context: 'phone', 'tel', 'mobile'– Fax context: 'fax', 'facsimile'– Email context: 'email', 'e-mail'– Bachelor (BS) context: appearance of 'B.S' or 'BS' or 'Bachelor'– Master (MS) context: appearance of 'M.S' or 'MS' or 'Master'– Ph.D (PhD) context: appearance of 'Ph.D' or 'Doctorate' or 'Doctor(ate) of Philosophy'– Research-interest context: multiple line property– Publication context: multiple line property– Degree: help to correctly differentiate BS/MS/PhD info when they are presenting in prose style / on the same line.

23TIM

Ta Nha Linh

13 March 2009

Field Identification - Methodology• Dictionaries

– Parscit dictionary: detect male names, female names, popular last names, month names, place names, publisher names, each is a single feature

– Major dictionary: to help in identifying researchers' major in their educational history, may also help in Research Interests

– Research dictionary: classified into high/mid/low confidence.

– Universities dictionary: of names of most of universities, according to Open Directory

24TIM

Ta Nha Linh

13 March 2009

Field Identification - Methodology

• Stylistic features– List feature

– Table features

– Section feature: based on html tags like <div>, <p>, <title>, header tags, list elements, table

25TIM

Ta Nha Linh

13 March 2009

Field Identification - PerformanceData set of 40 home pages, cross validation

Overall Precision: 70.66 – Recall: 62.73 – F1: 64.87

Classes Precision Recall F1

name 75.66% 51.34% 61.17

phone 53.38% 89.25% 66.80

fax 47.73% 72.41% 57.53

email 79.31% 70.77% 74.80

address 78.90% 74.57% 76.67

affiliation 30.27% 59.47% 40.12

position 79.46% 64.49% 71.20

research-interest

48.48% 36.04% 41.34

publications 71.05% 43.27% 53.79

Classes Precision Recall F1

bs-major 88.89% 78.05% 83.12

bs-uni 68.67% 57.00% 62.30

bs-year 90.00% 72.00% 80.00

ms-major 71.43% 32.26% 44.44

ms-uni 52.94% 52.94% 52.94

ms-year 77.78% 56.00% 65.12

phd-major 83.33% 73.17% 77.92

phd-uni 74.56% 72.03% 73.28

phd-year 100.00% 74.07% 85.11

26TIM

Ta Nha Linh

13 March 2009

Field Identification - Discussion

• Data fields to be annotated similar to those from ArnetMiner.– Extra: Name, Research Areas, Publications

– Missing: Image

• Stylistic feature used is minimal

27TIM

Ta Nha Linh

13 March 2009

Field Identification - Discussion

• F1 value is significantly lower than that of ArnetMiner’s– ArnetMiner has the researcher name as input, and uses features referring to researcher name to identify other fields. RICO has no prior knowledge about the page to be parsed.

Heuristic to improve confidence of ‘Name’

Make use of Affiliation name input

– Identifying ‘Research Interest’ and ‘Publications’ is challenging.

Improve ‘Publications’

28TIM

Ta Nha Linh

13 March 2009

Researchers Information COllector (RICO)

• Field Identification

• Home page Identification

• Post Processing

29TIM

Ta Nha Linh

13 March 2009

Home page Identification - Purpose

• Add-on component

• To complete automation of the system

30TIM

Ta Nha Linh

13 March 2009

Home page Identification – Related works

• Ahoy!– Input: Researcher name and (optional) institution name

– “Home page”: allocated page, classified by URL patterns

• RICO– Input: Institution name

– “Home page”: allocated page with biographical information, classified by contents

31TIM

Ta Nha Linh

13 March 2009

Home page Identification – Methodology• Collect a list of Universities domains

• Use Yahoo! BOSS to search for professors in the institutions

• For each valid web page, fetch the page, scan for words indicating ‘phone’, ‘mail’ and ‘professor’.

• Classify by the number of appearance of keywords.

• Home pages will be passed to Fields Identification component.

32TIM

Ta Nha Linh

13 March 2009

Home page Identification – Discussion

• Query used not able to get all relevant pages. Tune for majority: professors in institutions.– Target researchers in research organizations.

• Drawback: result set from Yahoo! BOSS may get duplicate pages, or sub-pages of a researcher’s home page Treated as 2 different records.– Need high confidence in overall system performance. But researcher names are not unique.

– Best if can eliminate duplication by analyzing URLs. But domain hierarchies differ within department, between departments, and between institutions.

33TIM

Ta Nha Linh

13 March 2009

Researchers Information COllector (RICO)

• Field Identification

• Home page Identification

• Post Processing

34TIM

Ta Nha Linh

13 March 2009

Post-processing - Purpose

• Input: CRF++ output file from Fields Identification.

• Group neighboring tokens identified with the same annotation tag

• Deduplication

• Store into database (current size ~ 170,000 researchers)

35TIM

Ta Nha Linh

13 March 2009

Outline

• Motivation

• Challenges

• Researchers Information COllector (RICO)

• Contributions

• Future Works

36TIM

Ta Nha Linh

13 March 2009

Contribution

• Produced an automated system for fetching researchers’ information from the world wide web.

• Introduced a number of features for Field Identification machine learning.

37TIM

Ta Nha Linh

13 March 2009

Outline

• Motivation

• Challenges

• Researchers Information COllector (RICO)

• Contributions

• Future Works

38TIM

Ta Nha Linh

13 March 2009

Future improvements• Field Identification

– Introduce more features, especially stylistic features– Strengthen features targeting Name, Research Interest and Publications tags– Cater for the <image> tag– Be able to handle pages using HTML frames– Be able to follow links on the page if necessary

• Home page Identification– Improve heuristics

• Post-processing– Be able to refine output from Fields Identification

• A new component to facilitate front end for user to query the database

39TIM

Ta Nha Linh

13 March 2009

THANK YOU!

Question?