Post on 18-Jan-2016
Semantic web Bootstrapping & Annotation
Hassan Sayyadi
sayyadi@ce.sharif.edu
Semantic web research laboratory
Computer department
Sharif university of technology
2
Outline• What is annotation?
• Why use annotation?
• Crawler
• Annotation model
• Annotation methods
• Our Implementation
3
Outline• What is annotation?
• Why use annotation?
• Crawler
• Annotation model
• Annotation methods
• Our Implementation
4
What is annotation?• People make notes to themselves in order to
preserve ideas that arise during a variety of activities
• The purpose of these notes is often to summarize, criticize, or emphasize specific phrases or events
• Semantic annotations are to tag ontology class instance data and map it into ontology classes.
5
Outline• What is annotation?
• Why use annotation?
• Crawler
• Annotation model
• Annotation methods
• Our Implementation
6
Why use annotation?
• To have the world knowledge at one's finger tips seems possible.
• The Internet is the platform for information.
• Unfortunately most of the information is provided in an unstructured and non-standardized form.
7
Why use annotation? (continue)
8
Outline• What is annotation?
• Why use annotation?
• Crawler
• Annotation model
• Annotation methods
• Our Implementation
9
Crawler
• A crawler is a program which traverses the Internet following these links from one page to the next.
10
Focused crawler• Not all the Internet knowledge is required for
every query.• This assumption seems reasonable because
most people work on a restricted domain and do not need the knowledge of the whole Internet
• Searching the whole Internet in this case is very inefficient and expensive.
• Free texts in the Internet contain various information in diverse domains.
11
Focused crawler (continue)
• The focus can be achieved by examining keywords
• Problems: – “Understanding“ the semantic of document– Extremely focusing on one topic
• Another way to focus is the Internet connectivity structure
12
Outline• What is annotation?
• Why use annotation?
• Crawler
• Annotation model
• Annotation methods
• Our Implementation
13
Annotation models
• Mark in web page
• Example:– SUT is one of the largest engineering
schools in the Islamic Republic of Iran– <university>SUT</university> is one of the
largest universities in the <country>Islamic Republic of Iran</country>
14
Annotation models (continue)• Generate RDF• Example:
– SUT is one of the largest engineering schools in the Islamic Republic of Iran
– <rdf:Description rdf:about="http://sharif.edu/#SUT"> <rdf:type>university</rdf:type>
<SHARIF:be_in rdf:resource="http://sharif.edu/#Islamic+Republic+of+Iran"/>
</rdf:Description> <rdf:Description rdf:about="http://sharif.edu/#Islamic+Republic+of+Iran”> <rdf:type>Country</rdf:type> </rdf:Description>
15
Outline• What is annotation?
• Why use annotation?
• Crawler
• Annotation model
• Annotation methods
• Our Implementation
16
Annotation methods
• Manually
• Semi-automatically
• Automatically
17
Automatic Annotation
• The fully automatic creation of semantic annotations is an unsolved problem.
• Automatic semantic annotation for the natural language sentences in these pages is a daunting task and we are often forced to do it manually or semi-automatically using handwritten rules
18
Manual Annotation• Manual annotation is more easily accomplished
today, using authoring tools, which provide an integrated environment for simultaneously authoring and annotating text.
• However, the use of human annotators is often fraught with errors due to factors such as annotator familiarity with the domain, amount of training, personal motivation and complex schemas
• Manual annotation is also an expensive process
19
Semi-automatic Annotation
• To overcome the annotation acquisition bottleneck, semiautomatic annotation of documents has been proposed.
20
Semi-automatic annotation
• assumptions:– vocabulary set is limited– word usage has patterns– semantic ambiguities are rare– terms and jargon of the domain appear
frequently
21
Semantic Annotation Platform (SAP)
22
Multistrategy SAPs• Multistrategy SAPs are able to combine
methods from both pattern-based and machine learning-based systems.
• No SAP currently implements the multistrategy approach for semantic annotation, although it has been implemented in systems for ontology extraction (such as On-To-Knowledge)
23
Semi-automatic annotation (continue)• Example
– I go to Shanghai
• Link structure is
more like a RDF
graph
24
The accuracy of concepts and relations about different algorithm
25
Automatic annotation
26
Source preprocessing
• Document Object Model (DOM)
• Text Model
• Layout Model
• NLP Model
27
Information Identification• Operators
– perform extraction actions on document access models
– Retrieval, Check, Execute• Strategies
– build operator sequences according to user time and quality requirements
• Source Description– build operator sequences according to user time
and quality requirements
28
Ontology population• The final stage of the overall process is to
decide which hypothesis represents the extracted information to insert into the ontology
• The module simulates insertions and calculates the cost according to the number of new instance creations, instance modifications or inconsistencies found
29
Outline• What is annotation?
• Why use annotation?
• Crawler
• Annotation model
• Annotation methods
• Our Implementation
30
Our implementation
• Crawler:– Crawl all link that contains:
• sharif.ir• sharif.edu• sharif.ac.ir
31
Our implementation• Source pre-processing
– Html to text• text = text.replaceAll("\n", "*_newline_*");• text = text.replaceAll("\\<script.*?\\</script\\>", "");• text = text.replaceAll("\\<style.*?</style.*\\>", "");• text = text.replaceAll("<\\!--.*?--\\>", "");• text = text.replaceAll("\\<.*?\\>", "");• text = text.replaceAll(" ", " ");• text = text.replaceAll("<", "<");• …• text = text.replaceAll("\\*_newline_\\*", "\n");
– Additional• text = text.replaceAll("\n(\n|| )*\n",".");• text = text.replaceAll(",", " and ");
32
Our implementation
• Information extraction:– JMontyLingua
• SUT is one of the largest engineering schools in the Islamic Republic of Iran
• ("be" "SUT" "one" "of largest engineering school" "in Islamic Republic" "of Iran")
33
Our implementation
• JMontyLingua problem:– SUT has computer, mechanic and electric
engineering departments – ("have" "SUT" "computer mechanic and
electric engineering departments")– ("have" "SUT" "computer and mechanic
and electric engineering departments")
34
Our inplementation• ("be" "SUT" “university" "in Islamic Republic" "of Iran")
• => ("be" "SUT" “university" "in Islamic Republic of Iran")
• =>SUT,be,university & SUT,be_in,Islamic Republic of Iran
• <rdf:Description rdf:about="http://sharif.edu/#SUT"> <rdf:type>university</rdf:type>
<SHARIF:be_in rdf:resource="http://sharif.edu/#Islamic+Republic+of+Iran"/>
</rdf:Description>
35
Any question?