Scalable Integration and Processing of Linked Data

Tutorial at WWW 2011

Scalable Integration and Processing of Linked Data

Andreas Harth, Aidan Hogan, Spyros Kotoulas, Jacopo Urbani

2

Outline

Session 1: Introduction to Linked DataFoundations and ArchitecturesCrawling and IndexingQuerying

Session 2: Integrating Web Data with ReasoningIntroduction to RDFS/OWL on the WebIntroduction and Motivation for Reasoning

Session 3: Distributed Reasoning: Because Size MattersProblems and ChallengesMapReduce and WebPIE

Session 4: Putting Things Together (Demo)The LarKC PlatformImplementing a LarKC Workflow

3

PART I: How can we query Linked Data?

PART 2: How can we reason over Linked Data? (start of Session 2)

4

Answer: SPARQL (W3C Rec. 2008)

…SPARQL 1.1 upcoming (W3C Rec. 201?)

5

SPARQL Protocol and RDF Query Language (SPARQL)

Introducing SPARQL

Standardised query language (and supporting recommendations) for querying RDF

~SQL-like language…but only if you squint…and without the vendor-specific headaches

6

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX oo: <http://purl.org/openorg/>

SELECT ?name ?expertise

FROM NAMED <http://data.southampton.ac.uk/>

WHERE { ?person foaf:name ?name . ?person rdf:type foaf:Person . ?person foaf:title ?title . FILTER regex(?title, "^Prof") OPTIONAL { ?person oo:availableToCommentOn ?expertiseURI . ?expertiseURI rdfs:label ?expertise }}

ORDER BY ?surname

The anatomy of a typical SPARQL query

Give me a list of names of professors in Southampton and their expertise (if available), in

order of their surname

PREFIX DECLARATIONS

RESULT CLAUSE

QUERY CLAUSE

SOLUTION MODIFIERS

DATASET CLAUSE

; foaf:familyName ?surname .

7




WHERE { ?person foaf:name ?name ; foaf:familyName ?surname . ?person rdf:type foaf:Person . ?person foaf:title ?title . FILTER regex(?title, "^Prof") OPTIONAL { ?person oo:availableToCommentOn ?expertiseURI . ?expertiseURI rdfs:label ?expertise }}

ORDER BY ?surname




PREFIX DECLARATIONS

RESULT CLAUSE

QUERY CLAUSE

SOLUTION MODIFIERS

DATASET CLAUSE

8


Prefix Declarations

foaf:Person ⇔ <http://xmlns.com/foaf/0.1/Person>

Use http://prefix.cc/ …

PREFIX DECLARATIONS

http://prefix.cc/

http://prefix.cc/rdfs

9





ORDER BY ?surname




PREFIX DECLARATIONS

RESULT CLAUSE

QUERY CLAUSE

SOLUTION MODIFIERS

DATASET CLAUSE

10


Result Clause

1. SELECT2. CONSTRUCT (RDF)

3. ASK 4. DESCRIBE (RDF)

RESULT CLAUSE

11

Return all tuples for the bindings of the variables ?name and ?expertise

-----------------------------------------------------------| “Professor Robert Allen” | “Control engineering” || “Professor Robert Allen” | “Biomedical engineering” || “Prof Carl Leonetto Amos” | || “Professor Peter Ashburn” | “Silicon technology” || “Professor Robert Allen” | “Control engineering” |-----------------------------------------------------------

Result Clause 1. SELECT…SELECT ?name ?expertise RESULT CLAUSE



12

Return all tuples for the bindings of the variables ?name and ?expertise

-----------------------------------------------------------| “Professor Robert Allen” | “Control engineering” || “Professor Robert Allen” | “Biomedical engineering” || “Prof Carl Leonetto Amos” | || “Professor Peter Ashburn” | “Silicon technology” || “Professor Robert Allen” | “Control engineering” |-----------------------------------------------------------

?name ?expertiseSELECT

Result Clause 1. SELECT DISTINCT…DISTINCT

unique



13

CONSTRUCT { ?person foaf:name ?name ; ex:expertise ?expertise .}

Return RDF using bindings for the variables: ex:RAllen foaf:name “Professor Robert Allen” ; ex:expertise “Biomedical engineering” , “Control engineering” .ex:PAshburn foaf:name “Peter Ashburn ” ; ex:expertise “Silicon technology” .

Result Clause 2. CONSTRUCT…

RESULT CLAUSE



14

ASK

… WHERE { … }

Is there any results?

Returns:true or false

Result Clause 3. ASK…RESULT CLAUSE

15

DESCRIBE ?person

… WHERE { ?person … }

Returns some RDF which “describes” the given resource…

No standard for what to return! Typically returns:

Result Clause 4. DESCRIBE…RESULT CLAUSE

all triples where the given resource appears as subject and/or objectOR

Concise Bounded Descriptions…

16

DESCRIBE ex:RAllen

(…can give URIs directly without need for a WHERE clause.)

Result Clause 4. DESCRIBE (DIRECT)…

RESULT CLAUSE

17





ORDER BY ?surname




PREFIX DECLARATIONS

RESULT CLAUSE

QUERY CLAUSE

SOLUTION MODIFIERS

DATASET CLAUSE

18


Dataset clause (FROM/FROM NAMED)DATASET CLAUSE

(Briefly)

Restrict the dataset against which you wish to querySPARQL stores named graphs: sets of triples which are associated with (URI) namesCan match across graphs!Named graphs typically corrrespond with data provenance (i.e., documents)! Default graph typically corresponds to the merge of all graphsMany engines will typically dereference a graph if not available locally!



19





ORDER BY ?surname




PREFIX DECLARATIONS

RESULT CLAUSE

SOLUTION MODIFIERS

DATASET CLAUSE

WHERE { ?person foaf:name ?name ; foaf:familyName ?surname . ?person rdf:type foaf:Person . ?person foaf:title ?title . FILTER regex(?title, "^Prof") OPTIONAL { ?person oo:availableToCommentOn ?expertiseURI . ?expertiseURI rdfs:label ?expertise }} QUERY CLAUSE

20

foaf:Person

foaf:name?person ?namerdf:type

foaf:title

?titleoo:availableToCommentOn

?expertiseURI rdfs:label ?expertise

[FILTER “^Prof”]foaf:familyName

?surname


Query clause (WHERE)

QUERY CLAUSE



“Professor Peter Ashburn”

“Silicon technology”“Professor”

✓ex:PAshburn

ex:Silicon

✓

“Ashburn”

21

WHERE { … {?person oo:availableToCommentOn ?expertiseURI . } UNION {?person foaf:interest ?expertiseURI . }…}

Quick mention for UNION

QUERY CLAUSE

Represent disjunction (OR)

Useful when there’s more than one property/class that represents the same information you’re interested in (heterogenity)

Reasoning can also help, assuming terms are mapped (more later)

22





ORDER BY ?surname




PREFIX DECLARATIONS

RESULT CLAUSE

SOLUTION MODIFIERS

DATASET CLAUSE

QUERY CLAUSE

23

ORDER BY ?surnameSolution Modifiers



SOLUTION MODIFIERS

Order output results by surname (as you probably guessed)

LIMIT

OFFSET

ORDER BY ?surname LIMIT 10 SOLUTION MODIFIERS

ORDER BY ?surname LIMIT 10 OFFSET 20 SOLUTION MODIFIERS

Only return 10 results

Return results 20‒30

…also…

24





ORDER BY ?surname



PREFIX DECLARATIONS

RESULT CLAUSE

QUERY CLAUSE

SOLUTION MODIFIERS

DATASET CLAUSE

What are you looking for?

Which results do you want?Where should we look?

How should results be ordered/split?

Shortcuts for URIs

The summary of a typical SPARQL query

25




WHERE { ?person foaf:name ?name . ?person rdf:type foaf:Person . ?person foaf:title ?title . FILTER regex(?title, "^Prof") OPTIONAL { ?person oo:availableToCommentOn ?expertiseURI . ?expertiseURI rdfs:label ?expertise }}

ORDER BY ?surname

Trying out a typical SPARQL query



; foaf:familyName ?surname .

http://lod.openlinksw.com/sparql

26

SparqlEndpoints (W3C Wiki)

http://www.w3.org/wiki/SparqlEndpoints(or just use Google)

List of Public SPARQL Endpoints:

http://www.w3.org/wiki/SparqlEndpoints

27

SPARQL 1.1

Currently a W3C Working Draft

http://www.w3.org/TR/sparql11-query/ (or just use Google)

Coming Soon:

http://www.w3.org/TR/sparql11-query/

28

“SPARQL by example”

By Cambridge SemanticsLee Feigenbaum & Eric Prud'hommeaux

http://www.cambridgesemantics.com/2008/09/sparql-by-example/ (or just use Google)

Highly recommend checking out:

http://www.cambridgesemantics.com/2008/09/sparql-by-example/

29

After the break…

Session 1: Introduction to Linked DataFoundations and ArchitecturesCrawling and IndexingQuerying

Session 2: Integrating Web Data with ReasoningIntroduction to RDFS/OWL on the WebIntroduction and Motivation for Reasoning

Session 3: Distributed Reasoning: Because Size MattersProblems and ChallengesMapReduce and WebPIE

Session 4: Putting Things Together (Demo)The LarKC PlatformImplementing a LarKC Workflow

30

Question: Find the people who have won both an academy award for best director and a raspberry award for worst director

Endpoint: (that is, if you want to use SPARQL… feel free to use whatever) http://dbpedia.org/sparql/ or http://google.com/ (to make it fair)

Hint: Look at http://dbpedia.org/page/Michael_Bay and http://dbpedia.org/page/Woody_Allen for examples (The same prefixes therein are understood by the endpoint, …so no need to declare them in the query)

During the break…

http://dbpedia.org/sparql/

http://google.com/

http://dbpedia.org/page/Michael_Bay

http://dbpedia.org/page/Woody_Allen

31

The Winning (?) Query:SELECT DISTINCT ?nameWHERE{ ?director dcterms:subject category:Worst_Director_Golden_Raspberry_Award_winners , category:Best_Director_Academy_Award_winners ; foaf:name ?name .}

The Answer:

…

And the answer is…

32

PART I: How can we query Linked Data?

PART 2: How can we reason over

Linked Data?…and why?!

33

… A Web of Data

Images from: http://richard.cyganiak.de/2007/10/lod/; Cyganiak, JentzschSeptember 2010

August 2007

November 2007 February 2008

March 2008

September 2008

March 2009

July 2009

http://richard.cyganiak.de/2007/10/lod/

34

Reasoning

explicit data

implicit data

How can consumers query the

implicit data

35

…so what’s The Problem?…

…heterogeneity

…need to integrate data from different sources

36

Take Query Answering…

Gimme webpages relating to

Tim Berners-Lee

foaf:page

timbl:i timbl:i foaf:page ?pages .

37

Hetereogenity in schema…

webpage: properties

foaf:page

foaf:homepage

foaf:isPrimaryTopicOf

foaf:weblog

doap:homepage

foaf:topic

foaf:primaryTopic

mo:musicBrainz

mo:myspace

…

= rdfs:subPropertyOf

= owl:inverseOf

http://trac.usefulinc.com/doap/

38

Linked Data, RDFS and OWL: Linked Vocabularies

SKOS

…

…Image from http://blog.dbtune.org/public/.081005_lod_constellation_m.jpg:; Giasson, Bergman

http://purl.org/goodrelations/


http://dc-2010.org/

39

Hetereogenity in naming…

Tim Berners-Lee: URIs

…

timbl:i

dblp:100007

identica:45563

adv:timblfb:en.tim_berners-lee

db:Tim-Berners_Lee

= owl:sameAs

http://richard.cyganiak.de/2007/10/lod/imagemap.html

http://images.google.com.hk/imgres?imgurl=http://www2005.org/keynotes/tim.jpg&imgrefurl=http://www2005.org/keynotes/&usg=__TWfydY0sN3IuwPp3y3Zb_oBkCNc=&h=2050&w=1411&sz=278&hl=zh-CN&start=5&zoom=1&itbs=1&tbnid=x8q3A6HSeRapDM:&tbnh=150&tbnw=103&prev=/images?q=tim+berners+lee&hl=zh-CN&safe=strict&gbv=2&tbs=isch:1

http://images.google.com.hk/imgres?imgurl=http://4.bp.blogspot.com/_cMDe93PF3Q0/Sw1DbCbYSVI/AAAAAAAAAKk/Us-BMFFje28/s1600/tim2_g1.7.gif&imgrefurl=http://blinking-links.blogspot.com/2009/11/tech-hacking-history.html&usg=__LaYkUQ8282f_od6N88ApjIxNDxU=&h=633&w=378&sz=143&hl=zh-CN&start=2&zoom=1&itbs=1&tbnid=7F-qcKdeY-6DnM:&tbnh=137&tbnw=82&prev=/images?q=tim+berners+lee&hl=zh-CN&safe=strict&gbv=2&tbs=isch:1

http://images.google.com.hk/imgres?imgurl=http://www.samogden.com/images/tim_berners-lee.jpg&imgrefurl=http://www.samogden.com/tim_berners-lee_photo.html&usg=__beqADInuZWHziJIyV8AxsVAxT2w=&h=451&w=432&sz=27&hl=zh-CN&start=12&zoom=1&itbs=1&tbnid=LTRieFKuHi4CzM:&tbnh=127&tbnw=122&prev=/images?q=tim+berners+lee&hl=zh-CN&safe=strict&gbv=2&tbs=isch:1

http://images.google.ie/imgres?imgurl=http://cdn.mikeabundo.com/wp-content/uploads/2009/03/tim-berners-lee.jpg&imgrefurl=http://mikeabundo.com/2009/03/14/twenty-years-of-the-world-wide-web/&usg=___idDcJPse_rC7jpEeO5VGxkfutM=&h=320&w=500&sz=31&hl=en&start=17&zoom=1&itbs=1&tbnid=5_qmVxHYPp5v6M:&tbnh=83&tbnw=130&prev=/images?q=tim+berners+lee&hl=en&newwindow=1&gbv=2&tbs=isch:1

http://images.google.ie/imgres?imgurl=http://i.telegraph.co.uk/telegraph/multimedia/archive/01175/berners-lee_1175434c.jpg&imgrefurl=http://www.telegraph.co.uk/news/uknews/road-and-rail-transport/6560308/Downing-Street-plan-to-put-transport-information-at-passengers-fingertips.html&usg=__vdiAPdeRBmo4CgYUGy2WkQ23zpk=&h=288&w=460&sz=12&hl=en&start=24&zoom=1&itbs=1&tbnid=39FQnj8z865C0M:&tbnh=80&tbnw=128&prev=/images?q=tim+berners+lee&start=18&hl=en&newwindow=1&sa=N&gbv=2&ndsp=18&tbs=isch:1

http://www.freebase.com/

http://images.google.ie/imgres?imgurl=http://upload.wikimedia.org/wikipedia/commons/6/62/Tim_Berners-Lee_CP.jpg&imgrefurl=http://commons.wikimedia.org/wiki/File:Tim_Berners-Lee_CP.jpg&usg=__P7vPVZGY9qdc7fGNQ0I1hI2Hxuk=&h=3215&w=2148&sz=1845&hl=en&start=21&zoom=1&itbs=1&tbnid=LtYcwmrU2diAGM:&tbnh=150&tbnw=100&prev=/images?q=tim+berners+lee&start=18&hl=en&newwindow=1&sa=N&gbv=2&ndsp=18&tbs=isch:1

40

Returning to our simple query…

Gimme webpages relating to

Tim Berners-Lee

foaf:page

timbl:i timbl:i foaf:page ?pages .

... 7 x 6 = 42 possible patterns

foaf:homepage foaf:isPrimaryTopicOf

doap:homepage foaf:topic foaf:primaryTopic

mo:myspace SKOS

dblp:100007

identica:45563adv:timbl

fb:en.tim_berners-leedb:Tim-Berners_Lee

http://purl.org/goodrelations/


http://dc-2010.org/

http://www.freebase.com/

41

…reasoning to the rescue?

42

Challenges……what (OWL) reasoning is feasible for Linked Data?

43

Linked Data Reasoning: Challenges


44

ScalabilityAt least tens of billions of statements (for the moment)

Near linear scale!!!

Noisy dataInconsistencies galorePublishing errors


45

Challenges (Semantic Web Wikipedia Article)Some of the challenges for the Semantic Web include vastness, vagueness, uncertainty, inconsistency and deceit. Automated reasoning systems will have to deal with all of these issues in order to deliver on the promise of the Semantic Web.Vastness: The World Wide Web contains at least 48 billion pages as of this writing (August 2, 2009). The SNOMED CT medical terminology ontology contains 370,000 class names, and existing technology has not yet been able to eliminate all semantically duplicated terms. Any automated reasoning system will have to deal with truly huge inputs.Vagueness: These are imprecise concepts like "young" or "tall". This arises from the vagueness of user queries, of concepts represented by content providers, of matching query terms to provider terms and of trying to combine different knowledge bases with overlapping but subtly different concepts. Fuzzy logic is the most common technique for dealing with vagueness.Uncertainty: These are precise concepts with uncertain values. For example, a patient might present a set of symptoms which correspond to a number of different distinct diagnoses each with a different probability. Probabilistic reasoning techniques are generally employed to address uncertainty.Inconsistency: These are logical contradictions which will inevitably arise during the development of large ontologies, and when ontologies from separate sources are combined. Deductive reasoning fails catastrophically when faced with inconsistency, because "anything follows from a contradiction". Defeasible reasoning and paraconsistent reasoning are two techniques which can be employed to deal with inconsistency.Deceit: This is when the producer of the information is intentionally misleading the consumer of the information. Cryptography techniques are currently utilized to ameliorate this threat.


46

Proposition 1 Web data is noisy.

Proof: 08445a31a78661b5c746feff39a9db6e4e2cc5cf

sha1-sum of ‘mailto:’common value for foaf:mbox_sha1sum

An inverse-functional (uniquely identifying) property!!!Any person who shares the same value will be considered the same

Q.E.D.

Noisy Data: Omnipotent Being

http://www.google.com/search?q=08445a31a78661b5c746feff39a9db6e4e2cc5cf

47

Alternate proof (courtesy of http://www.eiao.net/rdf/1.0)

rdf:type rdf:type owl:Property .rdf:type rdfs:label “type”@en .rdf:type rdfs:comment “Type of resource” .rdf:type rdfs:domain eiao:testRun .rdf:type rdfs:domain eiao:pageSurvey .rdf:type rdfs:domain eiao:siteSurvey .rdf:type rdfs:domain eiao:scenario .rdf:type rdfs:domain eiao:rangeLocation .rdf:type rdfs:domain eiao:startPointer .rdf:type rdfs:domain eiao:endPointer .rdf:type rdfs:domain eiao:header .rdf:type rdfs:domain eiao:runs .

Noisy Data: Redefining everything …and home in time for tea

http://www.eiao.net/rdf/1.0

48

foaf:Person owl:disjointWith foaf:Document .

Inconsistent Data: Cannot compute…

49

…herein, we look at (monotonic) rules.

Expressive reasoning (also) possible through tableaux, but yet to demonstrate desired scale

50

Rules

IF ⇒ THENBody/Antecedent/Condition Head/Consequent

?c1 rdfs:subClassOf ?c2 . ?x rdf:type ?c1 . ⇒ ?x rdf:type ?c2 .

foaf:Person rdfs:subClassOf foaf:Agent .timbl:me rdf:type foaf:Person .⇒ timbl:me rdf:type foaf:Agent .

Schema/Terminology/Ontological

Instance/Assertional

http://images.google.com.hk/imgres?imgurl=http://www.samogden.com/images/tim_berners-lee.jpg&imgrefurl=http://www.samogden.com/tim_berners-lee_photo.html&usg=__beqADInuZWHziJIyV8AxsVAxT2w=&h=451&w=432&sz=27&hl=zh-CN&start=12&zoom=1&itbs=1&tbnid=LTRieFKuHi4CzM:&tbnh=127&tbnw=122&prev=/images?q=tim+berners+lee&hl=zh-CN&safe=strict&gbv=2&tbs=isch:1

51

Rules (Inconsistencies [a.k.a. Contradictions])

IF ⇒ THEN?c1 owl:disjointWith ?c2 .

?x rdf:type ?c1 . ?x rdf:type ?c2 .

⇒ false

foaf:Person owl:disjointWith foaf:Document .ex:sleepygirl rdf:type foaf:Person .ex:sleepygirl rdf:type foaf:Document .

⇒ false

Body/Antecedent/Condition Head/Consequent

52

Materialisation (Forward-Chaining):

Write the consequences of the rules down

Executing rules: Materialisation

53

Materialisation

Forward-chaining MaterialisationAvoid runtime expense

Users taught impatience by GooglePre-compute for quick retrievalWeb-scale systems should scale well

More data = more disk-space/machines

54

INPUT:• Flat file of triples

(quads)

OUTPUT:• Flat file of (partial)

inferred triples (quads)

55

“Standard”RDFSOWL 2 RL (W3C Rec: 27 Oct. 2009)

“Non-standard”DLPpD* (OWL Horst)OWL–

…

What rulesets?

56

Let’s look at a recent corpus of Linked Data and see what schema’s inside

(and what the rulesets support)

Open-domain crawl May 2010 1.1 billion quadruples 3.985 million sources (docs) 780 pay-level domains (e.g., dbpedia.org) Ran “special” PageRank over documents

86 thousand docs contained some RDFS/OWL schema data (2.2% of docs... but <0.2% of triples)Summated ranks of docs using each primitive

What rules?

57

Survey of Linked Data schema: Top 15 ranks

# Axiom Rank(Σ) RDFS Horst O2R1. rdfs:subClassOf 0.295 ✓ ✓ ✓2. rdfs:range 0.294 ✓ ✓ ✓3. rdfs:domain 0.292 ✓ ✓ ✓4. rdfs:subPropertyOf 0.090 ✓ ✓ ✓5. owl:FunctionalProperty 0.063 ✘ ✓ ✓6. owl:disjointWith 0.049 ✘ ✘ ✓7. owl:inverseOf 0.047 ✘ ✓ ✓8. owl:unionOf 0.035 ✘ ✘ ✓9. owl:SymmetricProperty 0.033 ✘ ✓ ✓10. owl:TransitiveProperty 0.030 ✘ ✓ ✓11. owl:equivalentClass 0.021 ✘ ✓ ✓12. owl:InverseFunctionalProperty 0.030 ✘ ✓ ✓13. owl:equivalentProperty 0.030 ✘ ✓ ✓14. owl:someValuesFrom 0.030 ✘ ✓ ✓15. owl:hasValue 0.028 ✘ ✓ ✓

58

What about noise? ……need to consider the provenance of Web data

59

Consider source of schema data

Class/property URIs dereference to their authoritative documentFOAF spec authoritative for foaf:Person ✓MY spec not authoritative for foaf:Person ✘

Allow “extension” in third-party documentsmy:Person rdfs:subClassOf foaf:Person . (MY spec) ✓

BUT: Reduce obscure membershipsfoaf:Person rdfs:subClassOf my:Person . (MY spec) ✘

ALSO: Protect specificationsfoaf:knows a owl:SymmetricProperty . (MY spec) ✘

Authoritative Reasoning

60

More proof (courtesy of http://www.eiao.net/rdf/1.0)


60



61

Gong Cheng, Yuzhong Qu. "Integrating Lightweight Reasoning into Class-Based Query Refinement for

Object Search." ASWC 2008.

Aidan Hogan, Andreas Harth, Axel Polleres. "Scalable Authoritative OWL Reasoning for the Web." IJSWIS 2009. Aidan Hogan, Jeff Z. Pan, Axel Polleres and Stefan Decker. "SAOR: Template Rule Optimisations for Distributed Reasoning over 1 Billion

Linked Data Triples." ISWC 2010.

My thesis: http://aidanhogan.com/docs/thesis/ (or use Google).

Authoritative Reasoning: read more …w/ essential plugs

http://aidanhogan.com/docs/thesis/

62 62

Quarantined reasoning!

Separate and cache hierarchy of schema documents/dependencies…

Alternative to Authoritative Reasoning?

63 63

Quarantined Reasoning [Delbru et al.; 2008]

64 64


65 65


66 66

A-Box / Instance Data (e.g, a FOAF file)

T-Box / Ontology Data (e.g., the FOAF ontology and its indirect imports)


67

More proof (courtesy of http://www.eiao.net/rdf/1.0)




68

R. Delbru, A. Polleres, G. Tummarello and S. Decker. "Context Dependent Reasoning for Semantic Documents in Sindice. “ 4th

International Workshop on Scalable Semantic Web Knowledge Base Systems, 2008.

Quarantined Reasoning: read more

69

…what about owl:sameAs?

70 70

Consolidation for Linked Data


71

Use provided owl:sameAs mappings in the data

timbl:i owl:sameas identica:45563 .dbpedia:Berners-Lee owl:sameas identica:45563 .

Store “equivalences” found

timbl:i ->identica:45563 ->dbpedia:Berners-Lee ->

timbl:iidentica:45563dbpedia:Berners-Lee

Consolidation: Baseline

72

For each set of equivalent identifiers, choose a canonical term


Consolidation: Baseline

73

Afterwards, rewrite identifiers to their canonical version:

Canonicalisation

timbl:i rdf:type foaf:Person .identica:48404 foaf:knows identica:45563 .

dbpedia:Berners-Lee dpo:birthDate “1955-06-08”^^xsd:date .

dbpedia:Berners-Lee rdf:type foaf:Person .identica:48404 foaf:knows dbpedia:Berners-Lee .

dbpedia:Berners-Lee dpo:birthDate “1955-06-08”^^xsd:date .


74

Infer owl:sameAs through reasoning (OWL 2 RL/RDF)1. explicit owl:sameAs (again)2. owl:InverseFunctionalProperty3. owl:FunctionalProperty4. owl:cardinality 1 / owl:maxCardinality 1

foaf:homepage a owl:InverseFunctionalProperty .timbl:i foaf:homepage w3c:timblhomepage .adv:timbl foaf:homepage w3c:timblhomepage .

⇒timbl:i owl:sameas adv:timbl .

…then apply consolidation as before

Extended Consolidation

75

For our Linked Data corpus: 1. ~12 million explicit owl:sameAs triples (as before)2. ~8.7 million thru. owl:InverseFunctionalProperty3. ~106 thousand thru. owl:FunctionalProperty4. none thru. owl:cardinality/owl:maxCardinality

In terms of equivalences found (baseline vs. extended):~2.8 million sets of equivalent identifiers (1.31x baseline)~14.86 million identifiers involved (2.58x baseline)~5.8 million URIs !!(1.014x baseline)!!

Consolidation: Results

76

Conclusion…

77

Heterogeneity poses a significant problem for consuming Linked Data1. Heterogenity in schema2. Heterogenity in naming

…but we can use the mappings provided by publishers to integrate heterogeneous Linked Data corpora (with a little caution)

3. Lightweight rule-based reasoning can go a long way4. Deceit/Noise ≠ End Of World

Consider source of data!5. Inconsistency ≠ End Of World

Useful for finding noise in fact!6. Explicit owl:sameAs vs. extended consolidation:

Extended consolidation mostly (but not entirely) for consolidating blank-nodes from older FOAF exporters

Conclusions

78

How can we reason at Web scale?

Scalable/distributed rule-based materialisation over MapReduce using the WebPIE system

Next up…

79

timbl:i foaf:page ?pages .


dbpedia:Berners-Lee foaf:page ?pages .

80 80

Authoritative Reasoning (Appendix)

OWL 2 RL rule prp-inv1?p1 owl:inverseOf ?p2 . ?x ?p1 ?y . ⇒ ?y ?p2 ?x .

OWL 2 RL rule prp-inv2?p1 owl:inverseOf ?p2 . ?x ?p2 ?y . ⇒ ?y ?p1 ?x .

TBOX:foo:doesntKnow owl:inverseOf

foaf:knows . (from foo:)

ABOX:bar:Aidan foo:doesntKnow bar:Axel . bar:Stefan foaf:knows bar:Jeff .

AUTHORITATIVE INFERENCE:bar:Axel foaf:knows bar:Aidan .bar:Jeff foo:doesntKnow

bar:Stefan .

✓✘

Scalable Integration and Processing of Linked Data

Documents

Transcript of Scalable Integration and Processing of Linked Data