Publishing withTriple Pattern Fragments

Ruben Taelman - @rubensworksimec - Ghent University

1

Publishing with Triple Pattern FragmentsTPF server software

TPF client-side querying

Quad Pattern Fragments

Demo

2

Publishing with Triple Pattern FragmentsTPF server software

TPF client-side querying

Quad Pattern Fragments

Demo

3

http://linkeddatafragments.org/software/

JavaScript LDF serverPythonPerlRubyPHPJava...

TPF server implementations in different languages

4

Requires Node.js ≥ 4.0

Installing and running the LDF server with Node

5

[sudo] npm install -g ldf-server

ldf-server config.json <port> <workers>

Documentation: https://github.com/LinkedDataFragments/Server.js

Requires Docker

Installing and running the LDF server with Docker

6

docker pull linkeddatafragments/server.js

docker run -p <port>:3000 -it --rm \-v $(pwd)/config.json:/tmp/config.json \

ldf-server /tmp/config.json

Preconfigured LDF server with NGINX cache and web-client

Setting up a full stack with Docker (Compose)

7

docker-compose up

https://github.com/LinkedDataFragments/FullStackServer

{ "title": "My Linked Data Fragments server", "datasources": { "dbpedia": { "title": "DBpedia 2014", "type": "HdtDatasource", "description": "DBpedia 2014 with an HDT back-end", "settings": { "file": "data/dbpedia2014.hdt" } }, "dbpedia-sparql": { "title": "DBpedia 3.9 (Virtuoso)", "type": "SparqlDatasource", "description": "DBpedia 3.9 with a Virtuoso back-end", "settings": { "endpoint": "http://dbpedia.restdesc.org/", "defaultGraph": "http://dbpedia.org" } } }}

LDF server is configured with config.json

8

{ "title": "My Linked Data Fragments server", "datasources": { "dbpedia": { "title": "DBpedia 2014", "type": "HdtDatasource", "description": "DBpedia 2014 with an HDT back-end", "settings": { "file": "data/dbpedia2014.hdt" } }, "dbpedia-sparql": { "title": "DBpedia 3.9 (Virtuoso)", "type": "SparqlDatasource", "description": "DBpedia 3.9 with a Virtuoso back-end", "settings": { "endpoint": "http://dbpedia.restdesc.org/", "defaultGraph": "http://dbpedia.org" } } }}

Configure list of datasources

9

{ "title": "My Linked Data Fragments server", "datasources": { "dbpedia": { "title": "DBpedia 2014", "type": "HdtDatasource", "description": "DBpedia 2014 with an HDT back-end", "settings": { "file": "data/dbpedia2014.hdt" } }, "dbpedia-sparql": { "title": "DBpedia 3.9 (Virtuoso)", "type": "SparqlDatasource", "description": "DBpedia 3.9 with a Virtuoso back-end", "settings": { "endpoint": "http://dbpedia.restdesc.org/", "defaultGraph": "http://dbpedia.org" } } }}

Load an HDT file

10

{ "title": "My Linked Data Fragments server", "datasources": { "dbpedia": { "title": "DBpedia 2014", "type": "HdtDatasource", "description": "DBpedia 2014 with an HDT back-end", "settings": { "file": "data/dbpedia2014.hdt" } }, "dbpedia-sparql": { "title": "DBpedia 3.9 (Virtuoso)", "type": "SparqlDatasource", "description": "DBpedia 3.9 with a Virtuoso back-end", "settings": { "endpoint": "http://dbpedia.restdesc.org/", "defaultGraph": "http://dbpedia.org" } } }}

Act as a proxy to a SPARQL endpoint

11

HDTN-TriplesTurtleJSON-LDSPARQL-endpoint

Different data sources are possible

12

by extending lib/datasources/Datasource.js

Return triple stream given triple pattern, offset and limit

...or write your own datasource implementation

13

Exposing multiple datasources as if it was one

Useful for fragmented datasets that can’t be merged

Compose multiple datasources

14

https://github.com/LinkedDataFragments/Server.js/blob/master/config/config-composite.json

Publishing with Triple Pattern FragmentsTPF server software

TPF client-side querying

Quad Pattern Fragments

Demo

15

http://linkeddatafragments.org/software/

JavaScript LDF clientPythonPerlJava...

Different client implementations exist as well

16

No installation required at all!

http://client.linkeddatafragments.org/

Use the web client

17

Requires Node.js ≥ 4.0

Run from command line or include in your source code

Installing and running the LDF client with Node

18

[sudo] npm install -g ldf-client

ldf-client <tpf-endpoint-url> <sparql-query-path>

Documentation: https://github.com/LinkedDataFragments/Client.js

Requires Docker

Installing and running the LDF client with Docker

19

docker pull linkeddatafragments/client.js

docker run -it --rm \-v <sparql-query-path>:/tmp/query.sparql \

linkeddatafragments/client.js \<tpf-endpoint-url> /tmp/query.sparql

Publishing with Triple Pattern FragmentsTPF server software

TPF client-side querying

Quad Pattern Fragments

20

Quads are triples extended with a fourth element, the graph

<s> <p> <o> <g>.

Triples sometimes need some context

21

Quad Pattern Fragments (QPF)

Adding the fourth element to the interface

22

SELECT *FROM <http://example.org/graph0>FROM NAMED <http://example.org/graph1>WHERE {

GRAPH ?g {?s ?p ?o.

}}

QPF Clients are able to use quad features

23

QPF is backwards-compatible with TPF

24

TPF

QPFTPF

QPFIgnores graphs

Quad Pattern Fragments (WIP)https://github.com/LinkedDataFragments/Server.js/tree/feature-qpf-latest

https://github.com/LinkedDataFragments/Client.js/tree/feature-qpf-latest

25

Publishing with Triple Pattern FragmentsTPF server software

TPF client-side querying

Quad Pattern Fragments

Demo

26

Let’s try overloading the DBpedia TPF endpoint!

27

http://fragments.dbpedia.org/

Uptime of 99.9967%

28

29

30

Some query types are very slow

~1 hour!

Server running TPF server software with any dataset

31

TPF server software

RDF dataset

HDT dataset

SPARQL endpoint

...