Endpoint LinkedDataStack SetupaLocalSPARQL Endpoint...

Dockerizing LinkedData

Natanael Arndt

IntroductionCurrent Situation

Setting Up a SPARQLEndpoint

Linked Data Stack

Setup a Local SPARQLEndpoint

Other Approaches

The DLD Approach

Background andToolingContainer-BasedVirtualisation

Docker

Docker Compose

ArchitectureProvisioning of a ContainerSetup

Discussion andConclusion

Dockerizing Linked DataKnowledge Base Shipping to the Linked Open Data Cloud

Natanael Arndt, Markus Ackermann,Martin Brümmer and Thomas Riechert

Semantics, Vienna 2015

September 16th 2015

Logo: cc-by-sa 3.0 Rika Yoshida (https://github.com/R-Y), adapted by Natanael Arndt

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

▶ Low availability of popular SPARQL endpoints▶ Setting up a local SPARQL endpoint is a complex task+ Our approach could help to improve reproducibility of experiments

with/on RDF data

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

IntroductionSetting Up a SPARQL Endpoint

1. Installation of a triple store2. Loading the data to be published into the triple store3. Setting up a (publicly available) SPARQL endpoint4. Providing a presentation application to support the exploration of the

knowledge base5. Ensuring de-referencability of IRIs occurring in the published

knowledge base6. Maintain the setup and ensure its availability

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

IntroductionLinked Data Stack

Interlinking/ Fusing

Classifi-cation/

Enrichment

Quality Analysis

Evolution / Repair

Search/ Browsing/

Exploration

Extraction

Storage/ Querying

Manual revision/

Authoring

Linked DataLifecycle

Figure: The Linked Data Stack http://stack.linkeddata.org

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

IntroductionLinked Data Stack

Figure: Position in the Linked Data Stack http://stack.linkeddata.org

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

IntroductionSetup a Local SPARQL Endpoint

▶ The setup is a time consuming task▶ Requires a certain level of technical knowledge▶ Need of DevOp-competences or an administrator

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

IntroductionOther Approaches

▶ What about a hosted Linked Data publishing infrastructure e.g.http://dydra.com/

▶ Still depend on the availability of the service▶ No controlled environment▶ Privacy issues?

▶ Why aren’t we just using any packaging system?

▶ limits in platform choice, portability

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

▶ Why aren’t we just using any packaging system?▶ limits in platform choice, portability

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

IntroductionThe DLD Approach

▶ Containerised Linked Data publication infrastructure▶ Benefits regarding the maintainability and ease of setup▶ Modularisation of individual components, following the micro service

principle

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Background and ToolingContainer-Based Virtualisation

▶ Provides an isolated execution environment for applications▶ Keep all necessary dependencies together to run a process▶ Share the complete hardware and core Operation System (OS, in

contrast to full- and para-virtualisation)▶ Still each container has its own filesystem▶ Can allow reproducibility of execution environments

Examples of container-based virtualisation:FreeBSD jail, Linux Containers (LXC), Docker

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Background and ToolingDocker

Debian

Virtuoso

OntoWiki

Imagesprototypical services

OntoWiki

Virtuoso

Containersrunning instances

DockerfileRecipe for creating an image

Ubuntu

Debian

Virtuoso

OntoWiki

Registry

docker pull

docker rundocker push

docker build

overlay file systemfor write access

docker host/deamon (local) docker hub (remote)

- stores only diffs- provides versioning

Docker is a container-based virtualisation system running on mayor OS:Linux, MacOSX and Windows and on mayor cloud platforms (e.g. AmazonEC2, Microsoft Azure, Google Cloud Platform)

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Background and ToolingDocker Compose

OntoWiki Virtuoso

docker-compose.ymlRecipe for a setup of multiple containers

--link=virtuoso

--volume=/tmp/import:/import--publish=8088:80

docker run docker run

Docker Compose (former known as fig), allows to combine multiple microservices to a bigger service

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Architecture

Triple Store(Virtuoso,Open RDF,Fuseki, …)

Presentation,Exploration

& EditComponents

DatabaseFiles

Git Repository/Local Filesystem

ODBC/HTTP/SPARQL

HTTP/SPARQL

ODBC/SPARQL

Host System

Back-Up

Figure: Architecture and data-flow of the containerised micro services forpublishing knowledge bases

The following roles:▶ Triple Storage▶ Load & Back-Up▶ Presentation & Publication

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Architecture

Triple Store(Virtuoso,Open RDF,Fuseki, …)

Presentation,Exploration

& EditComponents

DatabaseFiles

Git Repository/Local Filesystem

ODBC/HTTP/SPARQL

HTTP/SPARQL

ODBC/SPARQL

Host System

Back-Up

Figure: Architecture and data-flow of the containerised micro services forpublishing knowledge bases

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Architecture

Triple Storage▶ Persistent storage is implemented using volumes▶ Provides SPARQL interface▶ In the current architecture the is the limitation of only one Triple Store

Load & Back-Up▶ A load component is responsible for pre-loading the store▶ Data is taken from a local volume or an online source▶ Backup Containers dump the data to a local volume or an online

repositoryPresentation & Publication

▶ Provide exploration interfaces,▶ Editing/curation interfaces or▶ Domain specific views

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Architecture

Figure: DLD containers at the docker hub, searching to “dld-”https://hub.docker.com/search/?q=dld-&isAutomated=0

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

ArchitectureProvisioning of a Container Setup

datasets:dbpedia -2015-endpoint -some-set:

graph_name: "http://dbpedia.org"file: "dbp -2015-a.rdf"

dbpedia -2015-endpoint -some-other -set:graph_name: "http://dbpedia.org"location: "http://dbpedia.org/files/dbp -2015-b ¬

environment_global:DEFAULT_GRAPH: "http://dbpedia.org"Listing 1: A DLD configuration file for publishing DBpedia data …

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

components:load: aksw/dld-load-virtuosostore:

image: aksw/dld-store -virtuoso7present:

ontowiki:image: aksw/dld-present -ontowikiports: ["8088:80"]

Listing 2: … using a setup with Virtuoso triple store, a suitable loadcomponent and OntoWiki for presentation/editing

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

DockerCompose

Configuration

Docker Registry(http://registry.hub.docker.com/)

DLDSetup

Configuration

Docker Images

LOD-CloudRDF Knowledge Bases

Volumes

Container Setup

LABELimage meta information

DockerDaemon

Figure: Workflow for creating a container setup

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Discussion and Conclusion

▶ Provide an Infrastructure for bootstrapping Linked Data publicationinfrastructure

▶ Lower the entry barrier for setting up a SPARQL endpoint▶ Improved portability of Linked Data setups to transferring a dld.yml file

Future Work

▶ Embed the container and setup descriptions in DataID [1]▶ Integration into the Linked Data Stack http://stack.linkeddata.org▶ Provide more images and ready to run setups▶ Improve the usability for domain experts▶ Provide a UI for configuring the DLD-Setup-Tool

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Discussion and Conclusion

Figure: Homepage with an example at: http://dld.aksw.org/

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

Thank you for your attention!$ ./dld.py$ cd wd-dld$ docker-compose up

Natanael Arndt

Linked Data Stack

Other Approaches

The DLD Approach

Docker

Docker Compose

M. Brümmer, C. Baron, I. Ermilov, M. Freudenberg, D. Kontokostas, andS. Hellmann.DataID: Towards semantically rich metadata for complex datasets.In Proceedings of the 10th International Conference on Semantic Systems,SEM ’14, pages 84–91. ACM, 2014.

Endpoint LinkedDataStack SetupaLocalSPARQL Endpoint...

Documents

Transcript of Endpoint LinkedDataStack SetupaLocalSPARQL Endpoint...

Semantic Wikis OntoWiki - support for agile, distributed knowledge engineering scenarios

Data Sheet: Endpoint Security Symantecâ„¢ Endpoint Protection

Endpoint Protection Solutions – TotalDefense Endpoint r12

Symantec™ Endpoint Protection 12.1 - D2B Informatique · 2020-05-25 · Symantec™ Endpoint Protection 12.1.4 Data Sheet: Endpoint Security Overview Symantec™ Endpoint Protection

McAfee Endpoint Security 10b2b-download.mcafee.com/products/evaluation/Endpoint... · 2016-07-07 · McAfee Endpoint Security McAfee® Endpoint Security is a comprehensive security

set aggressive-mode client-endpoint - · PDF fileset aggressive-mode client-endpoint April 2011 SEC-2123 set aggressive-mode client-endpoint To specify the Tunnel-Client-Endpoint attribute

ESET Endpoint Antivirusdownload.eset.com/manuals/eset_eea_userguide_rus.pdf5 1. ESET Endpoint Antivirus ESET Endpoint Antivirus представляет собой новый подход

Peer-Insights Review Endpoint Protection/Endpoint ... · Peer-Insights Review Endpoint Protection/Endpoint Protection Plus ... (McAfee) Tanium Tripwire ... Quality and availability

Endpoint Exchange

OntoWiki – A Tool for Social, Semantic Collaboration · in a prototypical OntoWiki on the basis of Powl [1], a framework for Semantic Web application development. To stress the

SMALL BUSINESS ENDPOINT PROTECTION · 2018. 8. 20. · Microsoft System Center Endpoint Protection McAfee Small Business Security Panda Endpoint Protection Malwarebytes Endpoint Security

Ivanti Endpoint Manager, Endpoint Security for Endpoint ...

Endpoint Security Fundamentals Part 3: Building the Endpoint Security Program

Enterprise Endpoint Protection - Performance Benchmarkseval.symantec.com/.../downloads/Passmark_Enterprise... · Enterprise Endpoint Protection Performance Benchmarks ... Endpoint

!!! !!! OntoWiki Mobile

WHITEPAPER ENDPOINT DETECTION AND … endpoint detection and response beyond antivirus proactive threat hunting at the endpoint

Endpoint Compliance

Webroot Business Endpoint Protection vs. Seven Competitors · 2020-06-07 · Sophos Endpoint Protection 30 . Webroot Business Endpoint Protection vs. Eight Endpoint Security Products

Fast and Effective Endpoint Security for Business …...Kaspersky Endpoint Security McAfee Total Protection for Endpoint Microsoft System Center 2012 Endpoint Protection Symantec Endpoint

Endpoint Modeling 101 - A New Approach to Endpoint Security