An Engine for Ontology-Based Stream Processing - Theory...
Transcript of An Engine for Ontology-Based Stream Processing - Theory...
![Page 1: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/1.jpg)
An Engine for Ontology-Based Stream ProcessingTheory and Implementation
Christian Neuenstadt
6. Februar 2018Lübeck
![Page 2: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/2.jpg)
Motivation STARQL Transformation Evaluation
Motivation - Use Case
C. Neuenstadt An Engine for Ontology-Based Stream Processing 2/ 30
![Page 3: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/3.jpg)
Motivation STARQL Transformation Evaluation
Query AnsweringSPARQL
C. Neuenstadt An Engine for Ontology-Based Stream Processing 3/ 30
![Page 4: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/4.jpg)
Motivation STARQL Transformation Evaluation
Query TransformationFrom SPARQL to SQL
C. Neuenstadt An Engine for Ontology-Based Stream Processing 4/ 30
![Page 5: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/5.jpg)
Motivation STARQL Transformation Evaluation
Query TransformationFrom SPARQL to SQL
C. Neuenstadt An Engine for Ontology-Based Stream Processing 5/ 30
![Page 6: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/6.jpg)
Motivation STARQL Transformation Evaluation
Query TransformationFrom SPARQL to SQL
C. Neuenstadt An Engine for Ontology-Based Stream Processing 6/ 30
![Page 7: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/7.jpg)
Transformation of Temporal QueriesOntology-Based Stream Processing
![Page 8: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/8.jpg)
Motivation STARQL Transformation Evaluation
The Idea of STARQL[S]treaming and [T]emporal ontology [A]ccess with a [R]easoning-based [Q]uery [L]anguage
• We have developed a new query language for ontology-based streams1) Uses temporal operators on state sequences2) Adopts current ontology standards3) Evaluates multiple streams
• We have implemented a query transformation strategy• We execute transformed STARQL queries in modern database environments
• DBMS for historic data• DSMS for live streams
C. Neuenstadt An Engine for Ontology-Based Stream Processing 8/ 30
![Page 9: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/9.jpg)
Motivation STARQL Transformation Evaluation
A static graph pattern
C. Neuenstadt An Engine for Ontology-Based Stream Processing 9/ 30
![Page 10: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/10.jpg)
Motivation STARQL Transformation Evaluation
A temporal graph pattern
C. Neuenstadt An Engine for Ontology-Based Stream Processing 10/ 30
![Page 11: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/11.jpg)
Motivation STARQL Transformation Evaluation
Graph pattern and streaming data
C. Neuenstadt An Engine for Ontology-Based Stream Processing 11/ 30
![Page 12: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/12.jpg)
Motivation STARQL Transformation Evaluation
Graph pattern and streaming data
C. Neuenstadt An Engine for Ontology-Based Stream Processing 12/ 30
![Page 13: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/13.jpg)
Motivation STARQL Transformation Evaluation
Graph pattern and streaming data
C. Neuenstadt An Engine for Ontology-Based Stream Processing 13/ 30
![Page 14: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/14.jpg)
Motivation STARQL Transformation Evaluation
Graph pattern and streaming data
C. Neuenstadt An Engine for Ontology-Based Stream Processing 14/ 30
![Page 15: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/15.jpg)
Motivation STARQL Transformation Evaluation
From temporal graphs to temporal states
C. Neuenstadt An Engine for Ontology-Based Stream Processing 15/ 30
![Page 16: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/16.jpg)
Motivation STARQL Transformation Evaluation
From temporal graphs to temporal states
C. Neuenstadt An Engine for Ontology-Based Stream Processing 16/ 30
![Page 17: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/17.jpg)
Motivation STARQL Transformation Evaluation
A window operator
SMsmt[NOW − 3s,NOW]→ 1s
C. Neuenstadt An Engine for Ontology-Based Stream Processing 17/ 30
![Page 18: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/18.jpg)
Motivation STARQL Transformation Evaluation
A window operator
SMsmt[NOW − 3s,NOW]→ 1s
C. Neuenstadt An Engine for Ontology-Based Stream Processing 18/ 30
![Page 19: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/19.jpg)
Motivation STARQL Transformation Evaluation
A window operator
SMsmt[NOW − 3s,NOW]→ 1s
C. Neuenstadt An Engine for Ontology-Based Stream Processing 19/ 30
![Page 20: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/20.jpg)
Motivation STARQL Transformation Evaluation
STARQL Example 1 - Threshold
∃i, x(R1(x, i) ∧ x > 93)
Example
1 SELECT ?x2 FROM S_Msmt [NOW-3s, NOW]-> 1s3 WHERE { :tempSensor :mountedAt :GasTurbine }4 HAVING EXISTS ?i IN (GRAPH ?i { :tempSensor :hasVal ?x }5 AND ?x > 93)
C. Neuenstadt An Engine for Ontology-Based Stream Processing 20/ 30
![Page 21: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/21.jpg)
Motivation STARQL Transformation Evaluation
STARQL Example 2 - Monotonic Increase
∀i, j, x, y(R1(sens, x, i) ∧ R2(sens, y, j) ∧ i < j→ x ≤ y))
Example
1 CONSTRUCT GRAPH NOW {:tempSensor rdf:type MonInc }2 FROM S_Msmt [NOW-3s, NOW]-> 1s3 WHERE { :tempSensor :mountedAt :GasTurbine }4 HAVING FORALL ?i, ?j, ?x, ?y IN (5 IF GRAPH ?i { :tempSensor :hasVal ?x }6 AND GRAPH ?j { :tempSensor :hasVal ?y }7 AND ?i < ?j8 THEN ?x <= ?y )
C. Neuenstadt An Engine for Ontology-Based Stream Processing 21/ 30
![Page 22: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/22.jpg)
Motivation STARQL Transformation Evaluation
Transformation of temporal Graph Patterns with STARQL
Static mapping example
?sens :type :Sensor ← SELECT SensorName AS ?sensFROM Sensors (1)
Time based mapping example
GRAPH i { ?sens hasVal ?y } ← SELECT sId as ?sens, val as ?yFROM Slice(Measurement,i,r,sl,st). (2)
i: index of the specific temporal stater: range of the window operatorsl: slide parameter of the window operatorst: sequencing strategy of the sequence generator
C. Neuenstadt An Engine for Ontology-Based Stream Processing 22/ 30
![Page 23: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/23.jpg)
Schematic Transformation of STARQL queries
![Page 24: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/24.jpg)
Motivation STARQL Transformation Evaluation
Comparison of implemented backend examples
PostgreSQL PipelineDB Exareme SparkLive Streams No Yes Yes YesStatic Data Yes Yes Yes YesHistoric Streams Yes No Yes YesAPI JDBC JDBC REST API REST API / built in
“Semantic access to streaming and static data at Siemens”Journal of Web Semantics 2017“Towards Analytics Aware Ontology Based Access to Static and StreamingData” ISWC 2016“OBDA for Temporal Querying and Streams” HiDeSt@KI 2015“A Stream-Temporal Query Language for Ontology Based Data Access”DL 2014 / KI 2014
C. Neuenstadt An Engine for Ontology-Based Stream Processing 24/ 30
![Page 25: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/25.jpg)
Motivation STARQL Transformation Evaluation
Comparison of implemented backend examples
PostgreSQL PipelineDB Exareme SparkLive Streams No Yes Yes YesStatic Data Yes Yes Yes YesHistoric Streams Yes No Yes YesAPI JDBC JDBC REST API REST API / built in
“Semantic access to streaming and static data at Siemens”Journal of Web Semantics 2017“Towards Analytics Aware Ontology Based Access to Static and StreamingData” ISWC 2016“OBDA for Temporal Querying and Streams” HiDeSt@KI 2015“A Stream-Temporal Query Language for Ontology Based Data Access”DL 2014 / KI 2014
C. Neuenstadt An Engine for Ontology-Based Stream Processing 25/ 30
![Page 26: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/26.jpg)
Motivation STARQL Transformation Evaluation
Comparison of implemented backend examples
PostgreSQL PipelineDB Exareme SparkLive Streams No Yes Yes YesStatic Data Yes Yes Yes YesHistoric Streams Yes No Yes YesAPI JDBC JDBC REST API REST API / built in
“Semantic access to streaming and static data at Siemens”Journal of Web Semantics 2017“Towards Analytics Aware Ontology Based Access to Static and StreamingData” ISWC 2016“OBDA for Temporal Querying and Streams” HiDeSt@KI 2015“A Stream-Temporal Query Language for Ontology Based Data Access”DL 2014 / KI 2014
C. Neuenstadt An Engine for Ontology-Based Stream Processing 26/ 30
![Page 27: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/27.jpg)
Motivation STARQL Transformation Evaluation
Comparison of implemented backend examples
PostgreSQL PipelineDB Exareme SparkLive Streams No Yes Yes YesStatic Data Yes Yes Yes YesHistoric Streams Yes No Yes YesAPI JDBC JDBC REST API REST API / built in
“Semantic access to streaming and static data at Siemens”Journal of Web Semantics 2017“Towards Analytics Aware Ontology Based Access to Static and StreamingData” ISWC 2016“OBDA for Temporal Querying and Streams” HiDeSt@KI 2015“A Stream-Temporal Query Language for Ontology Based Data Access”DL 2014 / KI 2014
C. Neuenstadt An Engine for Ontology-Based Stream Processing 27/ 30
![Page 28: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/28.jpg)
Motivation STARQL Transformation Evaluation
Experimental EvaluationPrototypical Implementation
Experiment 1: PostgreSQL / Spark (Historic Data)• Threshold and MonInc query executed on different data volumns• Time scales for larger dataset with INTRAstate comparison• But INTERstate comparisons are expensive!!
Experiment 2: Multi Core Evaluation• Prototypical implementation per window execution based on pl/pgSQL• Reduces data set per execution dramatically for interstate queries• Scales by number of cores• Overhead for each window execution is not applicable to Spark
C. Neuenstadt An Engine for Ontology-Based Stream Processing 28/ 30
![Page 29: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/29.jpg)
Motivation STARQL Transformation Evaluation
Related Work
SRBenchmark Evaluation
Language SPARQLStream C-SPARQL CQELS STARQLSupported queries 17 17 11 11
Missing functionalities of STARQL are: ASK queries(1) and Property Paths(6)
Overall comparisonQuery Language:
• All other three languages handle incoming triples as one graph per window.• Only C-SPARQL accesses timestamps or temporal ordering directly
Transformation:• Only SPARQLstream and STARQL can be transformed to relational algebra• C-SPARQL / CQELS use their own execution environment
C. Neuenstadt An Engine for Ontology-Based Stream Processing 29/ 30
![Page 30: An Engine for Ontology-Based Stream Processing - Theory ...moeller/PhDs/Neuenstadt/Presentation.pdf · “Semantic access to streaming and static data at Siemens” Journal of Web](https://reader036.fdocuments.in/reader036/viewer/2022063018/5fdc8869f7dd2b48b10f54ee/html5/thumbnails/30.jpg)
Motivation STARQL Transformation Evaluation
Summary/Outlook
• We have shown how we can query intra/inter state-based temporal sequenceswith temporal analytics in a new query language with syntax and semantics.
• We defined a new extended query transformation strategy that allows for anexecution on relational DB and streaming systems.
• We executed the transformed queries on large volumns of batch and streameddata successfully and showed their scalability regarding distributed windowexecution.
• Future extensions:1) Extend temporal operators and aggregation functions2) Optimize window execution on backend systems3) Extend ontology language
C. Neuenstadt An Engine for Ontology-Based Stream Processing 30/ 30