Download - Hands on TMQL

Einführung in Topic Maps

topicmapslab.detopicmapslab.de

Topic Maps Query Language

Sven Krosse M. Sc.

Topic Maps Lab an der Universität Leipzig

[email protected]


topicmapslab.de2

Oct. 2004

– matriculation at the university of cooperative

education at Leipzig

Sep 2007

– graduation in Computer Science with the diploma

Oct 2007

– matriculation at the university of Leipzig

introduce myself


topicmapslab.de3

May 2009

– joining the Topic Maps Lab at the university of

Leipzig

Sep 2009

– finishing my study with the Master of Computer

Science

– the subject of my master's thesis was the

implementation of TMQL4J

introduce myself


topicmapslab.de4

Agenda

1 the Ontopia Topic Maps Engine

1.1 Omnigator – a generic topic maps browser

1.2 Installing Ontopia with all extensions

2 TMQL4J – the first java-based TMQL Engine

2.1 Architecture

2.2 TMQL4Ontopia - Integration into Ontopia


topicmapslab.de5

3 Introduction of the language TMQL

3.1 Why do we need a language like TMQL?

3.2 How can we use TMQL?

3.3 Are there any alternatives for TMQL?

4 TMQL – the basics of the ISO draft

4.1 grammar levels

4.2 literals and references

4.3 navigation

4.4 variables

Agenda


topicmapslab.de6

5 Learning the high-level expressions

5.1 Path Expression

5.2 Select Expression

5.3 Flwr Expression

Agenda



the Ontopia Topic Maps Engine


topicmapslab.de8

the Ontopia Topic Maps Engine

● the topics map engine of the company Ontopia AS

● contains a many tools and utilities for using topic maps

● in summer 2009 the Ontopia topic maps engine

becomes open source

● supported by a big community

● supports the TMAPI 2.0 specification


topicmapslab.de9

Omnigator – generic topic maps browser

● part of the Ontopia engine

● web-based topic maps browser

● displaying information about topics and associations

● running on a Apache Tomcat server


topicmapslab.de10

Omnigator – generic topic maps browser


topicmapslab.de11

Installing Ontopia with all extensions

● needed components

● JRE 5.0 or higher

● Ontopia distribution

● tmql4j library

● tmql4Ontopia library

● a web browser :)



TMQL4J - the first Java-based TMQL Engine


topicmapslab.de13

TMQL4J

● TMQL Querying Engine written in Java

● current version 1.0.0

● abstracts from underlying topic map engine

● creates an abstraction layer on top of the TMAPI

● encapsulate the whole querying process

● implements additional cache mechanism

● simple configuration mechanism


topicmapslab.de14

TMQL4J - architecture

● designed as process chain of independent modules

● each module can replace by a domain-specific implementation

● based modules

– TMQL4J pre-processor

– TMQL4J lexer

– TMQL4J parser

– TMQL4J interpreter

– TMQL4J post-processor


topicmapslab.de15

TMQL4J – process chain

Initialization

environment and system variables

TMQL4J pre-processor TMQL4J parser

TMQL4J interpreter results

TMQL query

parser tree

TMQL4J lexerTMQL4J

post-processor

TMQL lexical tokens

matches transformed results

cleaned TMQL query


topicmapslab.de16


Initialization




TMQL query

parser tree

TMQL4J lexerTMQL4J

post-processor

TMQL lexical tokens


cleaned TMQL query

process flow of the TMQL querying process


topicmapslab.de17


Initialization




TMQL query

parser tree

TMQL4J lexerTMQL4J

post-processor

TMQL lexical tokens


cleaned TMQL query

data flow of the TMQL querying process


topicmapslab.de18

TMQL4J – application architecture

application

TMQL4J

TMAPI

topic maps engine

topic maps backend

direct access

indirect access

TMQL query


topicmapslab.de19

TMQL4Ontopia

● generic query plug-in for the Omnigator

● current version 0.0.1

● use the TMQL4J engine

● implements Ontopia specific modules

● allows result export as CSV file


topicmapslab.de20

result view of TMQL4Ontopia


topicmapslab.de21

parser tree view of TMQL4Ontopia



Introduction of the language TMQL


topicmapslab.de23

Introduction of the language TMQL

● TMQL is a topic maps query language

● part of ISO standardization ( ISO 18048 )

● discussions about parts of the final standard

● supports different styles of querying

● supports the definition of ontology in context of a query


topicmapslab.de24

Why do we need a language like TMQL?

● the number of applications using topic maps increases

fast

● the data base becomes to complex

● complex handling by using TMAPI code fragments

● complex terminology and ontology

● number of topics and associations too huge

● different views of the data in context of different

business processes


topicmapslab.de25

Why do we need a language like TMQL?

● Example TMAPI vs TMQL

→ extract all player names of the association

member_of


topicmapslab.de26

● TMAPI ( for example in Java )

TopicMapSystemFactory factory = TopicMapSystemFactory.newInstance();

TopicMapSystem topicMapSystem = factory.newTopicMapSystem();

TopicMap topicMap = topicMapSystem.getTopicMap("my-topic-map");

// type of the association

Topic type = (Topic)topicMap.getConstructById("member_of");

// extract all association items of the type member-of

TypeInstanceIndex index = topicMap.getIndex(TypeInstanceIndex.class);

for ( Association association : index.getAssociations(type)){

// extract all roles of the association item

for ( Role role : association.getRoles()){

// extract the role player and the names

names.addAll(role.getPlayer().getNames());

}


topicmapslab.de27

● TMQL4J

TopicMapSystemFactory factory = TopicMapSystemFactory.newInstance();

TopicMapSystem topicMapSystem = factory.newTopicMapSystem();

TopicMap topicMap = topicMapSystem.getTopicMap("my-topic-map");

// initialize the tmql4j engine

IQuery query = new QueryImpl("member_of << types >> players / tm:name")

TMQLRuntime runtime = new TMQLRuntime(topicMapSystem,

Arrays.asList(new TopicMap[] { topicMap }), query);

runtime.run();

// extract result set

names.addAll((ITupleSequence<Name>) runtime

.getStoredValue(TMQLRuntime.TMQL_RUNTIME_RESULTPROCESSING_RESULT));


topicmapslab.de28

How can we use TMQL?

● different opportunities

● TMQL4J

● PerlTM

● Hands on ;-)


topicmapslab.de29

Are there any alternatives for TMQL?

● there are other languages for querying topic maps

● Tolog

– a language similar to Prolog

– supports predicate-invocations and boolean

combinations

● Toma

– a language similar to SQL

– developed by Rani Pinchuk


topicmapslab.de30

Are there any alternatives for TMQL?

● but:

● there expressiveness are lower than TMQL

● TMQL supports different query styles for different domains



TMQL - the basics of the current draft


topicmapslab.de32

TMQL – the basics of the ISO draft

● only the basics of the ISO draft

● basics important to understand the syntax of TMQL

● learning high-level expressions by doing


topicmapslab.de33

TMQL – grammar levels

● define the terminals of the language

● based on Regular Expressions

example: $variable | SELECT | FOR

Token Level


topicmapslab.de34


● define the production rules of TMQL

● combination of terminals

example: tuple-expression := <value-expression>

Token Level

Canonical Level


topicmapslab.de35


Token Level

Canonical Level

Non-Canonical Level

● define the shortcuts as term substitutions

anchor / tm:name ==> anchor >> chracteristics tm:name


topicmapslab.de36

TMQL – literals or atoms

● atoms defined as constant literals

● atoms are data-types and operators

● supported data-types are an extract of the CTM data-

types and XSD date and dateTime


topicmapslab.de37

TMQL – literals or atoms

atom possible value

atom Undefined | boolean | number | date | dateTime | string | iri

undefined undef

boolean true or false

number decimal or integer

decimal /[+-]?\d+(\.\d+)?/

integer /[+-]?\d+/

date '-'? yyyy '-' mm '-' dd (zzzzzz)?

dateTime '-'? yyyy '-' mm '-' dd 'T' hh ':' mm ':' ss ('.' s+)? (zzzzzz)?

iri see RFC 3987

string /"([^"]|\")*"/ | /'([^']|\')*'/

source: http://www.isotopicmaps.org/tmql/tmql.html


topicmapslab.de38

TMQL – references

● references used to identify topics in context of a TMQL

query

● supports

– item-identifier

– subject-identifier

– subject-locators

● possible to use relative IRI as identifier or locator


topicmapslab.de39

TMQL – references example

● item-identifier

puccini

● subject-identifier

http://psi.ontopedia.net/Puccini

● subject-locator

http://psi.ontopedia.net/Puccini


topicmapslab.de40

TMQL – navigation

● based on the TMRM

● define 12 axes

● can use as a part of path expression

● symbolize navigation throw the abstract topic map graph

● each axis can used in two directions

● syntax

navigation := axis-name direction optional-type

direction := << | >>


topicmapslab.de41

TMQL – identifier axes

● one axis for each identifier type

● do not supports optional types

● forward-direction

● deliver the identifier of the given topic

● only supports topic items

● backward-direction

● deliver the topic with this identifier

● only supports string literals


topicmapslab.de42

TMQL – identifier axes examples

● item-identifier

http://psi.ontopedia.net/Puccini >> item

● subject-identifier

http://psi.ontopedia.net/Puccini >> indicators

● subject-locator

http://psi.ontopedia.net/Puccini << locators


topicmapslab.de43

TMQL – type-hierarchy axes

● supports is-instance-of and a-kind-of associations


● deliver the types of given instance ( type-instance)

● deliver the supertypes of given type ( supertype-subtype)

● supports topic items or association items


● deliver the instances of given type ( type-instance)

● deliver the subtypes of given type ( supertype-subtype)

● supports topic items or association items


topicmapslab.de44

TMQL – type hierarchy axes examples

● type-instance

http://psi.ontopedia.net/Puccini >> types

http://psi.ontopedia.net/Composer << types

● supertype-subtype

http://psi.ontopedia.net/Composer >> supertypes

http://psi.ontopedia.net/Composer << supertypes


topicmapslab.de45

TMQL – association axes


● deliver all roles of given association ( roles )

● deliver all players of given association ( players )

● deliver all players playing with given topic ( traverse )


● deliver all association items using the given role ( roles )

● deliver all association items using the given player ( players )

● deliver all associations items connected with the given

association by the same player ( traverse )


topicmapslab.de46

TMQL – association axes examples

● roles

http://psi.ontopedia.net/composed_by >> roles

http://psi.ontopedia.net/Composer << roles

● players

http://psi.ontopedia.net/composed_by >> players

http://psi.ontopedia.net/Puccini << players

● traverse

http://psi.ontopedia.net/Puccini >> traverse

http://psi.ontopedia.net/composed_by << traversee


topicmapslab.de47

TMQL – characteristics axis


● deliver all names and occurrences of given topic



● deliver the topic item using the given characteristic

● only supports names and occurrences


topicmapslab.de48

TMQL – characteristics axis examples

● all names of puccini

http://psi.ontopedia.net/Puccini >> characteristics

tm:name

● all occurrences of puccini


tm:occurrence


topicmapslab.de49

TMQL – scope axis


● deliver the scope of the given construct

● supports names, occurrences or associations


● deliver all constructs with the given scope



topicmapslab.de50

TMQL – scope axis examples

● the scope of the names of puccini


tm:name >> scope

● all constructs with the scope web

http://psi.ontopedia.net/Web << scope


topicmapslab.de51

TMQL – reifier axis


● deliver the reifier of the given construct

● supports names, occurrences or associations


● deliver the construct reifies by the given topic item



topicmapslab.de52

TMQL – reifier axis examples

● the construct reified by the topic item A Philatelic

History

http://home.prcn.org/~pauld/opera/ >> reifier

● the reifier of this construct

http://home.prcn.org/~pauld/opera/ >> reifier <<

reifier


topicmapslab.de53

TMQL – atomify axis


● deliver the value of the given characteristics

● supports names or occurrences


● deliver the characteristics with the given value

● supports all literals


topicmapslab.de54

TMQL – atomify axis examples

● the value of the name puccini

http://psi.ontopedia.net/Puccini >>

characteristics tm:name >> atomify

or

http://psi.ontopedia.net/Puccini / tm:name


topicmapslab.de55

TMQL – variables

● using in context of high-level expression

● supported by select expression and flwr expression

● different prefixes

$ only bind literals or items

@ only bind a tuple

% only bind a sequence


topicmapslab.de56

TMQL – variables

Tuple

Tuples are ordered collections of simple values (atoms and items).

Tuple-Sequence

Tuple sequences are sequences of tuples where all tuples have identical length. Tuple sequences can be generated with tuple expressions.


topicmapslab.de57

TMQL – protected variables

● variable names can post-fixed by different number of primes '

● variables only differ in number of primes

● can not be bind to the same value


topicmapslab.de58

TMQL – variables example

● variables

$composer // can be bind to puccini

@composers // can be bind to a tuple

%composers // can be bind to a sequence


topicmapslab.de59

TMQL – variables example

● tuple

( puccini , "puccini" , “1900-01-01” )

● sequence

{ ( puccini , "puccini" , “1900-01-01” ) ,

( fontane , "fontane" , “1900-01-01” ) }



TMQL - the high-level expressions

- learning by doing -


topicmapslab.de61

TMQL – Path Expression

● combination of navigation steps or predicate invocations

● predicate-invocation

– describe associations

– similar to tolog


topicmapslab.de62

TMQL – predicate-invocation example

composed_by ( composer : puccini , opera : le_villi , … )


topicmapslab.de63



association type


topicmapslab.de64



role type


topicmapslab.de65



player


topicmapslab.de66



ellipsis


topicmapslab.de67

TMQL – Path Expression exercises

Task

All names of all instances of composer


topicmapslab.de68


Solution

composer << types >> characteristics tm:nameor

http://psi.ontopedia.net/Composer << types >> characteristics tm:name

Task

All names of all instances of composer


topicmapslab.de69


Task

The names of all topis playing the role composer in association composed_by.


topicmapslab.de70


Solution

composed_by >> players composed >> characteristics tm:nameor

http://psi.ontopedia.net/composed_by >> players http://psi.ontopedia.net/Composer >> characteristics tm:name

Task

The names of all topis playing the role composer in association composed_by.


topicmapslab.de71

TMQL – filter and projection

● filters

– post-fixed after navigation

– surrounded by square brackets

– supported filters

– index [ number ]

– index-range [ number .. number ]

– types [ ^ type ]

– scope [ @ scope ]

– boolean-expressions [ … ]


topicmapslab.de72

TMQL – filter and projection

● projection

– post-fixed after navigation

– surrounded by round brackets

– defined as a tuple-expression

– simple navigation from current node

– current node symbolize by dot


topicmapslab.de73

TMQL – filter and projection example

● the first name of puccini

http://psi.ontopedia.net/Puccini >> characteristics tm:name [0]

● Projection of characteristics and associations played by

instances of composer

http://psi.ontopedia.net/Composer << types

( . >> characteristics , . << players )


topicmapslab.de74


composer << types ( . >> characteristics , . << players )

navigation


topicmapslab.de75



projection definition


topicmapslab.de76



current node


topicmapslab.de77



projection


topicmapslab.de78


Task

Create a projection of all name and all occurrence values of all instances of composer.


topicmapslab.de79


Solution

composer << types ( . / tm:name , . / tm:occurrence ) or

http://psi.ontopedia.net/Composer << types( . / tm:name, . / tm:occurrence )

Task

Create a projection of all name and all occurrence values of all instances of composer.


topicmapslab.de80


Task

All names of all instances of composer in scope short name.


topicmapslab.de81


Solutioncomposer << types >> characteristics tm:name @ short_name

or http://psi.ontopedia.net/Composer << types >> characteristics

[ @ http://psi.ontopedia.net/short_name ]

Task

All names of all instances of composer in scope short name.


topicmapslab.de82


Task

Extract the characteristics of all composers and filter them by the type date_of_death.


topicmapslab.de83


Solutioncomposer << types >> characteristics [ ^ date_of_death ]

orhttp://psi.ontopedia.net/Composer << types >> characteristics

[ ^ http://psi.ontopedia.net/date_of_death ]

Task

Extract the characteristics of all composers and filter them by the type date_of_death.


topicmapslab.de84

TMQL – Select Expression

● similar to SQL

● supports variables


topicmapslab.de85

TMQL – Select Clause

● only non-optional clause of a select expression

● starts with the keyword SELECT

● contains a number of value expressions

– variables

– navigations

– functions


topicmapslab.de86

TMQL – Select Clause example

SELECT http://psi.ontopedia.net/Puccini / tm:name ,

http://psi.ontopedia.net/Puccini / tm:occurrence


topicmapslab.de87

TMQL – Where Clause

● optional clause of a select expression

● starts with the keyword WHERE

● contains a number of boolean-expressions

– exists-clause

– forall-clause

– negation

– conjunction

– disjunction


topicmapslab.de88

TMQL – Where Clause example

SELECT $composer

WHERE $composer ISA http://psi.ontopedia.net/Composer


topicmapslab.de89

TMQL – Exists Clause● specifies the number of items satisfying a condition

● numerically unrestricted

– number of satisfying items is not important

– starts with keyword EXISTS or SOME

– can contain path expressions

● numerically restricted

– define the number of satisfying items as upper or lower

bounds

AT LEAST number set SATISFIES boolean-expression

AT MOST number set SATISFIES boolean-expression


topicmapslab.de90

TMQL – Exists Clause example

SELECT $composer

WHERE EXISTS $composer ISA http://psi.ontopedia.net/Composer

SELECT $composer

WHERE AT LEAST 4 $opera IN // http://psi.ontopedia.net/Opera

SATISFIES http://psi.ontopedia.net/composed_by

( http://psi.ontopedia.net/Composer : $composer ,

http://psi.ontopedia.net/Work : $work )


topicmapslab.de91

TMQL – Forall Clause

● all items have to satisfy the condition

EVERY variable IN content SATISFIES boolean-expression


topicmapslab.de92

TMQL – Forall Clause example

SELECT $composer

WHERE

EVERY $opera IN $composer << traverse

SATISFIES $opera == http://psi.ontopedia.net/Le_Villi


topicmapslab.de93

TMQL – boolean combination

● negation symbolized by the keyword NOT

● conjunction symbolized by the keyword AND

● disjunction symbolized by the keyword OR


topicmapslab.de94

TMQL – boolean combination example

SELECT $composer

WHERE NOT ( $composer ISA

http://psi.ontopedia.net/Composer

)SELECT $composer , $opera

WHERE

$composer ISA http://psi.ontopedia.net/Composer

AND

$opera ISA http://psi.ontopedia.net/Opera


topicmapslab.de95

TMQL – From Clause


● starts with the keyword FROM

● define the context of the select expression

● have to return a topic map or sequence of constructs


topicmapslab.de96

TMQL – From Clause example

SELECT $composer

FROM // http://psi.ontopedia.net/Musician



topicmapslab.de97

TMQL – Unique Clause


● only the keyword UNIQUE

● unify the result set


topicmapslab.de98

TMQL – Unique Clause example

SELECT $composer



UNIQUE


topicmapslab.de99

TMQL – Order-By Clause


● starts with the keyword ORDER BY

● define an order of the variable bindings of the where clause


topicmapslab.de100

TMQL – Order-By Clause example

SELECT $composer



ORDER BY $composer / tm:name


topicmapslab.de101

TMQL – Limit and Offset

● optional clauses of a select expression

● starts with the keyword LIMIT or OFFSET

● define the selection window after querying


topicmapslab.de102

TMQL – Limit and Offset example

SELECT $composer



ORDER BY $composer / tm:name

OFFSET 2

LIMIT 10


topicmapslab.de103

TMQL – Select Expression exercises

Task

Select all operas composed by puccini.


topicmapslab.de104


Solution

SELECT $opera WHERE composed_by ( composer : puccini , work : $opera )or

SELECT $opera WHERE http://psi.ontopedia.net/composed_by ( http://psi.ontopedia.net/Composer : http://psi.ontopedia.net/Puccini ,

http://psi.ontopedia.net/Work : $opera )

Task



topicmapslab.de105


Task

Select the names of all composers who composed at least one opera. Keep in mind that the work should not be an opera.


topicmapslab.de106


Solution

SELECT $composer / tm:name WHERE $opera ISA opera AND AT MOST 3 SATISFIES composed_by ( work : $opera , composer : $composer )

orSELECT $composer / tm:name WHERE $composer ISA

http://psi.ontopedia.net/Composer AND SOME $opera IN // http://psi.ontopedia.net/Opera SATISFIES

http://psi.ontopedia.net/composed_by ( http://psi.ontopedia.net/Work : $opera , http://psi.ontopedia.net/Composer : $composer )

Task

Select the names of all composers who composed at least one opera. Keep in mind that the work should not be an opera.


topicmapslab.de107


Task

Select the topic types which are a kind of composer or where each instance have at least one name.


topicmapslab.de108


Solution

SELECT $type WHERE $type AKO composer OREXISTS $type << types / tm:name

orSELECT $type WHERE $type AKO http://psi.ontopedia.net/Composer OR

EXISTS $type << types / tm:name

Task

Select the topic types which are a kind of composer or where each instance have at least one name.


topicmapslab.de109

TMQL – Flwr Expression

● similar to a programming language

● supports variables


topicmapslab.de110

TMQL – Return Clause

● only non-optional clause of a flwr expression

● starts with the keyword RETURN

● contains a number of value expressions

– variables

– navigations

– functions

– XTM and CTM fragments


topicmapslab.de111

TMQL – Return Clause example

RETURN http://psi.ontopedia.net/Puccini / tm:name


topicmapslab.de112

TMQL – For Clause

● optional clause of a flwr expression

● starts with the keyword FOR

● can contain more than one for clause

● define a variable set


topicmapslab.de113

TMQL – For Clause example

FOR $composer IN // http://psi.ontopedia.net/Composer

RETURN $composer


FOR $opera IN // http://psi.ontopedia.net/Opera

RETURN $composer, $opera


topicmapslab.de114

TMQL – Where Clause and Order-By

● optional clauses of a flwr expression

● same syntax and meaning like as a part of select expressions


topicmapslab.de115

TMQL – Where Clause and Order-By


FOR $opera IN // http://psi.ontopedia.net/Opera

WHERE

http://psi.ontopedia.net/composed_by(

http://psi.ontopedia.net/Composer : $composer ,

http://psi.ontopedia.net/Work : $opera)

ORDER BY $composer >> item

RETURN $composer, $opera


topicmapslab.de116

TMQL – Flwr Expression exercises

Task



topicmapslab.de117

TMQL – Flwr Expression exercises

Solution

FOR $opera IN opera WHERE composed_by ( composer : puccini , work : $opera )RETURN $opera

orFOR $opera IN http://psi.ontopedia.net/Opera WHERE

http://psi.ontopedia.net/composed_by ( http://psi.ontopedia.net/Composer : http://psi.ontopedia.net/Puccini , http://psi.ontopedia.net/Work : $opera )

RETURN $opera

Task




Thank you for your attention!