Download - SKOS – Shuttle · answer a SPARQL-endpoint of the ones known by SKOS Shuttle2. In the near future, further systems like Sharepoint™ will be also connected. Note: In some cases,

SKOS Shuttle RDF Push Synchronization Tutorial

SemwebLLCCopyright2016–2018Allrightsreserved 1

SKOS–Shuttle

(Welcome)

Tutorial„RDFPushSynchronization“

August2017

InthistutorialyouwilllearnSomespecialaspectsof

SKOSShuttleSynchronizationusingRDFPushNotifications

TableofContentsRDFPUSHNOTIFICATIONS(RPN)........................................................................................................................2CONFIGURINGNOTIFICATIONS...........................................................................................................................3RPNADMINISTRATION.......................................................................................................................................9WORKINGWITHASYNCEDPROJECT.................................................................................................................10SYNCEDCHANGESINPROJECT..........................................................................................................................12

ThesaurusManagementallowsfordevelopmentandmaintenanceofthesauri.



Onceathesaurusis“finished”forthecurrentperiod,itmustbedeployedina“production”environment,whichoperatesit.Howwouldyouensure,yourproductionenvironmentisusingthecurrentversionofyourthesaurus?Onewayistostopthethesaurusoperation,copythecurrentthesaurusversion,startagain.Thiswouldleadtoaninterruptionofserviceduetothismaintenanceoperation.SKOSShuttleallowsyouacontinuoussynchronizationfromyourmaintainedSKOSShuttlethesaurustooneorseveralproductionenvironment(s).RDFPUSHNOTIFICATIONS(RPN)UseCaseImagineyourtaxonomistsaremaintainingaSKOSShuttlethesaurusviaSKOSShuttle’sdetailview.Eachtimeaneditoperationisperformed,anRDFmodificationiscomputedand–locally–executed.Dependingontheconfigurationsettings(seebelow),directlyafteronemodification,aRESTPUSHNotificationissenttoalltheconnectedSPARQL-endpointsrunningintheproductionenvironment.ASemwebRDFPushNotification–shortly“RPN”isaRESTPushcalltoapredefinedserverwhichisinstructedtoreceiveitandprocessit1.Inthecurrentrelease2.7.8–March2017–behindthisURLshouldansweraSPARQL-endpointoftheonesknownbySKOSShuttle2.Inthenearfuture,furthersystemslikeSharepoint™willbealsoconnected.Note:Insomecases,itisadvisablethatRPNbecollectedandsentattheendofthedayoroftheweek.NotificationsaregeneratedautomaticallyandstoredinSKOSShuttledatabase.Theydefineakindof“channel”whereinformationissentto.Inthistutorialwewillspeakthereforeof“notificationchannel”andmeana“notificationreceivingservice”,regularlyaSPARQLendpoint.

1TheoreticallyanyRESTcallablesystem,e.g.Sharepoint™,Alfresco™,Huddle™orSamepage™2Pleaseseehttps://skosshuttle.ch/tutorials/skosshuttle-projectmanagement.pdffordetails,whichSPARQLendpointsareknowninSKOSShuttle.



CONFIGURINGNOTIFICATIONSForeachSKOSShuttleproject,Notificationsbehaviorsmustbeconfigured.Foreveryprojectyoucandefineanynumberofconfigurations(thusforanynumberofreceivingservices).Inordertoaddsome

configurations,switchtoyourprojectsectionbyclickingontheicon inthemenubarandfocusononeproject.Inthistutorialwewillfocusonaprojectwhereanenvironmentalthesaurus–EARTH–isloaded(picture1).

Picture1:ProjectEARTHwithloadedthesaurus

OpentheRPNconfigurationchannelssectionbyclickingontheicon (Picture2)oftheprojectEARTH

andaddanewRPNchannelconfigurationfortheprojectbypressingthe+symbol.

Picture2:ProjectEARTHwithopenedemptyRPNconfigurationsection

OnceyoucreatedanewRPNnotificationsconfigurationobject(Picture3),youhavetoenterdatatosetupthebehaviorofSKOSShuttleateveryeditchangeyoumake.



Picture3:BlankRDFPushNotificationconfigurationforprojectEARTH

ThisconfigurationshouldbeappropriatelyfilledinorderforSKOSShuttletosynchronizeeditchangestothetargetSPARQLendpoint.Descriptionoftheconfigurationcomponents:Configurationofnotificationreceivingservice:Nameoftheconfiguration EnterhereanynameitmightremindsyouwhataboutthisconfigurationisURLofthenotificationreceivingservice

FullURLoftheserviceprocessingtheRDFPushnotifications.GenerallyaSPARQLendpoint,inthefuturealsomoregeneralsystemslikeSharepoint™willbeattached.

GraphURIintargetsystem

IncaseyourtargetnotificationreceivingsystembeaSPARQLendpointandincaseRDFdatatherewereorganizedinaspecialgraph,giveherethefullURIofthatgraph.

UserforURL IncasethenotificationreceivingserviceneedsauthenticationPasswordforuser IncasethenotificationreceivingserviceneedsauthenticationConfigurationofnotificationparameters:Atstatementlevel Checkifyouwishasynchronizationateveryedit.Atentitylevel Checkifyouwishasynchronizationatentitylevel(currentlydisabled).SPARQL-UPDATE CheckifyouwishtoletsendaSPARQL-UPDATEqueryateveryedit.Configurationofnotificationparameters:allegro,blazegraph,graphdb,jena-fuseki,marklogic,stardog,virtuoso.

Checkthecorrespondingtargetnotificationreceivingservice.

Configurationofnotificationfrequency:



None Thenotificationwillbegeneratedbutnotsent.Direct Thenotificationwillbesentdirectlyafteritsgeneration.Hourly Atthebeginningofthehourallunsentnotificationswillbesent.Daily Atthebeginningofthedayallunsentnotificationswillbesent.Weekly Atthebeginningoftheweek(Sunday)allunsentnotificationswillbesent.Monthly Atthebeginningofthemonthallunsentnotificationswillbesent.Actions:Testnotificationaccess Thenotificationwillbegeneratedbutnotsent.Deleteallnotifications AllthenotificationsconfiguredbythisobjectaredeletedfromSKOSShuttledatabase.Flushwaitingnotifications Sendsdirectlyallthe(unsent)notificationsbeforethesettimeinterval.Note:itisadvisablethatthenotificationreceivingsystemsintheproductionaremaintainedbySKOSShuttle.Therearesomecaseswherethisismandatory.

- Changing(adding/deleting)customrelations- Changing(adding/deleting)customattributes- Changing(adding/deleting)OWLclassassignmentstoconcepts.

Oncenotificationsaregenerated,theyappearinsidetherightsidenotificationdisplayqueue(seebelow).WewilldefineherefortheprojectEARTHtwonotificationschannels.ImagineyouareinchargeofmaintainingtheEARTHthesaurusandtosynchronizechangestoanenvironmentdepartmentandtoanenvironmentagency.Wewillhavetodefinetwoconfigurationobjects(onewasalreadycreated)andfillthemwiththechannelinformation(Server,type,frequency,…).AliceSueistheresponsibletaxonomistfortheenvironmentagency:Configurationtoward“EnvironmentalAgency”productionSPARQLserverName EARTH_AT_ENVAGENCYSPARQL-Server https://ch.semweb.ch:9090/bigdata/namespace/EARTH_ENVAG/sparql Servertype BlazegraphUser …Password …Atstatement CheckedSPARQL-UPDATE CheckedFrequency DirectPicture4depictsthecorrespondingconfigurednotificationobjectforEARTH_AT_ENVAGENCY.FortheEnvironmentalDepartment,theconfigurationwillbe:



Configurationtoward“EnvironmentalAgency”productionSPARQLserverName EARTH_AT_ENVDEPSPARQL-Server https://ch.semweb.ch:9090/bigdata/namespace/EARTH_ENVDEP/sparql Servertype BlazegraphUser …Password …Atstatement CheckedSPARQL-UPDATE CheckedFrequency WeeklyPicture5depictsbothconfigurationobjectsforthehypotheticalchannels“EnvironmentAgency”and“EnvironmentDepartment”hostingeachaSPARQLserverwithasynchronizedcontentofEARTHthesaurus.

Picture4:ConfiguredNotificationChannelforEARTHat“EnvironmentAgency”



Picture5:ConfiguredNotificationChannelsforEARTHatboth“EnvironmentAgency”and“EnvironmentDepartment”

Thenextthingtocheck,iswhetherSKOSShuttleviatheheredefinedconfigurationscanaccessthetargetservers.Forthispurpose,pressjustthebutton(ineachconfiguration)“Testnotificationaccess”.IncaseSKOSShuttlecanreachthegiventargetserver,thisisdisplayedbyshowingatestidempotentSPARQL-UPDATEbeingexecuted.Thesetestnotificationsaredisplayedinthenotificationqueuesituatedontherightoftheconfiguration.Seepicture6.Ifthereweresomeproblemsaccessingthetargetserver,SKOSShuttledisplaysamessageboxwiththecorrespondingwarning.



Picture6:TestedNotificationChannelforEARTHat“EnvironmentAgency”

Picture6showsthattheconfiguredparametersallowSKOSShuttletoreachthetargetreceivingRESTservice–hereaBlazegraphSPARQLService.ThetestfunctioninstructsSKOSShuttletosendaSPARQLidempotentquery,whereastatementisassertedandretractedatthesametime.Ifthisoperationsucceeds,thenyouaredone:thechannelisconfiguredandcanbeused.



RPNADMINISTRATIONOncetheTaxonomisthasbeguntoeditthethesaurus,aseriesoforderednotificationsaregeneratedandstoredinSKOSShuttle’sdatabase.Ifthechannelwereconfiguredas“direct”,theneverynotificationisalsodirectlysenttothetargetreceivingserviceandtheserviceresponsestored.Thesenotificationsarevisibleinsidethechannelwheretheywereconfigured.Inordertoseethelast10notificationsinthedatabase,justhoveronthespacewherethenotificationtableisshownanditwillfillup.Therearesomecaseswhereyoumightbeinterestedtodiscoverwhathasbeensentandwhatdidnot–ifyouareaspecialist–andtosendsomenotificationagaintothetargetserviceinordertoreadhowitreacts(thismightbeonecaseofmaintenance).Picture7showswhichoperationsareavailableonstorednotifications.

Picture7:RPNadministrationtable

PleasenotethatRPNsaredisplayedandprocessedinatimeorderedway,mostrecentfirst.

DeleteRPN

Re-sendRPN

Show/hideRPN

SelectRPN

IDofRPN

FilterRPN’scontainingsometext

SetmaxnumberandoffsetforretrievedRPN’s

DeleteselectedRPN’s

Re-sendselectedRPN’s

InvertRPNs’selection

Re-loadRPNstableusingthefilterparameters

ThirdRPN

SecondRPN



WORKINGWITHASYNCEDPROJECTAfterhavingdefinedthesynchronizationchanneltoyourtargetproductionservers,thetaxonomistcanresumeorstartherworkasbefore.ThebreadcrumbofSKOSShuttlewillshowafurtherline,indicatinghowtheprojectissyncedwiththetargetservices(Picture8)–seetheorangerubberband.

Picture8:Detailviewofconcept“CONDITIONS”

EachunsentnotificationissignaledinSKOSShuttlebreadcrumbbydisplayingareddot.Ifforinstanceweaddedanewlinkoranalternativelabeltothisconcept,SKOSShuttlewillgenerateandsendRPNstotheserverattheEnvironmentAgencyanditwillgeneratethesamenotificationsfortheserverattheenvironmentaldepartmentwaitingforbeingsentatthestartofthenextweek.InthiscaseSKOSShuttleaccumulatesunsentnotificationsanditwillsignalthisusingasmalldotclosetotheinvolvedchannel(Pictures9and10).



Picture9:BreadcrumbareadisplayinganotificationthatthereareunsentRPNsfortheprojectafteraddinganalt.label

Picture10:BreadcrumbareadisplayinganotificationthatthereareunsentRPNsfortheproject

Hoveringonthereddotbringsyoutothechannelconfiguration,whereyoucan–ifneeded–analyzesorsendsdiretlytheRPNsusingtheRPNadministration.Sinceforthe“EnvironmentAgency”theRPNwasalreadysent,youwillfindinthetargetserverforEnvironmentAgencynewnotifications,whichproducesanupdateofthecorrespondingRDFstore.



SYNCEDCHANGESINPROJECTWhichchanges(edits,newlinks,etc.)aresynchronizedtothesyncedtargetserversinthechannels?Basicallyallthosechangeswhicharehistorized3plusRDFimports(notSnapshots),seeTable1.ThehistorizationengineofSKOSShuttletriggerstheRPNengineinordertocommunicatetothetargetserverswhathasbeenchanged.Historizedchanges,syncedtothetargetservers:Syncedchange Comment–ifneededInserting/deletingaConceptScheme

Inserting/deletingaConcept

Edits/Changeoflabelsorfields

Creation/Deletionofcustomrelations

Creation/Deletionofcustomattributes

Creation/DeletionofOWLdataproperties

Creation/Deletion/Updateofnamespaces

RDFfileimport ThisappliesRDFimportsatconceptlevelTable1:SyncedchangesinSKOSShuttle

Note:Thepresentedsynchronizationworkswellalsowiththehistorizationengine,thusperformingan“undo”willpropagatethe(inverted)SPARQL-UPDATEquery.Pleasecf.thetutorialonhistorizationCongratulations!Youaredone–inthistutorialyoulearnedhowtoconfigureRDFPushnotificationsinSKOSShuttle!Questions?Improvements?Justcontactus–https://ch.semweb.ch/en-contactoronhttps://skosshuttle.ch

3Seehttps://skosshuttle.ch/tutorials/skosshuttle-historization.pdfformoredetails.