3 - Specifications and Standards Quick Reference...

LearningAnalyticsCommunityExchange

SpecificationsandStandards-QuickReferenceGuidePublicWorkingDocument

By:AdamCooper

Published:31December2015

Keywords:learninganalytics,interoperability,standards,specifications

Thisdocumentpresentsalistofstandardsandspecifications,includingresearchwork,whichmayberelevanttopeoplebuildinglearninganalyticssystems.Abriefsummaryofthecapabilitiesofeachispresented,alongwithnotesonadoptionto-date.Theaiminwritingthisguideistoraiseawarenessofexistingtechnicalspecifications,andtosupportaprocessofduediligencethroughexplorationofpriorartwhenlearninganalyticssystemsaredesigned.

Audience:thisisarelativelytechnicaldocumentaimedatreaderswithexperienceinsoftwaredevelopmentandarchitecture,ordevelopmentofinteroperabilitystandards,etc.

SpecificationsAndStandards-QuickReferenceGuide

Contents

1. Introduction...................................................................................................................................1

BackgroundtoLearningAnalyticsInteroperability...........................................................................1

ANoteonTerminology:interoperability,specifications,standards.............................................1

TheMotivationforaQuickReferenceGuide....................................................................................1

StatusofThisDocumentandCalltoContribute...............................................................................2

2. HowtoUsethisQuickReferenceGuide........................................................................................2

OnlineCounterpart,theMendeleyGroup........................................................................................2

StructureoftheGuide.......................................................................................................................2

EvaluatingSpecificationsandStandards...........................................................................................4

3. TheQuickReferenceGuide...........................................................................................................6

Specificationstosupportanalytics,independentofdomainofapplication.....................................6

DataExchange...............................................................................................................................6

ModelsandMethods.....................................................................................................................8

Specificationsdealingwithhumanactivitygenerally........................................................................9

Logging...........................................................................................................................................9

Communication...........................................................................................................................11

Specificationsdealingwithlearning-relatedactivity,orrootedineducationortraining...............11

ObjectivesandAssessment.........................................................................................................11

Biographical.................................................................................................................................13

ContextualandReferential..........................................................................................................14

Multi-purposeSpecifications...........................................................................................................14

OtherSpecifications.........................................................................................................................16

Privacy.........................................................................................................................................16

Protocols......................................................................................................................................16

4. GlossaryofAbbreviations............................................................................................................16

5. References...................................................................................................................................18

6. About..........................................................................................................................................26


1

1. Introduction

BackgroundtoLearningAnalyticsInteroperabilityTheInstituteofElectricalandElectronicsEngineers(IEEE)definesinteroperabilitytobe:

“theabilityoftwoormoresystemsorcomponentstoexchangeinformationandtousetheinformationthathasbeenexchanged.”

AslearninganalyticsmovesfrombeingaresearchtopictoanICT-supportedserviceatscale,thepropertyofinteroperabilitybecomesmoreimportant.TheLACEBriefing“LearningAnalyticsInteroperability–TheBigPictureinBrief”[1]outlinesarangeofbothshort-termandlong-termbenefitsarisingfromincreasedinteroperability.

ANoteonTerminology:interoperability,specifications,standardsAstheIEEEdefinitionaboveindicates,“interoperability”isapropertyacollectionofsystems.Itmaybeachievedusingarangeoftechnicalandnon-technicalmeans,withwrittentechnicalspecifications–datadefinitions,HTTPRESTAPIs1,XMLbindings,etc.–onlybeingpartoftherecipeforachievingpracticalinteroperability.Thefocusofthisdocumentisonthesetechnicalspecifications.

Itiscommonfortheterm“standard”tobeappliedtosomekindsoftechnicalspecifications,butthereisnouniformityofusage;differentorganisationshavedifferentdefinitionsofwhattheyconsidertobeastandard,andthereismuchloosenessinterminologywhenthewordisusedoutsideformalstandardisationprocesses.Thisdocumentdoesnotadoptahard-and-fastdefinitionofwhatcomprisesa“standard”andtakesaninclusiveviewofthekindsoftechnicalspecificationthatareinscope,capturedinthetitleas“specificationsandstandards.”

TheMotivationforaQuickReferenceGuideTheaimoftheauthorinwritingthisguideistoraiseawarenessofexistingtechnicalspecifications,andtosupportaprocessofduediligencethroughexplorationofpriorartwhendesigninglearninganalyticssystems.Theguideisintendedtoreducetheincidenceofunintentionalinventionordeviationfromwhatalreadyexists.Theauthordoesnotassumethatanyofthesespecificationsarefitforuseastheystand.Inallcases,itwillbenecessarytoundertakeanapplication-andcontext-specificevaluation(seethesection“EvaluatingSpecificationsandStandards”,below),anditmaywellbethecasethatanexistingspecificationprovidesonlysomeinspirationforanewapproach,orthedefinitionsforsomeconcepts.Suchinspirationanddefinition-borrowingmovesusinthedirectionofevolved,andpossiblyconvergent,thinking.Ittakesusinthedirectionofinteroperability,evenifnottothedestination.

Thisguideisacatalogue2oftechnicalspecificationsthatmaybeofrelevancetoimplementationsoflearninganalytics.Foreach,itsetsouttoprovideabriefsummaryofthescopeandpurposeofthetechnicalspecification,alongwithreferencestoevidenceofimplementations,experiencesfromuse,

1Ingeneral,abbreviationsfortechnicaltermswillnotbeexpandedoninthetext.Aglossaryisprovidedas2Anothersourceofinformation,supportedbytheOpenKnowledgeFoundationandorientedtowardsopendataisawebpageentitled“Web-orientedDataFormats”and,maybefoundathttp://dataprotocols.org/data-formats/.


2

critique,etc.Evidencefromuseforlearninganalyticsisprioritised,andtheapproachtakenisoneofprovidingindicativeevidenceratherthanacomprehensiveaccount.

StatusofThisDocumentA“publicdraftforcomment”ofthisdocumentwaspublished9December2014.Thedraftwasreleasedasauseful,butincomplete,workinthehopethatitwill:

1. bringexistingworkofrelevancetolearninganalyticsinteroperabilitytotheattentionofmorepeople;

2. stimulatecontributionfrompeoplewithknowledgeandexperience;and3. stimulateexperimentationandreportingontheresultsoftrials.

BasedonfeedbackfromthecommunityandworkintheInteroperabilityworkpackageoftheLACEprojectthisdocumenthasbeenupdated(byToreHoel)tothecurrentversion.

Alsointhisversionoftheguide,evidenceisfrequentlylacking.Thisispartiallyaconsequenceofthestageofinvestigation;however,theshortageofevidenceisalsocontributedtobyafailuretopublishtheresultsofexperimentation–bothsuccessesandfailures-orduetoabsenceofpracticalexperienceofusingthespecificationinalearninganalyticssystem.

2. HowtoUsethisQuickReferenceGuide

OnlineCounterpart,theMendeleyGroupTheGuideisanaugmentedsnapshotofthecontentsoftheMendeleyLearningAnalyticsInteroperabilityGroupError!Bookmarknotdefined..TheLACEprojecthasaddedtothecontentsintheprocessofongoingdeskresearch.TheMendeleyGroupreferencesarebrowsableby“tags”thatmatchthestructureofthisdocument,butalsousestagstodistinguishbetweentechnicalspecifications,consensus-based“standards”,evidence,andtoidentifythespecificationbyname/abbreviation3.Inthiscase,thelabel“standards”indicatesasubjectiveassessmentofwhetherornotatechnicalspecificationhasbeenarrivedatbysomekindof,maybelimited,consensusprocess.Theauthorfindsthisdistinctiontobethemostusefulprimafaciewayofprioritisingspecificationsforreview(butseethecommentsonevaluation,below).

StructureofthisGuideThisGuideincludesspecificationswithgeneralapplicabilitytoanalyticsthroughtothosethatarespecifictothedomainoflearning,education,andtraining.Withinthisspectrumarespecificationsthatcapturehumanactivitybutarenotpurelyrelevanttolearninganalytics.Thesedistinctionshaveinformedtheapproachtostructuringthecatalogueofspecifications,whichborrowsfromthebroadcategoriesoutlinedinthe“BigPicture”[1].Thestructureisasfollows,anditshouldbenotedthatthetermsinquotationmarksmatchtagsusedintheMendeleyGroup.

3Thetagsusedaredocumentedathttp://bit.ly/1wmc7Gc


3

Specificationstosupportanalytics,independentofdomainofapplication

• “Dataexchange”-generalpurposeformatsandstructuresforaccessingortransferringdata,includingcomputedresults,predictions,andstatisticalmetadata.Thiscategoryisonlyusedwhennodomain-specificcategorywouldbesensible.

• “Modelsandmethods”-thespecificationdealswithanalyticalmethod(algorithmic,orworkflow)andmodelsproduced(statisticalormachine-learning).

Specificationsdealingwithhumanactivitygenerally

• “Logging”–inwhichhighlevelsemanticsarespecified,possiblywithanoptiontoincludeapplication-specificvocabularies.Thiscoverseverythingfromatomicactivitycapturethroughtocaptureofcompoundkindsofactivity.Technology-levellogssuchasweb-serversproducearenotincluded.Thiscategoryisreservedforcaseswherethespecificationisnottargetedataparticularkindofactivity4.

• “Communication”-thefocusofattentionisactivityinforums,chat,collaborativeediting,etc,andtherearespecificreferencestothesemanticsofsuchactivitiesinthespecification.

Specificationsdealingwithlearning-relatedactivity,orrootedineducationandtraining

• “Objectivesandassessment”-thereisspecificreferencetothesemanticsofassessment,includingstatementsofachievement,inthespecification.Thiscategoryalsoincludesintendedlearningoutcomes,inferredknowledge,datapertainingtotheassessmentinstrument,andalsotorecordsofself-regulatedlearning.

• “Biographical”–thespecificationexpressesdataaboutpeopleotherthantheiractions.Someofthiswouldbedescribedas"demographic"forpopulation-levelstudies.Thisincludes,forexample,socialrelations.

• “Contextualandreferential”-thisisaverylooseclassofdatathatmayassistintheinterpretationofactivitydata(e.g.identificationofthecoursebeingstudied,theaccessdeviceused)andwhichmaysometimesexistwithoutanyoneengagingintrackedlearning(e.g.learningdesigns,nationalcurriculumstandards).Contentmetadatawouldappearinthiscategory,butresourcemetadataspecificationsarenotcurrentlyincludedintheGuide.

OtherSpecifications

Thesedonotnaturallyfitintotheabovestructurebutarenonthelessofrelevancetolearninganalytics:

• “Privacy”–specificationsconcernedwithdisclosureanduseofperson-relatedinformation.GeneralinformationsecurityspecificationsarenotinscopeforthisGuide.

• “Protocol”–thespecificationincludes,orlargelydealswith,themethodsbywhichdataisexchanged,i.e.APIs.

4Specificationsareclassifiedandpresentedinthemostspecialisedclasswhenthiscanbedetermined.Forexample,althoughthereisaclearconceptualhierarchymovingfromdataexchange->logging->resourceuse->objectivesandassessment,QTIResultsonlyappearsin“objectivesandassessment”.


4

EvaluatingSpecificationsandStandardsThisGuidehasintentionallyavoidedpresentinganyvalue-ladencommentsonthelistedspecificationsforseveralreasons:

1. Therewillusuallybeanumberoffactorsarisingfromtheapplicationinquestion,andthecontextofuse,whichdeterminerelevance.

2. TheGuideisintendedtosupportactivityacrossaspectrumrangingfromadoptionwithintenttoconformtoanexistingspecification,throughtogaininginspirationforatechnicaldesign.

3. Thestrengthofevaluation,andhowitisparameterised,shoulddependonrisk,andriskvariesbymanyordersofmagnitudesbetweenaprivatelearning-experimentandamulti-millionEuro/Dollarinvestment.

Inpractice,itwillusuallybenecessarytomakedecisionsabouttherelevanceofatechnicalspecificationinaricherwaythansimplyaskingwhetherornotitmeetsaparticularsetofcriteriainadefinitionof“astandard”.Forhighrisksettings,theCommonAssessmentMethodforStandardsandSpecifications(CAMSS)[2]providesausefultemplate.Thiswasdesignedforselection,andpossiblymandation,ofstandardsbyEuropeangovernments;thetypicalscaleofcostforadoption,andhostilityfrommanyinthesoftwareindustryandthepress,demandsadefensivestance.Itisused,forexample,bytheUKGovernmentOpenStandardsBoard5.Thecurrentauthorisrathercautiousaboutchecklist-basedstandardsevaluation,particularlyintheabsenceofapilotprojecttoproperlyunderstandtheissues,althoughchecklistsprovideausefulsetofpromptsofaspectstoconsider[3].

Keyquestionstoasktypicallyrevolvearound:

1. Copyright,licenceandpatentconsiderations.Whatfreedomsdoyouhave?Thelowestriskscenarioisthatallcontributorstoaspecificationhaveexplicitlywaivedallrights,andtheIPRisownedbyanorganisationwithlong-termsustainabilityandacommitmenttofindanentitytoinheritspecificationsshouldtheywinddown,andwithaliberallicencetoimplementthespecification.Realityisoftensomedistancefromthisidealscenario,andtherearealsoissueswhereanovertly-acceptableideasuchasFRAND(fair,reasonableandnon-discriminatory)termsarelimitinginpractice(inthiscaseitisthatOpenSourceSoftwareenshrinesthefreedomtore-distributewhichisatoddswithrequirementsforroyaltypayment,nomatterhow“reasonable”).

2. TechnicalQualitymaybebestjudgedfromexperiencebutusefulproxiesinclude:theextentofparticipationinthespecificationdevelopmentandthedegreetowhichthereisevidenceofmeetingrequirementsfrommultiplestakeholders(whichisonereasonwhyconsensusprocessingeneralandOpenStandards6areimportant);theexistenceofindependentimplementations;evidenceofuseinanequivalentcontextandforasimilarapplicationtotheoneforwhichthespecificationisbeingevaluated.

3. Levelofadoption.Sincethepointofadoptionistopromoteinteroperability,valuescalesassomefunctionofadoption.Thisisnottosaythatwideadoptionisahardrequirement;anexistingspecificationwithsomeevidenceoftechnicalqualityislikelytobeagoodpointof

5https://gds.blog.gov.uk/2013/05/17/open-standards-board-sets-the-direction/6See,forexamplethe“OpenStandPrinciples”,http://open-stand.org/about-us/principles/


5

referenceforaone-offsystemintegration,nomatterhowwidelyimplemented,andanewspecificationthatsolvesrealproblemsmayquicklybecomewidelyimplemented.

4. Stabilityandpersistence.Thisisoftencoupledwiththepreviousthreepoints;awidelyadoptedspecificationfoundedoncollaborationamongmultiplestakeholderswithclearIPRandtransparentlygovernedistrustable.Aspecificationfrom,andlargelyusedby,asingleentityshouldbetreatedwithcautionbecausethereisminimalbarriertochangeorabandonment.

5. Fittoyourtechnicalarchitecture,althoughevenincasesofpoorfit,itmaybepossibletoborrowandadaptpatterns,datadefinitions,etc.

6. Fittoyourorganisationalandoperationalcontext.Whatassumptionsdoesthespecificationmakeaboutworkflow,patternsofactivity,roles,etc?


6

3. TheQuickReferenceGuideNote:thedescriptionsbelowaretakenfromthespecificationwhenasuitableabstractisavailable.

Specificationstosupportanalytics,independentofdomainofapplication

DataExchangeThesedataexchangespecificationscollectivelycoverawiderangeofapplicationscenariosfrombeinginputformatsforbatchdataminingthroughtofeedinglivedatatodashboardwidgets,foruseaspublishingformatsorforarchivalandaudit,etc.

ARFF–Attribute-RelationFileFormatAnARFF(Attribute-RelationFileFormat)fileisanASCIItextfilethatdescribesalistofinstancessharingasetofattributes.ARFFwasdevelopedbytheMachineLearningProjectattheDepartmentofComputerScienceofTheUniversityofWaikatoforusewiththeWekamachinelearningsoftware.ARFFfilescanbetreatedasCSVoncetheheadermetadataisremoved.

TechnicalSpecification[4]EvidentialRepositoryofmorethan600ARFFdatasets[5]SoftwareARFForiginatedwithWeka,asuiteofmachinelearningsoftwaredevelopedattheUniversityofWaikato[6]Alsosupportinseveralothertools,e.g.R[7]

CSV+-ModelforTabularDataandMetadataontheWebTabulardataisroutinelytransferredonthewebas"CSV",butthedefinitionof"CSV"inpracticeisveryloose.Thisdocumentoutlinesabasicdatamodelorinfosetfortabulardataandmetadataaboutthattabulardata.Italsocontainssomenon-normativeinformationaboutabestpracticesyntaxfortabulardata,formappingintothatdatamodel,tocontributetothestandardisationofCSVsyntaxbyIETF.Variousmethodsoflocatingmetadataarealsoprovided.

TechnicalW3CWorkingDraft[8]EvidentialTheWorkingDraftisbackedupbyacompilationofusecasesandrequirements[9]

DSPL–GoogleDataSetPublishingLanguageADSPLdatasetisabundlethatcontainsanXMLfileandasetofCSVfiles.TheCSVfilesaresimpletablescontainingthedataofthedataset.TheXMLfiledescribesthemetadataofthedataset,includinginformationalmetadatalikedescriptionsofmeasures,aswellasstructuralmetadatalikereferencesbetweentables.DatasetsdescribedinthisformatcanbeprocessedbyGoogleandvisualizedintheGooglePublicDataExplorer.

TechnicalDSPLXMLSchema[10]EvidentialGooglePublicDataExplorerusesDSPL[11]

JSON-statTheJSON-statformatisasimplelightweightJSONformatfordatadissemination.Itisbasedinacubemodelthatarisesfromtheevidencethatthemostcommonformofdatadisseminationisthetabular

TechnicalSpecification[12]Evidential


7

form.Inthiscubemodel,datasetsareorganizedindimensions.Dimensionsareorganizedincategories.

AdoptersofJSON-statincludestatisticsagenciesintheUK,Norway,Sweden,Catalunya,andGalicia7.SoftwareJavaScripttoolkit[13]sRpackage[14]

GraphMLGraphMLisacomprehensiveandeasy-to-usefileformatforgraphsbasedonXML.Itconsistsofalanguagecoretodescribethestructuralpropertiesofagraphandaflexibleextensionmechanismtoaddapplication-specificdata.ThisdocumentspecifiessyntaxandprocessingrulesfortheGraphMLlanguagecore(structurallayer)andtwoGraphMLextensionsthatallowaddingbase-typeattributesandparsinginformation.

TechnicalSpecificationversion1.1[15]EvidentialImplementedinSNAPP8[16]SoftwareGephi9implementationofGraphML[15]

Thereareseveralotherformatsforsocialnetworkdata,GraphMLhasbeenselectedbecauseofitspopularityandsupportintoolsalreadyusedforlearninganalytics.

ODataTheOASISODataTC[TechnicalCommittee]workstosimplifythequeryingandsharingofdataacrossdisparateapplicationsandmultiplestakeholdersforre-useintheenterprise,Cloud,andmobiledevices.AREST-basedprotocol,ODatabuildsonHTTP,AtomPub,andJSONusingURIstoaddressandaccessdatafeedresources.Itenablesinformationtobeaccessedfromavarietyofsourcesincluding(butnotlimitedto)relationaldatabases,filesystems,contentmanagementsystems,andtraditionalWebsites.ODataconsistsofasuiteofspecifications.ODatawasformerlyaMicrosoftOpenSpecificationPromisespecification.

TechnicalWorkinggrouphomepage[17]EvidentialUseinQuestionmarkOpenAssessmentPlatform[18]SoftwareCatalogueofODataproducersandconsumers[19]

PiwikReportingAPIThePiwikReportingAPIprovidesaccesstotypicalwebanalyticsreportdata.NB:thisisasinglesupplierspecificationbutPiwikisOpenSourceSoftware.

TechnicalReportingAPIdeveloperdocumentation[20]EvidentialPilotprojectusingPiwikforalearninganalyticsdashboard[21]

RDFDataCubeVocabularyTherearemanysituationswhereitwouldbeuseful Technical

7Linkstotheseexamplesmaybefoundathttp://json-stat.org/8SNAPPistheSocialNetworksAdaptingPedagogicalPracticetool,whichperformsreal-timesocialnetworkanalysisandvisualizationofdiscussionforumactivity.http://www.snappvis.org/9Gephiisacross-platforminteractivevisualizationandexplorationplatformforallkindsofnetworksandcommonlyusedforsocialnetworkvisualisation.http://gephi.github.io/


8

tobeabletopublishmulti-dimensionaldata,suchasstatistics,onthewebinsuchawaythatitcanbelinkedtorelateddatasetsandconcepts.TheDataCubevocabularyprovidesameanstodothisusingtheW3CRDF(ResourceDescriptionFramework)standard.ThemodelunderpinningtheDataCubevocabularyiscompatiblewiththecubemodelthatunderliesSDMX(StatisticalDataandMetadataeXchange),anISOstandardforexchangingandsharingstatisticaldataandmetadataamongorganizations.TheDataCubevocabularyisacorefoundationwhichsupportsextensionvocabulariestoenablepublicationofotheraspectsofstatisticaldataflowsorothermulti-dimensionaldatasets.

W3CRecommendation[22]EvidentialStudyonusingRDFDataCubeforLinkedOpenData,includingreferencestolearninganalytics[23]Listofimplementationsandconformancereports[24]

SDMX–StatisticalDataandMetadataeXchangeTheStatisticalDataandMetadataExchange(SDMX)initiativesetsstandardsthatcanfacilitatetheexchangeofstatisticaldataandmetadatausingmoderninformationtechnology,withanemphasisonaggregateddata.SDMXissponsoredbytheBankforInternationalSettlements,theEuropeanCentralBank,EUROSTAT(StatisticalOfficeoftheEuropeanUnion),theInternationalMonetaryFund,theOrganizationforEconomicCo-operationandDevelopment,theUnitedNations,andtheWorldBank.ThespecificationsincludeXMLandUN/EDIFACTsyntaxes,aregistryspecificationandwebserviceguidelines.

TechnicalSDMXversion2.1[25]Evidential/SoftwareSDMXimplementations[26]

Seealso:PAR(returnofpredictionstopartnerinstitutions).

Inadditiontothesewaysofexchangingthedata,therearealsoseveralspecificationsintendedtobeusedtodescribedatasetsordatasetcatalogues,suchasDCAT[27]andaproposaltoWebSchemas[28].Thesehavepotentialrolesforarchivalandpublishing.

ModelsandMethodsPMML–PredictiveModelMarkupLanguageThePredictiveModelMarkupLanguage(PMML)isanXML-basedlanguagewhichprovidesawayforapplicationstodefinestatisticalanddataminingmodelsandtosharemodelsbetweenPMMLcompliantapplications.

TechnicalPredictiveModelMarkupLanguagev4.2[29]EvidentialAwhitepaperonstandardsinpredictiveanalytics[30]OpenAcademicAnalyticsInitiative(OAAI)projectfindings[31]PaperonOAAIinJournalofLearningAnalytics[32]Aneducationaldataminingexample[33]SoftwarePMMLissupportedbyarangeofsoftwarefromdesktop,e.g.Knime,tocloud-based,e.g.ZementisPMMLpackageforR[34]PMMLpre-processingandGooglepredictionAPI[35]


9

VTL–ValidationandTransformationLanguage(SDMX)ThepurposeoftheVTListoallowaformalandstandarddefinitionofalgorithmstovalidatestatisticaldataandcalculatederiveddata.Thevalidationisassumedtobeaparticularcaseoftransformation,thereforetheterm“Transformation”ismeanttobemoregeneralandtoincludealsothevalidation.

TechnicalVersion1.0publishedin2015[36]

Specificationsdealingwithhumanactivitygenerally

LoggingActivityStreamsThisspecificationdetailstheserializationofastreamofsocialactivitiesusingtheJSONorAtomformats.Activitiesareimportantinthattheyallowindividualstoprocessthelatestnewsofpeopleandthingstheycareabout.

TechnicalVersion1.0specifications(JSONandAtom)[37]JSONActivityStreams2.0[38]EvidentialActivityStreamsprovidessomeofthefoundationforxAPI(seebelow).LearningRegistryParadataSpecification[39]Seealso“Additionalresources”,below.SoftwareActivityStreams2.0JavaReferenceImplementation[40]

CaliperAnalyticsIMSCaliperAnalyticsprovidesLearningMetricProfilesasstandardiseddescriptionsofactionsandrelatedcontexts.LearningSensorAPIscommunicatewithLearningEventsStoretoaggregatemetrics.ThespecificationleveragesexistingIMSspecifications,likeLearningToolInteroperability(LTI)specification,LearningInformationServicespecification,andQuestion&TestInteroperabilityspecification

TechnicalPublisheddocumentsandcode:IMSCaliperAnalyticsBackground;IMSCaliperAnalyticsBestPracticeGuide;IMSCaliperAnalyticsImplementationGuide;IMSCaliperAnalyticsConformanceandCertificationGuidev1.0,andCaliperAnalyticsv1PublicReposforSensorAPIs[97]EvidentialIMSGlobaloffersConformanceCertificationformembers.SoftwareCaliperEventStoreisavailableasaGitHubdownload

CAM–ContextualisedAttentionMetadataContextualizedAttentionMetadata(CAM)enablesthecaptureofuserinteractionswithlearningenvironments,allowingmodellingofauser'shandlingofdigitalcontentacrosssystem

TechnicalCAMSchema[41]Evidential


10

boundaries.AsCAMwasdevelopedtodescribeasmanytypesofattentionmetadataaspossible,CAMrecordsofausercannotmerelydescribetheuser'sfociofattentionbutratherhisentirecomputerusagebehaviour.CAMcanbeanalysedtoprovideanoverviewaboutwhere(i.e.withwhichapplication)andwhenanactiontakesplaceandwhathappensintheenvironment.CAManalysesenablethediscoveryofpopularity,usageburstsandtrendsoftools.Itcanalsouncoverpatternslikeapplicationsbecomingunpopular,risingstarsandnewapplicationstakingoverolderapplications.Informationaboutwhenanactiontakesplacecanbeusefulincontrolledenvironmentssuchasformallearningenvironmentswhereactivitiesareusuallyscheduled.

Atrust-basedrecommenderforteachers[42]UseinaPersonalLearningEnvironment[43]Bookchapter[44]Seealso“Additionalresources”,below.

TMF–PittsburghScienceofLearningDataShopTutorMessageFormatThisguideisintendedforasoftwaredeveloperwhowantstoevaluate,implement,orupdatelogginginaneducationaltutoring[ITS]application,orconvertexistinglogscreatedbyatutoringapplication.Inaddition,adeveloperworkingonalearningenvironmentcoulddescribetheapplication'seventsintheformatdescribedinthisdocument.

TechnicalSpecificationv4[45]EvidentialDataShop,anonlinerepositoryandanalyticaltoolsusingTMF[46]BookchapteronDataShop[47]

xAPI-ExperienceAPITheExperienceAPI[alsoknownasTinCanAPI]specificationdescribesadatamodelandwebservicethatallowsstatementsofexperiencetobedeliveredtoandstoredsecurelyinaLearningRecordStore(LRS).Thesestatementsofexperiencearetypicallylearningexperiences,buttheAPIcanaddressstatementsofanykindofexperience.TheExperienceAPIisdependentonActivityProviderstocreateandtracktheselearningexperiences;thisspecificationprovidesadatamodelandassociatedcomponentsonhowtoaccomplishthesetasks.

TechnicalVersion1.0.2specification[48]TinCanRegistryofvocabularies[49]EvidentialxAPIhasbeenwidelyimplemented,includingsomepilotsfocussingonlearninganalytics[21][50]

XES–eXtensibleEventStreamEventlogs,astheyoccurinpracticeandresearch,cantakeaplethoraofdifferentformsandinstantiations.Everysystemarchitecturethatincludessomesortofloggingmechanismhassofardevelopedtheirown,insular,solutionforthistask.XESisanXML-basedstandardforeventlogs.Itspurposeistoprovideagenerally-acknowledgedformatfortheinterchangeofeventlogdatabetweentoolsandapplicationdomains.Itsprimarypurposeisforprocessmining,i.e.theanalysisof

TechnicalXESv2.0[51]EvidentialIEEECISTaskForceonProcessMiningincludeactivitytostandardiseXES[52].DraftofOctober2015ispublised10SoftwareProMv6.4(ProcessMiningFramework)[53]

10http://www.win.tue.nl/ieeetfpm/doku.php?id=shared:20151002_xes


11

operationalprocessesbasedontheireventlogs.However,XEShasbeendesignedtoalsobesuitableforgeneraldatamining,textmining,andstatisticalanalysis.

ProMhasbeenusedforlearninganalyticsbutwithafore-runnerdataexchangeformat[54]

Additionalresourcescomparingseveraloftheabove,discussingofrequirementsandissues,describingprototypes,etc:

• “Aggregatingsocialandusagedatasetsforlearninganalytics:dataorientedchallenges”[55]• “AnOverviewofUsageDataFormatsforRecommendationsinTEL”[56]• “ASurveyonLinkedDataandtheSocialWebasfacilitatorsforTELrecommendersystems”

[57](alsoreferstoSIOC,seebelow)

CommunicationAtomSyndicationFormatAtomisanXML-baseddocumentformatthatdescribeslistsofrelatedinformationknownas"feeds".Feedsarecomposedofanumberofitems,knownas"entries",eachwithanextensiblesetofattachedmetadata.Forexample,eachentryhasatitle.TheprimaryusecasethatAtomaddressesisthesyndicationofWebcontentsuchasweblogsandnewsheadlinestoWebsitesaswellasdirectlytouseragents.

TechnicalIETFRFC4287[58]EvidentialAtomiswidelyused(seee.g.ActivityStreams,OData,Leap2A)

SIOCTheSIOC(Semantically-InterlinkedOnlineCommunities)CoreOntologyprovidesthemainconceptsandpropertiesrequiredtodescribeinformationfromonlinecommunities(e.g.,messageboards,wikis,weblogs,etc.)ontheSemanticWeb.

TechnicalSIOCCoreOntologySpecification[59]EvidentialListofSIOCimplementations(2009)[60]Bookchapteronlinkeddataforrecommendersystems[57]

ArgumentationInterchangeFormat[61],which“representsaconsensus‘abstractmodel’establishedbyresearchersacrossfieldsofargumentation,artificialintelligenceandmulti-agentsystems”mayberelevanttospecialcases.

SeealsoActivityStreamsandxAPI

Seealsothesection“Multi-purposeSpecifications”,below.

Specificationsdealingwithlearning-relatedactivity,orrootedineducationortraining

ObjectivesandAssessmentEuroLMAI–EuropeanLearnerMobilityAchievementInformationTheEuropeanLearnerMobilityAchievement Technical


12

InformationModel(EuroLMAI)aspirestoaddresstheidentifiedneedforaharmonizedsolutiontotherecordingandexchangeoflearnermobilityinformationwithintheEuropeanEducationArea.TheresultsofthisworkwillcontributetotheefforttowardsinteroperableEuropean-wideITsystemsthatmanageandexchangelearnerachievementinformation.TheEuroLMAIworkhasitsrootsintheEuropeantransparencytoolsandespeciallyintheEuropassframeworkforthetransparentdescriptionofqualificationsandcompetences.Europassprovidesthecommonbasisforthewell-structuredrecordingofalllife-longlearningopportunitiestaken,includingEuropeanHigherEducationstructuresandlearners'privateandinstitution-ownedinformation.

CENWorkshopAgreement[62](CWA,openaccess)EuropeanStandard[63](basedonCWA,feedue)EvidentialOutlineofusewithUKHigherEducationAchievementReport[64]

IMSLIS(LearnerInformationServices)OutcomesManagementThisdocumentcontainstheIMSGlobalOutcomesManagementServicev1.0InformationModel.ThisserviceisusedtoexchangeinformationaboutOutcomes.TheOutcomesdatamodelisbaseduponresultsthatcanbegroupedtogethertoreflectanassessmentactivityforaCourse.TheOutcomesdatamodelisbaseduponLineItems,ResultsandResultValues.Thebusinesstransactionsincludethesimplecreate,read,update,deleteandsimplesearchingoftheOutcomesdatamodel.Thisdocumentcontainsthedefinitionoftheabstractapplication-programminginterfacefortheOutcomesManagementService.

TechnicalVersion1datamodelandAPI(fullLISmodel)[65]Publicdraftv1.0ofadata-compatibleserviceforusewithIMSLTI[66]EvidentialImplementationsincludeBlackboard11andOraclePeopleSoft12.

IMSQTI(QuestionandTestInteroperability)ResultsReportingThisdocumentisareferenceguidetothedatamodelforreportingtheresultsofanassessment,andprovidesdetailedinformationaboutthemodelandspecifiestheassociatedrequirementsondeliveryengines.

TechnicalVersion2.1specification[67]Evidential-

InLOC–IntegratingLearningOpportunitiesandCompetencesInLOCprovidesamodelfortheinformationdefiningbothintendedLearningoutcomesandWorkcompetences(LOCs).Thatinformationisimportanttopersonal,professionalandvocationaldevelopment,humanresourcesandemployeeperformancemanagement,trainingandeducation,

TechnicalCENWorkshopAgreement(part1)[68]EvidentialPython/Djangosourcecode[69]

11https://help.blackboard.com/en-us/Learn/9.1_2014_04/Administrator/070_Server_Management_and_Integrations/System_Integration/Student_Information_System_%28SIS%29/030_SIS_Integration_Types/030_LIS12http://www.oracle.com/us/industries/education-and-research/056908.pdf


13

whetherintheworkplaceorinschool,vocationalorhighereducation.InLOChelpswiththemanagementandexchangeoflearningoutcomeandcompetenceinformation,bydefiningcommoncharacteristicsoflearningoutcomesandcompetencesandmodellingtheminformatsthatcanbeshared.

OpenBadges13–MozillaOpenBadgeSpecification(Assertions)OpenBadgesisaninitiativebytheMozillaFoundationtohelprecognizeandsupportlifelonglearningthroughabadgeecosystem.Assertionsarerepresentationsofanawardedbadge,usedtoshareinformationaboutbadgesthatyou'veearnedwiththeBackpack.Withthe1.0releaseoftheOBI(OpenBadgeInfrastructure),therearetwotypesofassertions:hostedandsigned.

TechnicalAssertionsspecification[70]EvidentialSourcecodeforOpenBadgeInfrastructureonGitHub[71]

ThereisalsoaworkshoppaperonHumanPerformanceMarkupLanguage[72],butnopublictechnicalspecificationisavailable.


BiographicalFOAF–FriendofaFriendFOAFisaprojectdevotedtolinkingpeopleandinformationusingtheWeb.Regardlessofwhetherinformationisinpeople'sheads,inphysicalordigitaldocuments,orintheformoffactualdata,itcanbelinked.FOAFintegratesthreekindsofnetwork:socialnetworksofhumancollaboration,friendshipandassociation;representationalnetworksthatdescribeasimplifiedviewofacartoonuniverseinfactualterms,andinformationnetworksthatuseWeb-basedlinkingtoshareindependentlypublisheddescriptionsofthisinter-connectedworld.FOAFdoesnotcompetewithsocially-orientedWebsites;ratheritprovidesanapproachinwhichdifferentsitescantelldifferentpartsofthelargerstory,andbywhichuserscanretainsomecontrolovertheirinformationinanon-proprietaryformat.

TechnicalFOAFVocabularySpecification0.99[73]EvidentialAtrust-basedrecommenderforteachers[42]

IMSLIS(LearnerInformationServices)PersonThisdocumentcontainstheIMSGlobalPersonManagementServicev2.0.1InformationModel.Thisserviceisusedtoexchangeinformationaboutindividualsincludingname,address,etc.Thebusinesstransactionsincludethesimplecreate,read,updateanddeleteofthePersondatamodelforasingleinstance.Thecorrespondingdatamodel

TechnicalSpecificationv2.0.1[74]EvidentialSeeentryforIMSLIS(LearnerInformationServices)OutcomesManagement

13http://openbadges.org/


14

fromtheIMSGlobalLearnerInformationPackage(LIP)v1.0specificationhasbeencombinedwiththeoriginaldatamodelinversion1oftheIMSGlobalPersonManagementServicespecification.


ContextualandReferentialASN–AchievementStandardsNetworkVocabularyTheAchievementStandardsNetworkCoreRDFnamespaceprovidesURIsforvocabularytermsusefulinRDFamarkupandindevelopmentofapplicationprofilesinthemannerdefinedbyDublinCore.CustomApplicationProfilesforcompetencyframeworksthatmeetparticularnationalorprojectneedscanbedesignedusing:(1)propertiesandclassesfromtheASNCorenamespace,(2)locallydefinedsub-classesandsub-propertiesoftheASNterms;and(3)otherRDFschemasandontologiesusefulinthecontextoftheapplicationprofile.

TechnicalClassandtermdefinitions[75]EvidentialUSeducationalstandardsinASNform[76]AustraliannationalcurriculuminASNform[77]

CEF–CurriculumExchangeFormatThisEuropeanStandardisapplicabletothedigitalexchangeofinformationabouttermsorconceptsrelatingtocurriculuminformation.Thisincludesvaluestobeusedinmetadatatodescribelearningresourcesandlearnerprofiles.ThemainusesofCEFinstancesandrelatedservicesareexpectedtobetheprovisionof:controlledvocabularies;navigationstructures;additionalcurriculuminformation;mappings.

TechnicalEuropeanStandard[78]Evidential-

Seealso:ActivityStreams,xAPI,IMSLISPerson,andInLOC.


Multi-purposeSpecificationsThissub-sectioncontainsspecificationsthatsupportmorethanoneofthecategories:objectivesandassessment,biographical,contextualandreferential,andcommunication.National/statedatasetdefinitions,asusedbylocalandnationalgovernmenteducationdepartments(e.g.CEDS[79]andCBDS[80]),arealsotypicallymulti-purposeandexpressdataattheleveloftheindividual.ThesearenotincludedinthisGuidebecausetheytypicallymodeldataaccordingtolocaleducationsystemidiosyncrasies,andbecausecomparisonwouldbeaconsiderableexercisewithminimalbenefits.Havingsaidthis,forasingle-jurisdictionimplementation,thesenationalstandardsarevaluablesourcesofguidanceaboutinformationthatislikelytobeavailable,andofrelativelyhighquality,intheITsystemsofeducationalestablishments.


15

Leap2A–ePortfolioPortabilityandInteroperabilityLeap2ArepresentsportfolioinformationasanAtomfeed,withoptionalattachments.Itpermitsexchangeofthreedifferentkindsofinformationthatareoftenincludedinportfolios:digitalartefactsmadeorjointlymadebytheportfolioholderinformationabouttheportfolioholder,theirabilities,achievements,experiences,activities,goals,plansandsuchlike;thingswrittenthatarenotspecificallyaboutoneofthethingsabove—thesemayincludeblogposts,comments,reflections,etc.

TechnicalLeap2Aspecification[81]EvidentialImport/exportsupportedinMaharaePortfolio[82]

MOOCdbMOOCdbisashareddatamodelstandardforthedataemanatingfromMassiveOpenOnlineCourses.Thedatamodelisplatformagnosticandisbasedonsomebasiccoreactionsthatstudentstakeonanonlinelearningplatform.Studentsusuallyinteractwiththeplatforminfourdifferentmodes:Observing,Submitting,Collaboratingandgivingfeedback.

TechnicalTheMOOCdbschema[83]EvidentialWorkshopproceedingsdescribingthedevelopmentofMOOCdb[84]

OAAI(OpenAcademicAnalyticsInitiative)DatasetFormatThisisthedatasetdocumentationfromtheOpenAcademicAnalytics(OAAI)Project.OAAIdeveloped,deployedandreleasedanopen-sourceecosystemforacademicanalytics,designedtoincreasestudentcontentmastery,semester-to-semesterpersistenceanddegreecompletioninpostsecondaryeducation.

TechnicalProjecttechnicaldocument[85]EvidentialPaperonOAAIinJournalofLearningAnalytics[32]

PAR–PredictiveAnalyticsReportingFrameworkDataCookbookThePARFrameworkisanon-profitmulti-institutionaldataminingcollaborativecomprisedoftwo-year,four-year,public,proprietary,traditionalandprogressiveinstitutionscontributingtheiranonymizedstudentdataandexpertisetoidentifycommonfactorscontributingtostudentlossandfindeffectivepracticesthatmeasurablyimprovestudentmomentumandprogressioninU.S.highereducation.

TechnicalCookbook[86]EvidentialThedatacookbookisanoperationaltoolforPAR.

Thereareanumberofotherresourcesthatdonotmeritanentryoftheirown,eitherbecausethepublishedworklackssufficientdetail,orbecausetheydescribeasingleimplementationorresearchactivity:

• Chatti,Dyckhoffetal[87][88]describethedevelopmentanduseofacross-platformdatamodeltosupportlearninganalytics,butgivenotechnicaldetails.

• Merceronetal[89]describeadatastructuretobeusedacrossLMSplatformstoexpressbasicactivitydata.


16

• TheOpenEdXDatastage[90]describesalosslesstranslationfromtheOpenEdxplatformtrackinglogstoarelationaldatabasestructure.

OtherSpecifications

PrivacyPMRM-PrivacyManagementReferenceModelandMethodologyThePrivacyManagementReferenceModelandMethodology(PMRM)addressestherealityoftoday’snetworked,interoperablecapabilities,applicationsanddevicesandthecomplexityofmanagingpersonalinformation(PI)acrosslegal,regulatoryandpolicyenvironmentsininterconnecteddomains.

TechnicalPrivacyManagementReferenceModelandMethodology(PMRM)V1.0[91]Evidential-

AlthoughPMRMisnotaninteroperabilityspecification,itisincludedintheQuickReferenceGuidebecauseofthesignificanceofthetopicofprivacywhenbuildinginteroperablesystems.OASISisalsothevenueforsomerelatedwork,the“PrivacybyDesignDocumentationforSoftwareEngineers(PbD-SE)”TechnicalCommittee,whichhasrecentlypublisheddraftprivacygovernanceanddocumentationstandardsforsoftwareengineers[92].

UMA–User-ManagedAccessUser-ManagedAccess(UMA)isaprofileofOAuth2.0.UMAdefineshowresourceownerscancontrolprotected-resourceaccessbyclientsoperatedbyarbitraryrequestingparties,wheretheresourcesresideonanynumberofresourceservers,andwhereacentralizedauthorizationservergovernsaccessbasedonresourceownerpolicy.

TechnicalUser-ManagedAccess(UMA)ProfileofOAuth2.0[93]EvidentialUserManagedAccess(UMA)CaseStudies[94]

ProtocolsIMSLearningToolsInteroperabilityTheIMSLearningToolsInteroperability™(LTI)specificationenablestheimplementationofsystemsinwhichremotetoolsandcontentareintegratedintoaLearningManagementSystem(LMS).Version1.xisthenextiterationoftheBasicLTIspecificationthatwasreleasedinMay2010,whileversion2.xaddsmoresophisticationtothespecifiedinteractions.

TechnicalVersion1.1.1(Final)[95]Version2.0(Final)[96]EvidentialNumerousv1implementationscertifiedbyIMS14Pilotuseforlearninganalytics[21]

SeealsothesespecificationswhichincludeAPIdescription:OData,xAPI,IMSLISPerson,andIMSLISOutcomes.

4. GlossaryofAbbreviationsAnoteaboutaudience:thisisarelativelytechnicaldocumentaimedatreaderswithexperienceinsoftwaredevelopmentandarchitecture,ordevelopmentofinteroperabilitystandards,etc.Thisglossaryisintendedtoassistreaderswhoarerelativelylesstechnicalinreadingthedocument,ratherthanprovidingfulldefinitions.

14http://www.imsglobal.org/cc/statuschart.cfm


17

Abbreviationsforspecificationsandstandardsthataredescribedinthebodyofthetextarenotincluded.

API ApplicationProgrammingInterface,themeansbywhichsoftwarecomponentsexchangedataordirectprocessing.

CEN theEuropeanCommitteeforStandardisation.

CSV CommaSeparatedValues(alsogeneralisedtoCharacterSeparatedValues),asimpletextualrepresentationoftabulardata.

CWA CENWorkshopAgreement,apre-standardisationconsensusdocumentproducedbyaCENWorkshop,anexpert-basedforum.

HTTP HyperTextTransportProtocol,defineshowmessagesbetweendevicesonthewebareformattedandexchanged“onthewire”.

IEEE theInstituteofElectricalandElectronicsEngineers,whichisresponsibleforsomestandards.

IETF theInternetEngineeringTaskForce,along-establishedinternetstandardisationbodywithanopenprocessbywhichpeoplesubmit“RequestsforComment(RFC)”totheIETFcommunityandselectRFCsarepublishedasde-factostandards.

IMS shortforIMSGlobalLearningConsortium,amembershiporganisationthatissuesconsortiumstandards.

ISO theInternationalOrganisationStandarization,thestandardsbodyrecognisedbytheUN.InformationtechnologystandardsaredevelopedbyISO/IECJTC1,aJointTechnicalCommittee(JTC)withtheInternationalElectrotechnicalCommission(IEC).

ITS IntelligentTutoringSystem.

JSON JavaScriptObjectNotation,atextualhuman-readable,andlight-weightexchangeformatfavouredbywebdevelopersbecausethedataisimmediatelyaccessibletoJavaScriptcodeinawebpage.

OAuth anopenstandardforauthorisationovertheweb,widelyusedtoprovidesecuredelegatedaccesswithoutdisclosingpasswordstothird-partyservices(e.g.toallowanapptoaccessyourtwitteraccount).

RDF ResourceDescriptionFramework,afamilyofW3Cstandardsforexpressinginformationonthewebinamachine-readableway.RDFiscorecomponentofthe“semanticweb”.

RDFa ResourceDescriptionFrameworkinAttributes,aW3CstandardforexpressingRDFconceptsinvariouskindsofweb-nativedocuments(typicallywebpages).

REST REpresentationalStateTransfer,anarchitecturalstyleforAPIsthatexploitsthearchitectureoftheweb,notablytheuseofHTTPandURLs.


18

UN/EDIFACT UnitedNations/ElectronicDataInterchangeForAdministration,CommerceandTransport,anapproachtodataexchangethatpre-datesXMLandJSON.

URI UniformResourceIdentifier,themeansbywhichresourcesareidentifiedontheweb.Inpracticalapplications,theURIsusedareactuallyURLs.

URL UniformResourceLocator,aparticularkindofURIthatgivesthe“address”ofaresourceontheweb.AURLisalsoknownasa“webaddress”.

W3C theWorldWideWebConsortium,theprincipalorganisationdevelopingstandardsfortheweb.

XML eXtensibleMarkupLanguage,atextualandhuman-readablerepresentationofstructuredinformation.An“XMLbinding”definesthewayconceptualdataelementsarerepresentedinXML,andmaybeexpressedinamachine-readableformknownasanXMLSchemausingXMLSchemaLanguage.

5. References

[1] A.R.Cooper,“LearningAnalyticsInteroperability-TheBigPictureinBrief,”Mar.2014[Online].Available:http://www.laceproject.eu/publications/briefing-01.pdf

[2] “CAMSSAssessmentCriteria,”2012.[Online].Available:https://webgate.ec.europa.eu/fpfis/mwikis/idabc-camss/index.php/CAMSS_Assessment_Criteria.[Accessed:06-Nov-2014]

[3] A.CooperandS.Wilson,“EvaluatingStandards–ADiscussionofPerspectives,IssuesandEvaluationDimension,”2009[Online].Available:http://wiki.cetis.ac.uk/images/e/e7/Evaluating_Standards_Public_v1p0.doc

[4] G.Paynter,L.Trigg,I.Witten,E.Frank,andR.Kirkby,“Attribute-RelationFileFormat(ARFF),”2008.[Online].Available:http://www.cs.waikato.ac.nz/ml/weka/arff.html.[Accessed:09-Apr-2014]

[5] Various,“MachineLearningDataSetRepository.”[Online].Available:http://mldata.org/repository/tags/data/arff/.[Accessed:06-Nov-2014]

[6] “Weka3-DataMiningwithOpenSourceMachineLearningSoftwareinJava.”[Online].Available:http://www.cs.waikato.ac.nz/ml/weka/.[Accessed:06-Nov-2014]

[7] RCoreTeam,“foreign:ReadDataStoredbyMinitab,S,SAS,SPSS,Stata,Systat,Weka,dBase,...”2014[Online].Available:http://cran.r-project.org/package=foreign

[8] J.TennisonandG.Kellogg,“ModelforTabularDataandMetadataontheWeb(W3CWorkingDraft),”2014.[Online].Available:http://www.w3.org/TR/tabular-data-model/.[Accessed:08-Apr-2014]


19

[9] J.Tandy,D.Ceolin,andE.Stephan,“CSVontheWeb:UseCasesandRequirements,”2014.[Online].Available:http://www.w3.org/TR/2014/WD-csvw-ucr-20140701/.[Accessed:17-Nov-2014]

[10] Google,“DatasetPublishingLanguageXMLSchema,”2012.[Online].Available:https://developers.google.com/public-data/docs/schema/dspl9

[11] “GooglePublicDataExplorer.”[Online].Available:http://www.google.com/publicdata/directory

[12] X.Badosa,“Format.JSON-stat,”2013.[Online].Available:http://json-stat.org/format/.[Accessed:09-Apr-2014]

[13] X.Badosa,“JSON-statJavaScriptToolkit.”[Online].Available:https://github.com/badosa/JSON-stat

[14] A.SchumacherandH.Malmedal,“ReadandwriteJSON-statdatasets.”2014[Online].Available:http://cran.r-project.org/package=rjstat

[15] “Gephi:GraphMLFormat.”[Online].Available:http://gephi.github.io/users/supported-graph-formats/graphml-format/.[Accessed:31-Oct-2014]

[16] “SNAPP-SocialNetworksAdaptingPedagogicalPractice.”[Online].Available:http://www.snappvis.org/?page_id=4

[17] “OASISOpenDataProtocol(ODATA)4.0,”2014.[Online].Available:https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=odata

[18] S.Lay,“WhatisOData,andwhyisitimportant?|GettingResults--TheQuestionmarkBlog,”Questionmarkblog,2013.[Online].Available:http://blog.questionmark.com/what-is-odata-and-why-is-it-important.[Accessed:06-Nov-2014]

[19] “Ecosystem|OData-TheProtocolforRESTAPIs.”[Online].Available:http://www.odata.org/ecosystem/.[Accessed:06-Nov-2014]

[20] “ReportingAPIReference-PiwikAnalytics-DeveloperDocs.”[Online].Available:http://developer.piwik.org/api-reference/reporting-api.[Accessed:09-Apr-2014]

[21] A.BergandA.Zeckoski,“Letthestudentactivitystreamsflow-Apereo2013,”2013.[Online].Available:https://docs.google.com/presentation/d/1svLBrNIlMQFYHUjB17GxURqk7CgZVltk1XAXmGP619A/edit#slide=id.gc75c19b5_4_18.[Accessed:06-Nov-2014]

[22] R.(ed)CyganiakandD.(ed)Reynolds,“TheRDFDataCubeVocabulary,”W3CRecommendation,2014.[Online].Available:http://www.w3.org/TR/vocab-data-cube/.[Accessed:09-Apr-2014]

[23] E.Daga,M.D’Aquin,A.Gangemi,andE.Motta,“Earlyanalysisanddebuggingoflinkedopendatacubes,”inSecondInternationalWorkshoponSemanticStatistics,2014[Online].Available:https://semstats2014.files.wordpress.com/2014/10/semstats2014_submission_2.pdf


20

[24] “DataCubeImplementations-GovernmentLinkedData(GLD)WorkingGroupWiki.”[Online].Available:http://www.w3.org/2011/gld/wiki/Data_Cube_Implementations.[Accessed:17-Nov-2014]

[25] “StatisticalDataandMetadataeXchangev2.1,”2011.[Online].Available:http://sdmx.org/?page_id=10.[Accessed:09-Apr-2014]

[26] “SDMXImplementations.”[Online].Available:http://sdmx.org/?page_id=14.[Accessed:06-Nov-2014]

[27] F.Maali,J.Erickson,andP.Archer,“DataCatalogVocabulary(DCAT)-W3CRecommendation,”2014.[Online].Available:http://www.w3.org/TR/vocab-dcat/.[Accessed:17-Nov-2014]

[28] “WebSchemasDatasetsProposal.”[Online].Available:http://www.w3.org/wiki/WebSchemas/Datasets.[Accessed:11-Nov-2014]

[29] DMG,“PredictiveModelingMarkupLanguagev4.2.”[Online].Available:http://www.dmg.org/pmml-v4-2.html.[Accessed:09-Apr-2014]

[30] J.Taylor,“StandardsinPredictiveAnalytics,”2014.[Online].Available:http://decisionmanagementsolutions.com/standards-in-predictive-analytics

[31] J.Baron,E.Lauria,andS.M.Jayaprakash,“OpenAcademicAnalyticsInitiative,”2013.[Online].Available:https://confluence.sakaiproject.org/pages/viewpage.action?pageId=75671025

[32] S.M.Jayaprakash,E.W.Moody,E.J.M.Lauria,J.R.Regan,andJ.D.Baron,“EarlyAlertofAcademicallyAt-RiskStudents:AnOpenSourceAnalyticsInitiative,”J.Learn.Anal.,vol.1,no.1,pp.6–47,2014.

[33] S.Ventura,C.Romero,andC.Hervás,“AnalyzingRuleEvaluationMeasureswithEducationalDatasets:AFrameworktoHelptheTeacher,”inEducationalDataMining2008:1stInternationalConferenceonEducationalDataMining,Proceedings,2008,pp.177–181[Online].Available:http://www.educationaldatamining.org/EDM2008/uploads/proc/18_Ventura_4.pdf

[34] G.Williams,T.Jena,W.C.Lin,M.Hahsler,Z.Inc,H.Ishwaran,U.B.Kogalur,andR.Guha,“GeneratePMMLforvariousmodels.”CRAN,2014[Online].Available:http://cran.r-project.org/package=pmml

[35] “PMMLPreprocessing-GooglePredictionAPI—GoogleCloudPlatform.”[Online].Available:https://cloud.google.com/prediction/docs/pmml-schema.[Accessed:31-Oct-2014]

[36] ValidationandTransformationLanguage(VTL)version1.0.2015.[Online].Available:https://sdmx.org/?page_id=5096

[37] J.M.Snell,M.Atkins,W.Norris,C.Messina,M.Wilkinson,andR.Dolin,“ActivityStreamsSpecifications1.0,”2011.[Online].Available:http://activitystrea.ms/specs/


21

[38] J.M.Snell,“JSONActivityStreams2.0,”2014.[Online].Available:http://tools.ietf.org/html/draft-snell-activitystreams-08.[Accessed:21-May-2014]

[39] “LearningRegistryParadataSpecificationV1.0,”2011.[Online].Available:https://docs.google.com/document/d/1IrOYXd3S0FUwNozaEG5tM7Ki4_AZPrBn-pbyVUz-Bh0/edit#heading=h.lqc5y1f59stc

[40] J.M.Snell,“ActivityStreams2.0JavaReferenceImplementation.”[Online].Available:https://github.com/OpenSocial/activitystreams

[41] “CAMSchema.”[Online].Available:https://sites.google.com/site/camschema/home

[42] S.Fazeli,H.Drachsler,F.Brouns,andP.Sloep,“ATrust-basedSocialRecommenderforTeachers,”Proc.2ndWork.Recomm.Syst.Technol.Enhanc.Learn.,pp.49–60,2012[Online].Available:http://ceur-ws.org/Vol-896/paper4.pdf.[Accessed:12-Feb-2013]

[43] H.-C.Schmitz,M.Scheffel,M.Friedrich,M.Jahn,K.Niemann,andM.Wolpers,“CAMeraforPLE,”Lect.NotesComput.Sci.,vol.5794,pp.507–520,2009[Online].Available:http://www.springerlink.com/index/F10085480428J152.pdf.[Accessed:14-Feb-2013]

[44] H.C.Schmitz,K.Niemann,M.Wolpers,andU.Kirschenmann,“ContextualizedAttentionMetadata,”inHumanAttentioninDigitalEnvironments,C.Roda,Ed.CambridgeUniversityPress,2007,pp.1082–9873.

[45] PSLC,“GuidetotheTutorMessageformat(v4),”2013.[Online].Available:http://pslcdatashop.web.cmu.edu/dtd/guide/tutor_message_dtd_guide_v4.pdf

[46] “PSLCDataShop.”[Online].Available:https://pslcdatashop.web.cmu.edu/

[47] K.R.Koedinger,R.S.J.d.Baker,K.Cunningham,A.Skogsholm,B.Leber,andJ.Stamper,“ADataRepositoryfortheEDMCommunity:ThePSLCDataShop,”inHandbookofEducationalDataMining,C.Romero,S.Ventura,M.Pechenizkiy,andR.S.J.d.Baker,Eds.CRCPress,2011,pp.43–55.

[48] ADL,“ExperienceAPIv1.0.2,”2014.[Online].Available:https://github.com/adlnet/xAPI-Spec

[49] “TinCanAPIRegistry.”[Online].Available:https://registry.tincanapi.com/

[50] C.Glahn,“UsingtheADLExperienceAPIforMobileLearning,Sensing,Informing,Encouraging,Orchestrating,”inSeventhInternationalConferenceonNextGenerationMobileApps,ServicesandTechnologies,2013,pp.268–273[Online].Available:http://dl.acm.org/citation.cfm?id=2552412.2552453.[Accessed:11-Nov-2014]

[51] C.W.GüntherandE.Verbeek,“XESStandardDefinitionv2.0,”2014.[Online].Available:http://www.xes-standard.org/_media/xes/xesstandarddefinition-2.0.pdf.[Accessed:29-Apr-2013]

[52] “IEEECISTaskForceonProcessMining.”[Online].Available:http://www.win.tue.nl/ieeetfpm/doku.php?id=start.[Accessed:11-Jul-2014]


22

[53] W.vanderAalst,P.vandenBrand,B.vanDongen,D.Fahland,C.W.Günther,E.Verbeek,andM.Westergaard,“ProcessMiningFrameworkv6.4.”[Online].Available:http://www.promtools.org/prom6/prom64.html.[Accessed:06-Nov-2014]

[54] N.Trcka,M.Pechenizkiy,andW.vanderAalst,“ProcessMiningfromEducationalData,”inHandbookofEducationalDataMining,C.Romero,S.Ventura,M.Pechenizkiy,andR.S.J.d.Baker,Eds.CRCPress,2011,pp.123–142.

[55] K.Niemann,M.Wolpers,G.Stoitsis,G.Chinis,andN.Manouselis,“Aggregatingsocialandusagedatasetsforlearninganalytics:data-orientedchallenges,”inLAK’13:ProceedingsoftheThirdInternationalConferenceonLearningAnalyticsandKnowledge,2013,pp.245–249.

[56] K.Niemann,M.Scheffel,andM.Wolpers,“AnOverviewofUsageDataFormatsforRecommendationsinTEL,”Work.Recomm.Syst.TEL,pp.95–100,2012[Online].Available:http://ceur-ws.org/Vol-896/paper9.pdf.[Accessed:11-Feb-2013]

[57] S.Dietze,H.Drachsler,andD.Giordano,“ASurveyonLinkedDataandtheSocialWebasfacilitatorsforTELrecommendersystems,”inRecommenderSystemsforTechnologyEnhancedLearning:ResearchTrendsandApplications,N.Manouseli,H.Drachsler,K.Verbert,andO.C.Santos,Eds.SpringerScience&BusinessMedia,2014[Online].Available:http://stefandietze.files.wordpress.com/2013/03/diedragio-recsystel-cameraready.pdf.[Accessed:17-Nov-2014]

[58] M.NottinghamandR.Sayre,“TheAtomSyndicationFormat(RFC4287),”2005.[Online].Available:http://tools.ietf.org/html/rfc4287

[59] U.BojārsandJ.G.Breslin,“SIOCCoreOntologySpecification,”2010.[Online].Available:http://sioc-project.org/ontology.[Accessed:23-Apr-2013]

[60] U.Bojārs,J.G.Breslin,andA.Passant,“SIOCOntology:ApplicationsandImplementationStatus,”2009.[Online].Available:http://rdfs.org/sioc/applications/.[Accessed:17-Nov-2014]

[61] C.Chesñevar,J.McGinnis,S.Modgil,I.Rahwan,C.Reed,G.Simari,andS.Willmott,“Towardsanargumentinterchangeformat,”Knowl.Eng.Rev.,vol.21,no.4,pp.293–316,2006[Online].Available:http://argument-act.googlecode.com/svn-history/r64/trunk/docs/aif_final.pdf.[Accessed:25-Apr-2013]

[62] CEN,“WORKSHOPAGREEMENTEuropeanLearnerMobilityAchievementInformation(EuroLMAI),”2010.[Online].Available:http://www.cen-wslt.din.de/sixcms_upload/media/3378/CWA16132.pdf

[63] “EuropeanLearnerMobility-Achievementinformation(EuroLMAI)EN15981:2011.”CEN,2011[Online].Available:http://standards.cen.eu/dyn/www/f?p=204:35:0::::FSP_SURR_WI:31329&cs=1240F23EA86D3E39B77C248D44BA6944F

[64] A.Paull,“HigherEducationAchievementReport(HEAR)XMLDataSpecification:RelationshipwithStandards,”2012[Online].Available:http://www.xcri.co.uk/hear/HearTechStandardsBrief_v1.0.docx


23

[65] C.Smythe,“IMSOutcomesManagementServiceInformationModelv1.0,”2013.[Online].Available:http://www.imsglobal.org/lis/lisv2p0p1/OMSInfoModelv1p0.html.[Accessed:23-Apr-2013]

[66] S.Vickers,“IMSGlobalLearningToolsInteroperabilityTMOutcomesManagementv1.0PublicDraft,”2014.[Online].Available:http://www.imsglobal.org/lti/ltiv1p2pd/ltiOMIv1p0pd.html.[Accessed:07-Nov-2014]

[67] W.Kraan,S.Lay,andP.Gorissen,“IMSQuestion&TestInteroperabilityResultsReportingv2.1,”2012.[Online].Available:http://www.imsglobal.org/question/qtiv2p1/imsqti_resultv2p1.html.[Accessed:23-Apr-2013]

[68] S.Grant,M.Collett,M.vanCoille,J.Gorecki,J.Najjar,C.Sgouropoulou,C.M.Stacke,H.Voss,andN.Batchelor,“InLOC-Part1:InformationModelforLearningCompetences(CWA16655-1),”2013.[Online].Available:ftp://ftp.cen.eu/CEN/Sectors/List/ICT/CWAs/CWA16655-1.pdf

[69] H.Vos,“InLOCImplementioninPython/Django.”2013[Online].Available:https://github.com/henkvos/InLOC

[70] Mozilla,“BadgeSpecification(Assertions),”2013.[Online].Available:https://github.com/mozilla/openbadges-specification/blob/master/Assertion/latest.md

[71] “MozillaOpenBadgesGithubRepository.”[Online].Available:https://github.com/mozilla/openbadges

[72] E.Stacy,J.Ayers,J.Freeman,andC.Haimson,“Representinghumanperformancewithhumanperformancemeasurementlanguage,”inProceedingsoftheFall2006SimulationInteroperabilityWorkshop,2006[Online].Available:http://www.aptima.com/publications/2006_Stacy_Ayers_Freeman_Haimson.pdf.[Accessed:23-Oct-2014]

[73] D.BrickleyandL.Miller,“FOAF(FriendofaFriend)VocabularySpecification0.99,”2014.[Online].Available:http://www.foaf-project.org/

[74] C.Smythe,“IMSGlobalPersonManagementServiceInformationModelVersion2.0.1,”2013.[Online].Available:http://www.imsglobal.org/lis/lisv2p0p1/PMSInfoModelv2p0p1.html#_Toc297023211.[Accessed:07-Nov-2014]

[75] “ASNVocabulary,”2012.[Online].Available:http://standards.asn.desire2learn.com/index.php?title=ASN_Vocabulary.[Accessed:09-Apr-2014]

[76] “ASN:USDataset-USEducationalStandards(AchievementStandardsNetwork).”[Online].Available:http://asn.desire2learn.com/.[Accessed:07-Nov-2014]

[77] “MachineReadableAustralianCurriculum-TechnicalInformation.”[Online].Available:http://www.australiancurriculum.edu.au/technical/machinereadable.[Accessed:07-Nov-2014]


24

[78] “CurriculumExchangeFormat(CEF)-DatamodelEN15943:2011.”CEN,2011[Online].Available:http://standards.cen.eu/dyn/www/f?p=204:35:0::::FSP_SURR_WI:31331&cs=1D3AB2F4E5893983D4EB92855755AFD1A

[79] “CommonEducationDataStandards(CEDS)v4.”[Online].Available:https://ceds.ed.gov/dataModel.aspx.[Accessed:07-Nov-2014]

[80] “Commonbasicdataset(CBDS),”2014.[Online].Available:https://www.gov.uk/government/collections/common-basic-data-set.[Accessed:07-Nov-2014]

[81] S.Grant,“TheLeap2ASpecificationfore-PortfolioPortabilityandInteroperability,”2011.[Online].Available:http://www.leapspecs.org/2A/core-specification

[82] “Mahara1.9usermanual(Leap2Aimportandexport).”[Online].Available:http://manual.mahara.org/en/1.9/portfolio/export.html#index-0.[Accessed:17-Nov-2014]

[83] “MOOCdb.”[Online].Available:http://moocdb.csail.mit.edu/wiki/index.php?title=MOOCdb.[Accessed:25-Feb-2014]

[84] K.Veeramachaneni,F.Dernoncourt,C.Taylor,Z.Pardos,andU.O.Reilly,“MOOCdb :DevelopingDataStandardsforMOOCDataScience,”inProceedingsofthe1stWorkshoponMassiveOpenOnlineCoursesatthe10thAnnualConferenceonArtificialIntelligenceinEducation,2013,pp.1–8[Online].Available:http://arxiv.org/abs/1406.2015

[85] “OAAI(OpenAcademicAnalyticsInitiative)DatasetFormat,”2012.[Online].Available:https://confluence.sakaiproject.org/download/attachments/75671025/Required+Dataset+format.docx?version=1&modificationDate=1339166212000&api=v2

[86] PAR,“PredictiveAnalyticsReportingFrameworkDataCookbook,”2013.[Online].Available:https://community.datacookbook.com/public/institutions/par

[87] A.Dyckhoff,D.Zielke,M.Bültmann,M.A.Chatti,andU.Shroeder,“DesignandImplementationofaLearningAnalyticsToolkitforTeachers,”J.Educ.Technol.Soc.,vol.15,pp.58–76,2012[Online].Available:http://www.ifets.info/journals/15_3/5.pdf.[Accessed:13-Feb-2013]

[88] M.A.Chatti,A.L.Dyckhoff,U.Schroeder,andH.Thüs,“Areferencemodelforlearninganalytics,”Int.J.Technol.Enhanc.Learn.,vol.4,no.5/6,p.318,2012[Online].Available:http://www.inderscience.com/link.php?id=51815.[Accessed:11-Feb-2013]

[89] A.Krüger,A.Merceron,andB.Wolf,“Adatamodeltoeaseanalysisandminingofeducationaldata,”Proc.3rdInt.Conf.Educ.DataMin.,pp.131–140,2010[Online].Available:http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.174.6315&rep=rep1&type=pdf.[Accessed:22-Feb-2013]

[90] “DatastageDataAccessHow-To-AppendixB:ExplanationofOpenEdXdatafields.”[Online].Available:http://datastage.stanford.edu/#appB.[Accessed:20-May-2014]


25

[91] J.Sabo,M.Willett,P.F.Brown,D.N.Jutla,andG.Janssen,“PrivacyManagementReferenceModelandMethodology(PMRM)V1.0,”2013.[Online].Available:http://docs.oasis-open.org/pmrm/PMRM/v1.0/PMRM-v1.0.html.[Accessed:09-Apr-2014]

[92] A.Cavoukian,F.Carter,D.Jutla,J.Sabo,F.Dawson,J.Fox,T.Finneran,andS.Fieten,“PrivacybyDesignDocumentationforSoftwareEngineersVersion1.0,”2014.[Online].Available:http://docs.oasis-open.org/pbd-se/pbd-se/v1.0/csd01/pbd-se-v1.0-csd01.html.[Accessed:17-Nov-2014]

[93] T.(Ed)Hardjono,“User-ManagedAccess(UMA)ProfileofOAuth2.0,”2014.[Online].Available:http://docs.kantarainitiative.org/uma/draft-uma-core.html.[Accessed:21-Oct-2014]

[94] “UserManagedAccess(UMA)CaseStudies.”[Online].Available:http://kantarainitiative.org/confluence/display/uma/Case+Studies

[95] G.McFall,L.Neumann,S.Vickers,andM.McKell,“IMSGlobalLearningToolsInteroperabilityImplementationGuideVersion1.1.1,”2012.[Online].Available:http://www.imsglobal.org/LTI/v1p1p1/ltiIMGv1p1p1.html.[Accessed:07-Nov-2014]

[96] G.McFall,L.Neumann,andS.Vickers,“IMSGlobalLearningToolsInteroperabilityImplementationGuideVersion2.0,”2014.[Online].Available:http://www.imsglobal.org/lti/ltiv2p0/ltiIMGv2p0.html#_Toc361225537.[Accessed:07-Nov-2014]

[97] "CaliperAnalyticsTMBackground."[Online].Available:https://www.imsglobal.org/activity/caliperram


26

6. About...

AcknowledgementsTheauthorwouldliketothankToreHoelforcontributingreferencestotheMendeleyGroupandforreviewingthefirstdraft.RebeccaFergusonmadesomehelpfulsuggestionstoimprovereader-friendliness,andBrianKellyprovidedmewiththebenefitofathoroughreviewonpointsofstyleandclarity.

ThisdocumentwasproducedwithfundingfromtheEuropeanCommissionSeventhFrameworkProgrammeaspartoftheLACEProject,grantnumber619424.

AbouttheAuthorAdamworksforCetis,theCentreforEducationalTechnologyandInteroperabilityStandards,attheUniversityofBolton,UK.HeratherenjoysdatawranglingandhackingaboutwithR.HeisamemberoftheUKGovernmentOpenStandardsBoard,andamemberoftheInformationStandardsBoardforEducation,SkillsandChildren’sServices,andisastrongadvocateofopenstandardsandopensystemarchitecture.Adamisleadingtheworkpackageoninteroperabilityanddatasharing.

Aboutthisdocument(c)2015,AdamCooper,Cetis,UniversityofBolton.

LicensedforuseunderthetermsoftheCreativeCommonsAttributionv4.0licence.Attributionshouldbe“byAdamCooper,fortheLACEProject(http://www.laceproject.eu)”.

Formoreinformation,seetheLACEPublicationPolicy:http://www.laceproject.eu/publication-policy/.Note,inparticular,thatsomeimagesusedinLACEpublicationsmaynotbefreelyre-used.

Thisisapublicdraftdocumentforcomment;thelatestversionisavailablefrom:http://www.laceproject.eu/dpc/standards-specifications-quick-reference-guide/.Thefinalversionwillbelinked-tofromthere.

AboutLACETheLACEprojectbringstogetherexistingkeyEuropeanplayersinthefieldoflearninganalytics&educationaldataminingwhoarecommittedtobuildcommunitiesofpracticeandshareemergingbestpracticeinordertomakeprogresstowardsfourobjectives.

Objective1–PromoteknowledgecreationandexchangeObjective2–IncreasetheevidencebaseObjective3–ContributetothedefinitionoffuturedirectionsObjective4–Buildconsensusoninteroperabilityanddatasharing

http://www.laceproject.eu @laceproject

3 - Specifications and Standards Quick Reference...

Documents

Transcript of 3 - Specifications and Standards Quick Reference...