Semantic MediaWiki for

Assessment in Education

Lloyd Rutledge, Open Universiteit

SMWCon Fall 2016, September 29th, 2016

OU courses with wiki-based assessment

• Model-Driven Development (MDD)

– Exam on paper about data modeling

– Cathedron tool for self-study and preparation

• Development Practicum (Ontwikkelpracticum)

– MDD techniques and tools using Cathedron

– Figures from 5 orders delivered as databases

• Semantic Web

– Data modeling, but different than for databases

– Figures from 5 assignments delivered as datasets

• Graduate thesis researching Semantic Web and Semantic Wikis

Pagina 2

Pagina 3

Model-driven development

from OU course T37121 Model-driven development workbook

MD

D e

xa

mq

ue

stio

n

MD

D e

xa

ma

nsw

er

MD

D a

sse

ssm

en

t in

pu

t

MD

D a

sse

ssm

en

t in

pu

t

Gro

upin

g of

test

con

cept

s Pulldown for comment options

Other Form templates for additional information

MD

D a

sse

ssm

en

t o

utp

ut

MD

D a

sse

ssm

en

t o

utp

ut

MDD assessment summary

• Student delivers exam on paper

• Grader assesses “pile” of exams

• Semantic Forms replaces ink on paper

• Pulldowns per point select comment and points

• Wiki adds points to calculate grade

• Wiki generates administrative overview spreadsheet

• Wiki digitally stores comments per exam

Pagina 12

Practicum Route

Practicum assignments

• 4: Create a data model with rules using Cathedron

– Student: Liver database with model and rules

– Teacher dogfood: SQL queries for data structure and rules

• 5: Convert big old database to model from 4

– SQL queries for old data in new structure

• 4 & 5: Queries generate readable analysis and wiki form code

• To do

– 6 - Student programs rules: even with queries

– 2 & 3: Student analyzes of data structure in document table

SQL maps data patterns to comments

Pagina 16

SELECT COUNT(*) AS "TGL: CNT_achternaam" FROM Contactpersoon WHERE achternaam != '';

SELECT COUNT(*) AS "TGL: CNT_voorletters" FROM Contactpersoon WHERE voorletters != '';

SELECT COUNT(*) AS "TGL: CNT_voornaam" FROM Contactpersoon WHERE voornaam!= '';

SELECT COUNT(*) AS "TGL: CNT_tussenvoegsels" FROM Contactpersoon WHERE tussenvoegsel!= '';

SELECT COUNT(*) AS "TGL: CNT_geslacht" FROM Contactpersoon WHERE geslacht != '';

SELECT COUNT(*) AS "TGL: CNT_M" FROM Contactpersoon WHERE geslacht = 'M';

SELECT COUNT(*) AS "TGL: CNT_V" FROM Contactpersoon WHERE geslacht = 'V';

SELECT COUNT(*) AS "TGL: CNT_O" FROM Contactpersoon WHERE geslacht != 'M' AND geslacht != 'V';

SELECT COUNT(*) AS "TGL: CNT_telefoon" FROM Klant_nfa WHERE nfa_type='telefoon' OR nfa_type='Telefoon';

SELECT COUNT(*) AS "TGL: CNT_email" FROM Klant_nfa WHERE nfa_type='e-mail' OR nfa_type='e-Mail';

SELECT COUNT(*) AS "TGL: CNT_mobiel" FROM Klant_nfa WHERE nfa_type='mobiel' OR nfa_type='Mobiel';

SELECT COUNT(*) AS "TGL: CNT_xport_datum_2099" FROM Opdracht WHERE datum_transport='01-01-2099';

SELECT COUNT(*) AS "TGL: CNT_Spoed" FROM Opdracht WHERE spoed_='J';

SELECT COUNT(*) AS "TGL: CNT_Spoed_in_note" FROM opdrachtnotitie WHERE tekst LIKE '%spoed%';

SELECT COUNT(*) AS "TGL: CNT_Verzekerde" FROM opdracht WHERE verzekerde_waarde IS NOT NULL;

SELECT COUNT(*) AS "TGL: CNT_Gewenst" FROM opdracht WHERE Datum_transport_gewenst IS NOT NULL;

SELECT COUNT(*) AS "TGL: CNT_Nmrs_w_b" FROM klant_nfa WHERE nfa_type = 'telefoon' AND nummer_adres LIKE '%b%';

SELECT COUNT(*) AS "TGL: CNT_datum_blokkering" FROM Klant WHERE datum_blokkering IS NOT NULL;

SELECT COUNT(*) AS "TGL: CNT_blokkeringsreden" FROM Klant WHERE blokkeringsreden IS NOT NULL;

SELECT COUNT(*) AS "TGL: CNT_Inv_klant" FROM opdracht WHERE klant='3974' OR klant='4877' OR klant='6783' OR klant='8479' OR k

SQL results help grader

Pagina 17

SQL results converted to SMW template code

Pagina 18

s/^ *TGL: # Value rules \n====* *\n *[1-9].*$/|OPR 4 rules value geen=No/p

s/^ *TGL: # Value rules \n====* *\n *0.*$/|OPR 4 rules value geen=Yes/p

s/^ *TGL: # att'ed Rules \n====* *\n *[1-9].*$/|OPR 4 rules geen atts=No/p

s/^ *TGL: # att'ed Rules \n====* *\n *0.*$/|OPR 4 rules geen atts=Yes/p

s/^ *TGL: CNT_voorletters \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP voorletters=No/p

s/^ *TGL: CNT_voorletters \n==* \n *0[^0-9]*$/|OPR 5 Klant CP voorletters=Yes/p

s/^ *TGL: CNT_voornaam \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP voornaam=No/p

s/^ *TGL: CNT_voornaam \n==* \n *0[^0-9]*$/|OPR 5 Klant CP voornaam=Yes/p

s/^ *TGL: CNT_tussenvoegsels \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP tussenvoegsels=No/p

s/^ *TGL: CNT_tussenvoegsels \n==* \n *0[^0-9]*$/|OPR 5 Klant CP tussenvoegsels=Yes/p

s/^ *TGL: CNT_geslacht \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP geslacht=No/p

s/^ *TGL: CNT_geslacht \n==* \n *0[^0-9]*$/|OPR 5 Klant CP geslacht=Yes/p

s/^ *TGL: CNT_telefoon \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant Tel nfa=No/p

s/^ *TGL: CNT_telefoon \n==* \n *0[^0-9]*$/|OPR 5 Klant Tel nfa=Yes/p

s/^ *TGL: CNT_email \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant Email at=No/p

s/^ *TGL: CNT_email \n==* \n *0[^0-9]*$/|OPR 5 Klant Email at=Yes/p

s/^ *TGL: CNT_mobiel \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant Tel mobile=No/p

s/^ *TGL: CNT_mobiel \n==* \n *0[^0-9]*$/|OPR 5 Klant Tel mobile=Yes/p

s/^ *TGL: CNT_xport_datum_2099 \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Opdracht Datums=Yes/p

SQL results converted to SMW template code

Pagina 19

s/^ *TGL: # Value rules \n====* *\n *[1-9].*$/|OPR 4 rules value geen=No/p

s/^ *TGL: # Value rules \n====* *\n *0.*$/|OPR 4 rules value geen=Yes/p

s/^ *TGL: # att'ed Rules \n====* *\n *[1-9].*$/|OPR 4 rules geen atts=No/p

s/^ *TGL: # att'ed Rules \n====* *\n *0.*$/|OPR 4 rules geen atts=Yes/p

s/^ *TGL: CNT_voorletters \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP voorletters=No/p

s/^ *TGL: CNT_voorletters \n==* \n *0[^0-9]*$/|OPR 5 Klant CP voorletters=Yes/p

s/^ *TGL: CNT_voornaam \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP voornaam=No/p

s/^ *TGL: CNT_voornaam \n==* \n *0[^0-9]*$/|OPR 5 Klant CP voornaam=Yes/p

s/^ *TGL: CNT_tussenvoegsels \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP tussenvoegsels=No/p

s/^ *TGL: CNT_tussenvoegsels \n==* \n *0[^0-9]*$/|OPR 5 Klant CP tussenvoegsels=Yes/p

s/^ *TGL: CNT_geslacht \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant CP geslacht=No/p

s/^ *TGL: CNT_geslacht \n==* \n *0[^0-9]*$/|OPR 5 Klant CP geslacht=Yes/p

s/^ *TGL: CNT_telefoon \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant Tel nfa=No/p

s/^ *TGL: CNT_telefoon \n==* \n *0[^0-9]*$/|OPR 5 Klant Tel nfa=Yes/p

s/^ *TGL: CNT_email \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant Email at=No/p

s/^ *TGL: CNT_email \n==* \n *0[^0-9]*$/|OPR 5 Klant Email at=Yes/p

s/^ *TGL: CNT_mobiel \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Klant Tel mobile=No/p

s/^ *TGL: CNT_mobiel \n==* \n *0[^0-9]*$/|OPR 5 Klant Tel mobile=Yes/p

s/^ *TGL: CNT_xport_datum_2099 \n==* \n *[1-9][0-9][0-9].*$/|OPR 5 Opdracht Datums=Yes/p

Pagina 20

Practicum assessment input

Pagina 21

Practicum assessment inputO

rder

give

n th

atH

elp

s gr

adin

g

Some values automatically prefilled by SQL queries

Pagina 22

Practicum assessment output

Pagina 23

Practicum assessment output

Generated email to student

Pagina 24

Comment page

Query for overview of grade for assignment

Pagina 25

Assignment grade analysis

Practicum assessment summary

• Student delivers assignments as databases

• Digital deliverable enables some automation of comments

• Semantic Forms buttons enable quick observations

• SMW query results generate feedback e-mail

• Enables new feedback more flexibly than MDD

– New comments from new insights during grading easier

– Drawback is grade not definite but relative

– Each comment plus or minus points not portion of grade

Pagina 27

Pagina 28

Semantic Web

• Development Practicum techniques adapted

– SPARQL scripts, not SQL

– Testing wiki remains the same

• Linked with research topic

– Semantic wiki's research focus

– Semantic wiki’s turned out not to be very suitable for student

assignments

Masters thesis research

• Making Semantic MediaWiki’s that provide …

– Assessment based on Rubrics

– Administration of educational program policy

– All apply Semantic Web logic and academic domain models

• Generating form property clustering and order from …

– Simple ontology heuristics, compared to text processing

– Complex ontology heuristics and data analysis

• Intelligent feedback from patterns in data models

– For business rules in non-wiki tool

– Principles apply here too

Pagina 33

Sqwizzki — The Quizzing Semantic Wiki

Summary about SMW for assessment

• Semantic Forms enables quick input

• Forms automatically prefilled from digital deliverables

– Then grader: confirms, corrects, adds more

• Feedback for students automatically generated

• Learning analytics via SMW queries

• Also: generated links to SPARQL endpoint

– General student data, such as grades in other courses

– XLST-template generates XHTML from result table

Pagina 35

Questions

Pagina 36