Authoring System in TARGET
-
Upload
georg-oettl -
Category
Documents
-
view
300 -
download
0
description
Transcript of Authoring System in TARGET
TUG-KMI
Authoring System in TARGETwww.reachyourtarget.org
Georg Ottl
Knowledge Management InstituteCognitive Science Section
April 29, 2010
Georg Ottl April 29, 2010 Page 1/37
TUG-KMI
Outline
Research environment
Competence performance assessment
Experts competence structure modeler
Probabilistic graphical modelsFactor graphs
Georg Ottl April 29, 2010 Page 2/37
TUG-KMI
Outline
Research environment
Competence performance assessment
Experts competence structure modeler
Probabilistic graphical modelsFactor graphs
Georg Ottl April 29, 2010 Page 3/37
TUG-KMI
Transformative, Adaptive, Responsive and enGagingEnvironmenT (TARGET)
I Serious game based learningenvironment
I Enterprise CompetenceDevelopment
I Improve competences in theproject management andinnovation domain
Georg Ottl April 29, 2010 Page 4/37
TUG-KMI
Five key concepts of TARGET
Georg Ottl April 29, 2010 Page 5/37
TUG-KMI
TARGET Learning Process
Georg Ottl April 29, 2010 Page 6/37
TUG-KMI
Role of TUG-KMI in TARGET
I TUG-KMI responsible for TARGET learning processI TUG-KMI responsible for workpackage competence
developmentI Competence performance assessment componentI Story adaptation/interventions
I Integration competence development/TARGET learningprocess
Georg Ottl April 29, 2010 Page 7/37
TUG-KMI
Competence performance assessment mockup
Georg Ottl April 29, 2010 Page 8/37
TUG-KMI
Outline
Research environment
Competence performance assessment
Experts competence structure modeler
Probabilistic graphical modelsFactor graphs
Georg Ottl April 29, 2010 Page 9/37
TUG-KMI
Competence performance assessment
CompetencesProblems Competence
Assessment
Georg Ottl April 29, 2010 Page 10/37
TUG-KMI
TARGET competence performance assessment
I Interpret observable performance in game experiences inregards to a competence state1.
I Include motivational state emotional state in interpretation
I Competence assessment as basis for macro andmicroadaptive2 interventions and adaptations.
I Computational model to automatically assess competencestate.
1Klaus Korossy. “Modeling Knowledge as Competence and Performance”.In: Knowledge Spaces: Theories, Empirical Research, Applications. Ed. byDietrich Albert and Josef Lukas. Mahwah, NJ: Lawrence Erlbaum Associates,1999, pp. 103–132.
2Dietrich Albert et al. “Microadaptivity within Complex Learning Situations- a Personalized Approach based on Competence Structures and ProblemSpaces”. In: Proceedings of the international Conference on Computers inEducation (ICCE 2007). 2007.
Georg Ottl April 29, 2010 Page 11/37
TUG-KMI
TARGET competence performance assessment modelauthoring
I Interpretation of the game experiences in terms ofcompetences can be done by a social community throughinspection.
I Creation of a model by using the social communityobservations input (cold start problem)
I Experts create a model to automatically interpret performance
Georg Ottl April 29, 2010 Page 12/37
TUG-KMI
TARGET competence performance assessmentrequirements
I Knowledge model/competence state exchange with HRMSystems such as SAP.
I Assessment in realtime3 to enable microadaptive interventions.
3O. Conlan et al. Realtime Knowledge Space Skill Assessment forPersonalized Digital Educational Games. IEEE, 2009, pp. 538–542.
Georg Ottl April 29, 2010 Page 13/37
TUG-KMI
Basic principle of probabilistic assessment of thecompetence state
1. If the learner has the competence ci , than increase thelikelihood of all competence states γci containing ci anddecrease the likelihood of all competence states γ 6 ci .
2. If the learner does not have the competence ci , than decreasethe likelihood of all competence states γci containing ci andincrease the likelihood of all competence states γ 6 ci .
Georg Ottl April 29, 2010 Page 14/37
TUG-KMI
Assessment calculation complexity reduction
I No structure. Possibly 2n states to be updated on everyperformance observation
I Definition of a partial order relation on competencesexploiting the properties of the “PrerequesiteOf” relation typereduces amount of possible competence states to be takeninto consideration.
I Can experts or the community directly create a competenceassessment model??
Authoring tools can help to create a model for competenceassessment.
Georg Ottl April 29, 2010 Page 15/37
TUG-KMI
Mathematical and computational model for competenceassessment
I Nondeterministic assessment4
I Traditional, multiplicative update rule56
I Belief propagation networks such as Bayesian Networks7
4C. Hockemeyer. “A Comparison of non-deterministic procedures for theadaptive assessment of knowledge”. In: Psychologische Beitrage 44.4 (2002),pp. 495–503.
5Jean-Claude Falmagne and Jean-Paul Doignon. “A class of stochasticprocedures for the assessment of knowledge”. In: British Journal ofMathematical and Statistical Psychology 41 (1988), pp. 1–23.
6Jean-Claude Falmagne and Jean-Paul Doignon. “A markovian procedurefor assessing the state of a system”. In: Journal of Mathematical Psychology32.3 (1988), pp. 232–258.
7M. Villano. “Probabilistic Student Models: Bayesian Belief Networks andKnowledge Space Theory”. In: Proceedings of the Second InternationalConference on Intelligent Tutoring Systems. Springer, 1992, 491–498.
Georg Ottl April 29, 2010 Page 16/37
TUG-KMI
Outline
Research environment
Competence performance assessment
Experts competence structure modeler
Probabilistic graphical modelsFactor graphs
Georg Ottl April 29, 2010 Page 17/37
TUG-KMI
Current state Competence Modeler
I Support to create competence assessment models
I Using the well studied PrerequesiteOf relation8
I Support experts (psychologists) to create knowledgestructures.
8Dietrich Albert et al. Knowledge Structures. Ed. by Dietrich Albert. NewYork: Springer Verlag, 1994.
Georg Ottl April 29, 2010 Page 18/37
TUG-KMI
Hasse diagram visualization
Figure: Popular knowledge space visualizations
Georg Ottl April 29, 2010 Page 19/37
TUG-KMI
Current state Competence ModelerProblems solved and Related Problems
I Computer supported Hasse diagramm creation
I Visualizations done with the Java Universal Network/GraphFramework (JUNG) framework9
9J. Madadhain et al. “Analysis and visualization of network data usingJUNG”. In: Journal of Statistical Software 10 (2005), pp. 1–35.
Georg Ottl April 29, 2010 Page 20/37
TUG-KMI
Hasse diagram visualization
→
Figure: Version 0.13 and 0.16
Georg Ottl April 29, 2010 Page 21/37
TUG-KMI
Georg Ottl April 29, 2010 Page 22/37
TUG-KMI
Current state Competence ModelerProblems solved and Related Problems
I Creation of a Hasse diagram reduced to the problem ofcalculating the minimal transitive reduction of a graph whichwas shown to have the same complexity as calculation of thetransitive closure of a graph10.
I Effective calculation and detection of cycles by maintainingthe online topological order of the graph11
I Visualizations done with the JUNG12
10A. V. Aho, M. R. Garey, and J. D. Ullman. “The Transitive Reduction of aDirected Graph”. In: SIAM Journal on Computing 1.2 (1972), pp. 131–137.
11David J. Pearce and Paul H. J. Kelly. “A dynamic topological sortalgorithm for directed acyclic graphs”. In: J. Exp. Algorithmics 11 (2006),p. 1.7.
12J. Madadhain et al. “Analysis and visualization of network data usingJUNG”. In: Journal of Statistical Software 10 (2005), pp. 1–35.
Georg Ottl April 29, 2010 Page 23/37
TUG-KMI
Current State Competence ModelerOn-Line Demo Afternoon
I https://dev-css.tu-graz.ac.at/
Georg Ottl April 29, 2010 Page 24/37
TUG-KMI
Outline
Research environment
Competence performance assessment
Experts competence structure modeler
Probabilistic graphical modelsFactor graphs
Georg Ottl April 29, 2010 Page 25/37
TUG-KMI
Probabilistic Graphical ModelsWhatfor?
I Simple way to visualize the structure of a probabilistic modelI Graphical representation allows insights into the properties of
the modelI Insights into conditional independence properties
I Complex computations can be expressed in terms of graphicalrepresentations; use of graph based inference algorithms thatexploit graph properties for calculation.
Georg Ottl April 29, 2010 Page 26/37
TUG-KMI
Graph Terminology
I A graph comprises verticesV = (a, b, c , d) connectedby edges
DefinitionA graph G is a pair G = (V ,E ), where V is a (finite) set ofvertices and E ⊆ V × V is a (finite) set of edges.
Georg Ottl April 29, 2010 Page 27/37
TUG-KMI
DefinitionA graph G is called undirected iff
∀A,B ∈ V : (A,B) ∈ E ⇒ (B,A) ∈ E (1)
Two ordered pairs (A,B) and (B,A) are identified and representedby only one undirected edge.
DefinitionA graph G is called directed iff
∀A,B ∈ V : (A,B) ∈ E ⇒ (B,A) 6∈ E (2)
An edge (A,B) considered to be a directed edge from A towardsB
Georg Ottl April 29, 2010 Page 28/37
TUG-KMI
Visual Representation Graph Models
Figure: Undirected GraphFigure: Directed Graph
Georg Ottl April 29, 2010 Page 29/37
TUG-KMI
Probabilistic Graphical Models (1/2)Whatfor?
I In a probabilistic graph model every vertice represents arandom variable
I The edges express probabilistic relationships between thevariables
I Directed Graphical probabilistic ModelsI Bayesian Networks
I Undirected Graphical Probabilistic ModelsI Markov Random FieldsI Loose coupling between statistical variables.
Georg Ottl April 29, 2010 Page 30/37
TUG-KMI
Graphical probabilistic models
I Question: Can directed probabilistic models such as asBayesian networks be used for assessment. How does believepropagation relate to the classical update rule?
I Question: How is the relation between directed andundirected probabilistic graphical models?
I Use of directed and undirected graphical probabilistic modelsto assess the players state
I Efficient sum and dot product calculation.
Georg Ottl April 29, 2010 Page 31/37
TUG-KMI
A flexible probabilistic graphical model, the Factor Graph
I Factor Graphs13 as a single representation for directed andundirected graphical probabilistic models
I Factor Graphs were successfully applied for Bayesian Networksand Markovian Models
I Multiple applications in artificial intelligence and signalprocessing based on Factor Graphs
13Frank Kschischang et al. “Factor Graphs and the Sum-Product Algorithm”.In: IEEE Transactions on Information Theory 47 (2001), pp. 498–519.
Georg Ottl April 29, 2010 Page 32/37
TUG-KMI
Factor graph conversion (1/3)
Example 1:(A simple probabilistic graph)
Let f (S1,S2,S3) be a function of threevariables, and suppose that f can beexpressed as a productf (S1,S2, S3) = p(S1)p(S2)p(S3|S1, S2)
S1
S3
S2
Georg Ottl April 29, 2010 Page 33/37
TUG-KMI
Factor graph conversion (2/3)
Example 1:(A factor graph)
Let f (S1,S2,S3) be a function of threevariables, and suppose that f can beexpressed as a productf (S1,S2, S3) = p(S1)p(S2)p(S3|S1, S2)
S1
f
S3
S2
Georg Ottl April 29, 2010 Page 34/37
TUG-KMI
Factor graph conversion (3/3)
I Efficient algorithms available to calculate probabilities(Sum-Product algorithm)
I Makes extensive use of “conditional independent” propertiesI Parallelization possible
I Approximative algorithms
I Efficient marginalization
Georg Ottl April 29, 2010 Page 35/37
TUG-KMI
Thank You!
Georg Ottl April 29, 2010 Page 36/37
TUG-KMI
[1] A. V. Aho, M. R. Garey, and J. D. Ullman. “The TransitiveReduction of a Directed Graph”. In: SIAM Journal onComputing 1.2 (1972), pp. 131–137.
[2] Dietrich Albert et al. Knowledge Structures. Ed. byDietrich Albert. New York: Springer Verlag, 1994.
[3] Dietrich Albert et al. “Microadaptivity within ComplexLearning Situations - a Personalized Approach based onCompetence Structures and Problem Spaces”. In:Proceedings of the international Conference on Computers inEducation (ICCE 2007). 2007.
[4] O. Conlan et al. Realtime Knowledge Space Skill Assessmentfor Personalized Digital Educational Games. IEEE, 2009,pp. 538–542.
[5] Jean-Claude Falmagne and Jean-Paul Doignon. “A class ofstochastic procedures for the assessment of knowledge”. In:
Georg Ottl April 29, 2010 Page 36/37
TUG-KMI
British Journal of Mathematical and Statistical Psychology41 (1988), pp. 1–23.
[6] Jean-Claude Falmagne and Jean-Paul Doignon. “Amarkovian procedure for assessing the state of a system”. In:Journal of Mathematical Psychology 32.3 (1988),pp. 232–258.
[7] C. Hockemeyer. “A Comparison of non-deterministicprocedures for the adaptive assessment of knowledge”. In:Psychologische Beitrage 44.4 (2002), pp. 495–503.
[8] Klaus Korossy. “Modeling Knowledge as Competence andPerformance”. In: Knowledge Spaces: Theories, EmpiricalResearch, Applications. Ed. by Dietrich Albert andJosef Lukas. Mahwah, NJ: Lawrence Erlbaum Associates,1999, pp. 103–132.
Georg Ottl April 29, 2010 Page 36/37
TUG-KMI
[9] Frank Kschischang et al. “Factor Graphs and theSum-Product Algorithm”. In: IEEE Transactions onInformation Theory 47 (2001), pp. 498–519.
[10] J. Madadhain et al. “Analysis and visualization of networkdata using JUNG”. In: Journal of Statistical Software 10(2005), pp. 1–35.
[11] David J. Pearce and Paul H. J. Kelly. “A dynamictopological sort algorithm for directed acyclic graphs”. In: J.Exp. Algorithmics 11 (2006), p. 1.7.
[12] M. Villano. “Probabilistic Student Models: Bayesian BeliefNetworks and Knowledge Space Theory”. In: Proceedings ofthe Second International Conference on Intelligent TutoringSystems. Springer, 1992, 491–498.
Georg Ottl April 29, 2010 Page 37/37
TUG-KMI
Acronyms
JUNG Java Universal Network/Graph Framework
Georg Ottl April 29, 2010 Page 37/37