GIAF USA Winter 2015 - Measuring collaboration in a multiplayer game
17
-
Upload
lauren-cormack -
Category
Data & Analytics
-
view
557 -
download
1
Transcript of GIAF USA Winter 2015 - Measuring collaboration in a multiplayer game
Measuring Collaboration in a Multiplayer Game
Deirdre Kerr & Jessica Andrews Educational Testing Service
12/10/2015
What We Do
Awesome!
Learning Sciences Data Mining
StatisticsBehavioral Analytics
Goals Develop a methodology for scaling
qualitative analytics Interpretable, actionable information About individual behavior Without requiring in-person observations
Define collaborative behavior Often defined as team performance Different definitions in different contexts
Develop a measure of collaboration Based on behaviors Independent of performance
Why Collaboration? Traditionally qualitative
Reflected in problem solving process (not result) Not measurable outside behaviors
Worthwhile problem Of concern to employers Unclear impact on performance No good preexisting measures
Actionable information Remediation Incentives Accurate scoring of individuals in a group
Example Environment
Generate an Ontology
Expand to a Behavioral Ontology
LegendNode ShapesOvals = latent or calculated variablesRounded Rectangles = inferred actionsRectangles = observed actions
Node ColorsLight Gray = task independentDark Gray = task dependent
Expand to a Cognitively Enhanced Ontology
Extract Features
Create Chains-of-Evidence
Action Set Label
Action Set Label
I-TAF FrameworkOverview
Bayesian Networks
Action Set Label FrequencyFrequentAverageRare
Collaborative SkillExcellentGoodFairPoor
Action Set Label OccurrencePresentAbsent
Learned
Predicted/Measured
Calculated
Using I-TAF with a Different Game Use the same ontology & behavioral
ontology Update dark gray nodes in the cognitively
enhanced ontology To represent different affordances in the new
game Update features Update chains-of-evidence
To extract the SAME action set labels Use the same model (Bayesian Network)
JOIN IN THE CONVERSATION PARTICIPATE IN THE NEXT GIAF
GIAF on LinkedIn www.deltadna.com/[email protected]
