PLAY Minecraft!Assessing secondary engineering education using game challenges within a participatory learning environment (Research to Practice)Strand: Addressing the NGSS: Supporting K-12 Teachers in Engineering Pedagogy and Engineering-Science Connections

Erin Shaw, Computer Scientist, University of Southern California, Information Sciences InstituteMinh La & Richard Phillips, Undergraduates, USC Computer ScienceErin Reilly, Director Innovation Lab, USC Annenberg School of Communications & JournalismThank you to Aninoy Mahapatra, Sam Shuster and Dr. Anthony Maddox

WarningThis paper is not about playing Minecraft! Sorry :(

We studiedhow children talk about Minecraft!

This is good stuff, too :)

Project overviewGoalsApply participatory learning to STEMAssess learning using Next Generation Science StandardsExplore computational methods for assessing learningKey ideasSocial media/multimedia based science learningDiscourse/context based instructional assessment

Project overviewWhatUSC Undergraduate Research Associates Program (URAP)Ten weeks during summer of 2013WhoTwo undergraduates (2B junior and sophomore) Graduate student advised on NLP/ML toolsProgrammer advised on platform and databaseCommittee consisted of faculty and staff from schools of engineering, communication and education

OutlineLearning theory Participatory learningPlatform PLAYground Subject MinecraftStandards NGSS Science & Engineering PracticesMethods Topic modelingData Focus sessionProcessing Data miningAnalysis Results display

1. Learning theory Participatory learningParticipatory cultures (Jenkins)Have low barriers to engagementSupport artistic expression, sharing creations, mentorshipFoster New Media Literacies (NML)e.g., performance, appropriation, distributed cognitionFocus on social skills developed through NMLsFour Cs of participatory learningCreativity, Connection, Collaboration, Circulation

2. Platform - PLAYgroundPLAY! (Participatory Learning and You!)Project consists of technology and pedagogyPLAYgroundMultimedia authoring environmentUsers create ChallengesDesigned around 4 Cs of PLPiloted by high school teachersNever used for STEM subjects

PLAYground canvas

3. Subject MinecraftWhat is Minecraft?Popular, free-to-play computer gameHas dynamic virtual worlds built of LEGO-like blocksPlayers can combine blocks to craft items and to build fantastical structures Minecraft is a virtual engineering environment

Minecraft creations

Minecraft wiki site

4. Standards NGSS SEPs

NGSS emphasize 3 dimensionsScience & Engineering Practices (SEPs)Behaviors that scientists engage in as they investigate and build models and theories about the natural world And that engineers use as they design and build models and systems Crosscutting ConceptsProvide an organizational schema for interrelating knowledge from various science fieldsDisciplinary Core IdeasFocus curriculum, instruction and assessments on the most important aspects of science

NGSS dimensions

5. Methods Topic modelingBuild corpora for subject and standardsMinecraft Scrape wikiSEPs Copy all SEP text from NGSS documentBoth - Clean text (NLTK)ProcessPre-determine categoriesFit topic model (MALLET)Resulted in each SEP and Minecraft category being associated with a topic distribution (each topic consists of weighted vector of words)

Final topic categoriesSEP categoriesAnalyzing and interpreting dataAsking questions and defining problemsConstructing explanations and designing solutionsDeveloping and using modelsEngaging in argument from evidenceObtaining, evaluating, and communicating informationPlanning and carrying out investigationsUsing mathematics and computational thinkingMinecraft categoriesBlocks, Items, Enchanting, Mobs, Trading, Redstone Circuits, and Modifications.

6. Data Focus sessionSmall user study conductedFive boysgrade 4(2), grade 8(2) and grade 11(1)Two hours30 minutes discussing seed challenges (undergraduate created)60 minutes creating own challenges30 minutes discussing peer challengesSimple questionnaire

Seed challenge discussion

Student challenges

7. Processing Data miningExport database tablesOrganize user interactionsCollect, link and order dataAttach context, prompts, questions, etc.Output to table for validation

8. Analysis Results displayAnalyze user discourse Performed topic similarity matching of SEP and Minecraft topics using topic distributions from pre-processed categories

Engineering standards applicationG4G8aG11G8bG14G15G8b mathematically giftedG4 in only one categoryG11 in seven categories

Minecraft topics applicationG4G8aG11G8bG14G15

Standards application per studentG11 in seven categories

DiscussionResultsApplication of SEPs increases as grade level increasesMathematically gifted student identified through SEP assessGood subject category identificationMethodsValidate, acquire more text, data, further experimentationNew research to improve results using wordnetFuturePropose to apply techniques to middle school sciences

THE END

Thank you for listening.

Questions?

For more information please contact me at shaw@isi.edu

*