What is Stealth Assessment?
-
Upload
games-for-learning-institute-g4li -
Category
Education
-
view
2.701 -
download
5
description
Transcript of What is Stealth Assessment?
![Page 1: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/1.jpg)
Valerie ShuteFlorida State University
Games for Learning @ G4C (June 22, 2011)
Stealth Assessment Games, Learning, and
![Page 2: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/2.jpg)
Games, Learning, Assessment
Claim 1
Good games can act as transformative environments to support skill development and deep, meaningful learning.
Claim 2
Learning is best when it is active, interesting, goal-oriented, and contextualized (i.e., features of good games).
Claim 3
Stealth assessmentcan collect dynamic evidence of learning in real-time, at various grain sizes (and use info to support learning).
+ + +
![Page 3: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/3.jpg)
Define 21st century competencies & good games
Overview evidence-centered design
Describe stealth assessment
Illustrate approach in a worked example
Game Plan
![Page 4: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/4.jpg)
The world has changed a lot in the past 100 years. Education has not.
Classroom photo, 1910 Classroom photo, 2010
It’s Time for a Change
![Page 5: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/5.jpg)
Today’s kids need new
skills to be successful
and productive
(e.g., creativity, collabor
ation, communication).
The world in which we
live is complex and
interconnected
(especially compared to
the really old days).
21st Century Skills
![Page 6: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/6.jpg)
Curiosity
Critical Thinking
Collaboration
Grit
Systems Thinking
Empathy
Creativity
![Page 7: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/7.jpg)
One problem with embracing new skills is the lack of valid & reliable assessments for them.
Old ways of testing won’t work. Learning and succeeding in our complex world can’t be optimally be measured by MC tests.
Re-think assessment!
Challenges
![Page 8: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/8.jpg)
Games
![Page 9: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/9.jpg)
Good Game Elements
1. Interactive problem solving
2. Specific goals/rules
3. Adaptive challenges
4. Control
5. Ongoing feedback
6. Uncertainty
7. Sensory Stimuli
![Page 10: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/10.jpg)
Interactivity
Goals
Challenges
ControlFeedback
Uncertainty
Stimuli
Gestalt of Games
![Page 11: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/11.jpg)
(e.g., Mislevy, Steinberg, & Almond, 2003)
Evidence-Centered Design
![Page 12: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/12.jpg)
Competency Model
What do you want to say about the
person?
Evidence Model
What observations would provide best
evidence for what you want to say?
Task/Action Model
Model
What kinds of tasks let you make the
necessary observations?
Assessment Design
![Page 13: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/13.jpg)
Design & Diagnosis
![Page 14: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/14.jpg)
Stealth Assessment
![Page 15: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/15.jpg)
Accurate & Rich Learner Models
Seamless & Ubiquitous Assessment
Aligned along 3 dimensions: vertically, horizontally, and temporally
Stealth Assessment Features
![Page 16: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/16.jpg)
Advances in measurement let us administer evidence-based assessments to:
Extract ongoing information from a learner
Make accurate inferences of competencies
React in immediate and helpful ways.
Accomplished via automated scoring and machine-based reasoning techniques.
When assessment is so seamlessly woven into the fabric of the learning environment that it’s invisible, this is stealth assessment.
Stealth Assessment
![Page 17: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/17.jpg)
Elder Scrolls IV
Oblivion
![Page 18: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/18.jpg)
First person 3D RPG set in a medieval world
Can be one of many characters (e.g., knight, mage, elf), each who has (or can obtain) various weapons, spells, and tools
Primary goal—gain rank & complete quests
Quests may include locating a person to obtain info, figuring out a clue for future quests, etc.
Multiple mini quests along the way, and a major quest that results in winning the game (100s of hr of game play)
Players have the freedom to complete quests in any order
Elder Scrolls IV: Oblivion
![Page 19: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/19.jpg)
Success in Oblivion
Cognitive Dispositions
Background
Knowledge
Reasoning
Skills
Working
Memory
Reading
Comp
Problem
Solving
ReflectionExploratory
Behavior
PersistenceCreativity
Creative
Problem
Solving
Attention Efficiency Novelty
Competency Model
![Page 20: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/20.jpg)
Efficiency Novelty
Scene 2Scene 1
Evidence Model
Competency Model
Action Model
Creative Problem Solving
Scene 1
Action Indicators Scene 2
Unobservables
Observables
The Glue
Example ECD Models
Problem Solving Creativity
![Page 21: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/21.jpg)
Relevant* Action Novelty Efficiency
Swim across the river n = 0.12 e = 0.22
Levitate over the river n = 0.33 e = 0.70
Freeze water with a spell and slide across n = 0.76 e = 0.80
Find a bridge over the river n = 0.66 e = 0.24
Dig a tunnel under the river n = 0.78 e = 0.20
* Relevant refers to any action included in a successful solution.
Problem: Cross river filled with dangerous fish to get to the cave on the other side.
Action Model w/Indicators
![Page 22: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/22.jpg)
Novelty: 1 – frequency
Efficiency: Inverse fn (resources, time)
Action: Find a bridge over the river
Indicators: Novelty = 1 - 0.34 = 0.66
Efficiency = 1 / [(3 0.4) + (5 0.6)] = 0.24
• Resources Used = Weapon (1, fight monster w/sword) + Health (1, damage from monster) + Object (1, magic potion) = 3 resources (weight = 0.4)
• Time expended = 5 minutes (weight = 0.6)
Indicators Per Action
![Page 23: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/23.jpg)
CreativeProblemSolving
LowHigh
0.600.40
Creativity
LowHigh
0.110.89
ProblemSolving
LowHigh
0.640.36
ObservedNovelty
0 to 0.250.25 to 0.50.5 to 0.750.75 to 1
0001
0.78 0.07
Novelty
LowHigh
0.020.98
Efficiency
LowHigh
0.860.14
ObservedEfficiency
0 to 0.250.25 to 0.50.5 to 0.750.75 to 1
1000
0.20 0.07
Dig a tunnel under the river: e = 0.20; n = 0.78
Bayes Model—Case 1
![Page 24: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/24.jpg)
Bayes nets can be used in various ways to improve learning and performance.
Continuously gather evidence for accurate, real-time estimates of comp’s.
Info on competencies used by (a) teachers(to adjust instruction & give good feedback), (b) system (to select new gaming experiences), and/or (c) students (to reflect on how they’re doing).
Current estimates of competency levels can be integrated into the game and displayed as progress indicators.
This elevates valued competencies to the same level as health and weapons!
Wrapping it Up
![Page 25: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/25.jpg)
To address educational challenges and harness potential of immersive games, I presented an ECD-inspired idea re:
• Specifying competencies to be acquired from the game
• Defining EMs that link game behaviors to competencies
• Updating the learner model regularly
Using ECD, stealth assessment, and automated data collection and analysis tools is meant to collect valid evidence of students’ emerging competencies, and reduce teachers’ workload allowing them to focus on fostering student learning.
Next steps: adapt content (feedback, difficulty levels, scaffolding, etc.) in game to fit current needs of player, test plug ’n play of models, evaluate learning effects, etc.
Summary
![Page 27: What is Stealth Assessment?](https://reader034.fdocuments.in/reader034/viewer/2022042507/559470741a28ab8d0e8b46ce/html5/thumbnails/27.jpg)
In the Stone Age, kids
learned how to start a fire
with sticks. Hunting skills
were important too.
The world in which they
lived was fairly simple and
flat.
Really Old Days
Return