Improving Online Educational Resources using Cognitive Science
Joseph Jay WilliamsOffice of Online LearningStanford University
September 2013
Slides & Discussion at tiny.cc/inacolwebinar
2
Improving Online Educational Resources using Cognitive Science(& Online Collaborations between Scientists and Educators)
Joseph Jay [email protected]
www.josephjaywilliams.com/educationLytics Lab, Office of Online Learning, Graduate School of Education
Stanford UniversitySlides at tiny.cc/inacolwebinar
3
What’s different about Online Educational Resources?
• A lot!
• Align Scientists & Educators– Practical Improvements– Scientific Research
• Refine Resources through Repeated Improvement• Facilitate Collaboration• Like Wikipedia?
Content Exercise
4
Overview
• I. Applying cognitive science to online learning– I-a: How to increase motivation?– I-b: Increase learning without changing materials?– I-c: Teaching Learning Strategies– I-d: Lessons for Growth Mindset + Learning Strategies
• II. Online collaborations between Scientists & Educators– II-a: Learning, Education & Research Network (LEARN)– II-b: R.E.P.E.A.T. criteria for Research-Practice friendly resources
5
I. Reviewing & Synthesizing Cognitive Science Research
• Williams, J.J. (2013). tiny.cc/improveonlinelearningImproving Learning in MOOCs by Applying Cognitive Science. Paper to be presented at the MOOCshop Workshop, International Conference on Artificial Intelligence in Education, Memphis, TN.
• Harvey, A. G., Lee, J., Williams, J. J., Hollon, S., Walker, M., Thompson, M., & Smith, R. (in press). Improving outcome of psychosocial treatments by enhancing memory and learning. Perspectives in Psychological Science.
• www.josephjaywilliams.com/education
• Willingham, D. T. Why Don't Students Like School. Jossey-Bass (2010)• Willingham, D. T. (2012). When can you trust the experts: How to tell good science
from bad in education.
6
I-a: How to increase motivation?
• Many ways to increase motivation…• Change students’ implicit beliefs about whether
intelligence is fixed or malleable (Dweck, 2011; Yeager & Walton, 2012)
7
Implicit beliefs about Intelligence
• On a scale from 1 to 10, how much do you agree that?
• Your intelligence is something very basic about you that you can’t change very much.
• No matter how much intelligence you have, you can always change it quite a bit.
• Fixed Mindset vs. Growth Mindset (Dweck, 2006)– Self-fulfilling prophecy– Avoid uncomfortable challenges vs. Pursue Learning Opportunities– Avoid asking questions vs. Examining your mistakes
8
Research to link Science and Practice
• Experiment: Using Motivation Research to boost students’ learning of Math online on Khan Academy
• Approach:• 1. Quantify outcomes in real-world resource
• Math exercises at www.khanacademy.org
• 2. Synthesize recommendations from research• Believing intelligence is malleable increases motivation
• 3. Embed experiment to evaluate• Upcoming slides…
9
1. Learning from Math exercises on www.KhanAcademy.org
• Typical Exercise
1. Number of Problems Completed
2. Percent Correct
10
2. Synthesize Scientific Findings
• Williams, J.J. (2013)Improving Learning in MOOCs by Applying Cognitive Science. Paper presented at the MOOCshop Workshop, International Conference on Artificial Intelligence in Education, Memphis, TN.
• www.josephjaywilliams.com/education • Teaching Growth Mindset of intelligence• Research at Stanford by Dweck (2008) & Yeager &
Walton (2011)
11
3. Add motivational messagesPractice-as-usual
Remember, the more you practice the smarter you become!
Growth Mindset Message
12
3. Embedded in vivo Experiment
• Growth Mindset Message
• "Remember, the more you practice the smarter you become.”,
• "Mistakes help you learn. Think hard to learn from them.”
• Practice-as-usual
• Benefit of Growth Mindset Message?
Jascha Sohl-Dickstein
13
Number of Problems...
Practice-as-usual
Growth Mindset Message
Results: Problems Attempted & Accuracy
50 000+ students
Series1
Accuracy
14
Practice-as-usualGrowth Mindset Message
Positive Message
3. Add Positive messages
Some of these problems are hard. Do your best!
15
Does any positive message work?
• Growth Mindset Message
• "Remember, the more you practice the smarter you become.”,
• "Mistakes help you learn. Think hard to learn from them.”
• Positive Message• "Some of these problems are
hard. Just do your best."• "This might be a tough problem,
but we know you can do it.”
• Practice-as-usual
16
Number of Problems ...
Practice-as-usual
Growth Mindset Message
Positive Message
Results to Analyze
Series1
Accuracy
Research to link Science & Practice
1. Outcomes in Online
Resources
2. Recommendatio
ns from Research
3. Experiments
Khan Academy Math Exercises
Malleability of Intelligence
Ongoing Experiments
? ? ?
17
18
I-b: Increase learning without changing materials?
• Promote reflection with Socratic questions• Questions before: Problem-Based Learning• Questions during: Prompt for explanations• Questions after: Use assessments for instruction
Slides & Discussion at tiny.cc/inacolwebinar
19
Add object
to bucket
Integrate webpage
with internet
Content Exercise
Learning: Add vs. Integrate
Before: Start with Questions & Problems
• Problem Based Learning (Hmelo-Silver, 2006; Needham & Begg, 1998; Schwartz, 1998)
20
How do you…?Is it possible to…?
During: Request explanations
• Fonseca & Chi, 2011• Renkl, 1997• McNamara, 2004• Rittle-Johnson, 2006• Williams & Lombrozo, 2010
21
Why…? How?What are you thinking? What next?
22
Explanation and Learning
• Does explaining…– Provide a General boost to Learning Engagement– Selectively guide learners to look for patterns
• The Subsumptive Constraints Theory: Interpret target of why-explanation in terms of a broader generalization (Williams & Lombrozo, 2010)
• Discover general patterns (Williams & Lombrozo, 2010, Cognitive Science)
• Use pre-existing knowledge (Williams & Lombrozo, 2013, Cog. Psych.)
• May mistakenly overgeneralize by ignoring specific examples (Williams et al, 2013, JEP: General)
After: Use Assessments as Instructional Tools
• “Testing Effect” & Ten Benefits of Testing (Roediger et al, 2011)
23
Right away: Study+Study ~= Study+TestDay/Week later: Study+Study < Study+TestLearners think: Study+Study > Study+Test
After: Achieve Mixing Effect with assessments
• Mixing Effect (Rohrer, 2009)
24
25
Applying Cognitive Science
• Learning: Add vs. Integrate Knowledge (Instructionism vs. Constructivism)
• Questions before: Problem-Based Learning• Questions during: Prompt for explanations• Questions after: Use assessments for instruction• Further resources:
www.josephjaywilliams.com/education
I-c: Teaching Learning Strategies
• Spend a lesson teaching a concept vs. general strategy?
• Online: Iteratively refine excellent lessons• Online: Repeatedly reinforce habits & educational
behaviors
• Teach “What? Why? How?” self-questioning/explanation strategies
26
27
Add “Socratic” prompts to explain & reflectClickable link. + Prompts embedded into hints.
Click here to learn about the What? Why? How? strategy
28
Embedded Prompts between Hint/Solution Steps
29
Self-questioning strategy: What? Why? How?
30
I-d: Lessons for Growth Mindset + Learning Strategies
• Learning Assistant: tiny.cc/learningassistant
• Short 5 minute videos• Guided prompts
• Growth Mindset• Self-Questioning strategy
• Experiment: How do lessons influence grades?
31
II. Collaborations between Scientists & Educators
• How to gain best of both worlds?• Ask Questions, “Crowdsource” answers• The Internet removes barriers to communication &
connects people
32
II-a: Learning, Education And Research Network (LEARN)
• www.learnnetwork.net• Creating a virtual community:• Educators, Researchers, Developers• Mailing List, & Member list• Discussion Forum – any teacher can ask a question,
so can any researcher• Wiki of resources• Collaborate in improving online educational
resources
33
II-b: R.E.P.E.A.T. Criteria: Research on Online Resources
• Can you collaborate to help your students & improve online resources?
• Yes, if Online Educational Resources are:– Realistic– Experimental– Product– Evaluated– Accessible– Theoretically motivated
• REPEAT – iteratively improve through revision & collaboration• http://edlab.tc.columbia.edu/ • Organizations like iNACOL!
34
Review
• I. Applying cognitive science to online learning– I-a: How to increase motivation?– I-b: Increase learning without changing materials?– I-c: Teaching Learning Strategies– I-d: Lessons for Growth Mindset + Learning Strategies
• II. Online collaborations between Scientists & Educators– II-a: Learning, Education & Research Network (LEARN)– II-b: R.E.P.E.A.T. criteria for Research-Practice friendly resources
• Send advice & questions! [email protected]
Contact Info
• Joseph Jay Williams• [email protected]• www.josephjaywilliams.com/education• Slides at http://tiny.cc/inacolwebinar
Top Related