Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The...
-
Upload
bryan-bruce -
Category
Documents
-
view
213 -
download
0
Transcript of Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The...
![Page 1: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/1.jpg)
Scientifically Informed Digital Learning Interventions
Financial and Intellectual Support:• The William and Flora Hewlett Foundation• The National Science Foundation• A.W. Mellon Foundation• Carnegie Mellon University
One Example: The Open Learning Initiative at Carnegie Mellon
![Page 2: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/2.jpg)
The Challenge• To learn ways to design and build fully
web-based courses which by rigorous assessments are proven to be as good or better than traditional teaching methods
• Why?– Increasing access– Improving effectiveness– Providing flexibility for faculty and students– Containing costs
![Page 3: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/3.jpg)
A Flaw and an Opportunity
• Current structure of higher education presents substantial roadblocks to the application of proven results and methodologies from the learning sciences
• eLearning interventions, developed by teams rather than individuals, are more conducive to making the practice of education more scientific and effective
![Page 4: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/4.jpg)
OLI Guiding Assumptions• Digital learning interventions can make a
significant difference learning outcomes• Designs grounded in contemporary learning
theory and scientific evaluation have the best chance of achieving that goal
• A possible, acceptable outcome of the OLI efforts is failure or mixed failures and successes – we are doing “action research,” not promoting eLearning for its own sake
![Page 5: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/5.jpg)
OLI Guiding Assumptions• Formative assessment will be a major
feature (and a major component of the cost) of the designs and iterative improvements of the courses
• IT staff working with faculty is too limited a partnership – learning scientists, HCI experts, and assessment experts must be part of design, development, production and iterative improvement
![Page 6: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/6.jpg)
Open Learning Initiative Courses• Statistics• Modern Biology• Chemistry• French• Engineering Statics• Causal and Statistical Reasoning• Economics• Logic and Proofs• Physics• Empirical Research Methods• Computational Discrete Mathematics
![Page 7: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/7.jpg)
Try it Yourself
• http://www.cmu.edu/oli
• Don’t expect an “OCW experience”…this project has a different set of goals than OCW
• “Clicking around” will be unsatisfying: these interventions are designed to support a novice learner in acquiring knowledge working on their own
![Page 8: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/8.jpg)
Key Elements in OLI Courses
• Theory Based: Course and individual lesson designs based on current theories in the learning sciences
• Feedback Loops: Courses record student activity for robust feedback mechanisms
• Diversity of Perspectives, Roles and Contexts: Courses developed and deployed by teams that include faculty content experts, learning scientists, software engineers
![Page 9: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/9.jpg)
Theory Based: Build on Prior/Informal Knowledge
![Page 10: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/10.jpg)
Theory Based: Provide Immediate Feedback in the Problem Solving Context
![Page 11: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/11.jpg)
Theory Based: Promote Authenticity, Flexibility & Applicability
• Learning environments with ambiguous problems that require flexible application of procedural knowledge
![Page 12: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/12.jpg)
Feedback Loops in Learning
![Page 13: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/13.jpg)
Evaluation• Chemistry: Post-test scores by treatment group show
significant positive correlation for the OLI treatment. Most significant indicator was time spent in Virtual Lab Activities – made all other variables drop out.
• Biology: End of the 3rd week showed an advantage for the OLI section. There was a positive and significant association between students’ time spent working on particular activities and performance on quiz questions testing the corresponding topics even after total time with OLI has been regressed out
![Page 14: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/14.jpg)
Evaluation• Statistics 1st Study:
![Page 15: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/15.jpg)
n Average % correct
Pre 488 43.3
Post 488 51.2
Increase: 7.9% Increase: 7.9% [t(487) = 13.8, p <.001][t(487) = 13.8, p <.001]
n Average % correct
Pre 24 55.8
Post 24 66.5
Increase: 11.7%Increase: 11.7%[t(23) = 4.7, p <.001][t(23) = 4.7, p <.001]
CAOS Sample:CAOS Sample:
CMU OLI Course Sample:CMU OLI Course Sample:
Evaluation
![Page 16: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/16.jpg)
Measured learning Outcome
% correct CAOS % correct CMU
Pre Post Pre Post
Box plots provide accurate estimates of % data above & below only for quartiles
22.2 22.2 22.2 50.0
Correctly estimate and compare SD’s for different histograms.
31.5 46.4 66.7 83.3
41.8 46.4 59.3 75.0Correlation does not imply causation
51.9 49.4 48.1 70.8
Calculating appropriate conditional probabilities given table of data
49.6 47.4 70.4 70.8
Evaluation
![Page 17: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/17.jpg)
Accelerated Learning Study
• Taught Carnegie Mellon Introductory Statistics course in a blended mode (one in class meeting per week) in half a semester
• The OLI Statistics course was the “textbook”• OLI course provided the professor immediate
feedback on students’ performance• We compared learning outcomes in the two
different treatments
![Page 18: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/18.jpg)
Accelerated Learning Study• OLI students significantly outperformed
Traditional “control” students on the CAOS post-test. CAOS Post-test
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
OLI TRAD
Pro
port
ion
Corr
ect
![Page 19: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/19.jpg)
Accelerated Learning Study• OLI students showed significantly greater gains (pre
to post) than the Traditional “control” students on the CAOS test.
0
0.05
0.1
0.15
0.2
0.25
OLI TRAD
Gain
Score
![Page 20: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/20.jpg)
Student Satisfaction
– End of course survey for online section:
• All students reported at an increase in their interest in statistics.
• 75% Definitely Recommend
25% Probably Recommend
0% Probably not Recommend
0% Definitely not Recommend
![Page 21: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/21.jpg)
Feedback Loop – Current Research
Instructors can use such data to adjust their teaching to students’ needs.
![Page 22: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/22.jpg)
Learning Curve Analysis on Stoichiometry Data
![Page 23: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/23.jpg)
The Vision – Digital Dashboard for Teaching and Learning:• Instructor assigns students to work
through online instruction• System collects data as students work • System automatically analyzes and
organizes the data to present instructor with the students’ current “learning state”
• Instructor reviews this data summary and adapts instruction accordingly
![Page 24: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/24.jpg)
The Anticipated Benefits
• Instructors get a window onto students’ progress
• They can adapt their teaching accordingly
• Students get better feedback to monitor and adjust their learning
• Strengthens the student-instructor connection
![Page 25: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/25.jpg)
Core OLI Community• Faculty Content Experts• Learning Scientists• Human Computer Interaction• Software Engineers• Evaluation/Assessment Specialists• Learners• A community of scholars from diverse
disciplines who are committed to improving quality and access to instruction. The collaborative nature of the OLI course design process inspired participating faculty to rethink their approach to classroom teaching.
![Page 26: Scientifically Informed Digital Learning Interventions Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science.](https://reader035.fdocuments.in/reader035/viewer/2022062715/56649d9f5503460f94a8a36f/html5/thumbnails/26.jpg)
“Improvement in post-secondary education will require converting teaching from a ‘solo sport’ to a community-based research activity”
Herbert Simon
www.cmu.edu/oli
[email protected]@cmu.edu
(Candace Thille – Director)