Post on 05-Jul-2020
1
Adaptive Learning SystemsEvaluating and Implementing the Right One (for you)
University of Florida 2017
Dale P. JohnsonAdaptive Program Manager
2
What have we learned (and why it matters)?
Adaptive software is necessary but not sufficient to ensure student success.
Cengage (Learning Objects) – psychologyCerego - medical school coursesCogBooks – biology and US historyKhan Academy – developmental mathKnewton – developmental mathMcGraw Hill ALEKS - college algebraMcGraw Hill LearnSmart Connect - chemistryPearson MyMathLab with Knewton - college algebraPearson Mastering with Knewton - physicsSmartSparrow – custom science course
3
Your needs and interests today?
• Who’s already working with adaptive learning systems?
• Who’s trying to decide whether to use them?
• Who’s evaluating results from a completed project?
• Who’s in the wrong seminar?
• Get the MOST out of this session:
ACTIVITIES:What do you want to accomplish today?What does your institution want to accomplish?
4
Why is this discussion important?
• Biology 100 – The Living World (300 non-majors)
Persistence - Performance - Satisfaction
Financials
• Biology 100 – The Living World (300 non-majors)
Persistence - 10% withdraw rate Performance - 80% grade C or better Satisfaction
Low student evaluations for large lecture sectionsFrequent faculty turnover
FinancialsWe lose money when students repeat the course.We spend money training and retaining faculty.
5
Getting Started with Adaptive Learning
Understand adaptive systems & terminology
Building and Maintaining Adaptive Learning
Identify challenges and opportunities with adaptive systems Analyze the costs and benefits of adaptive systems
Session Learning Objectives
6
What’s the best way to help?
What do they need help with?
Who needs help?
Students ON-Track
What is the promise of adaptivity?
Understand adaptive systems & terminology
From mass production to mass PERSONALIZATION.
7
How are the AL systems different?
Personalized
LMS ADAPTIVE
- Lesson Plan Fixed Variable
- Content Common
Individual- Presentation Group
Understand adaptive systems & terminology
8
How do the systems help students?
Personalize
Respect their prior knowledgeRespond to their learning needsReduce gaps in their understanding
Understand adaptive systems & terminology
9
How do the systems help faculty?
Tracking
Monitor which students need assistanceMeasure curriculum performanceMaximize course outcomes
Understand adaptive systems & terminology
10
February 21, 2017
Learning Benefits from these Pedagogical Techniques*
Formative Evaluation (d=.90) Acceleration (.88) Effective Feedback (.73) Meta-cognition (.69) Mastery Based Learning (.58) Concept Mapping (.57) Interactive content (.52)
*Source: John Hattie’s Visible Learning (2009)
15 year, 800+ meta analysis on achievement
Cohen’s d = effect size which is an indication of whether something is practically significant.
10
Understand adaptive systems & terminology
What should we expect to gain?
When d is > .40 there were excellent learning gains.
11
Where are we in the technology cycle?
WE ARE ALL HERE!
Adaptive learning systems are still experimental.
Understand adaptive systems & terminology
12
Lesson sequence Content selection
Algorithm (analytics) – recommendations Assessment – rapid remediation Assignment – decision tree Agency – student chooses
Understand adaptive systems & terminology
What is adapting to the student?
What is guiding the adaptation?
13
(or Competency, Skill, Topic, Unit, Module, etc.)
Components form an “Ed Molecule”
Understand adaptive systems & terminology
What makes up a measurable Lesson?
Objective
ContentAssessment
14
Understand adaptive systems & terminology
How are the Lessons sequenced?
Basic Math Algebra Calculus
Course Configuration
Lesson 1
L-3
L-5
L-4
L-6
L-7 L-x
L-2
Lesson Chart (or Knowledge Map, Skill Graph, etc.)
15
AlgorithmAssessmentAssignmentAgency
Lesson Library
LessonInstructional
MaterialsPractice
ProblemsAssessment Instruments
LessonInstructional
MaterialsPractice
ProblemsAssessment Instruments
LessonObjective
Assessments
Content
How do students experience Lesson adaptivity?
Understand adaptive systems & terminology
StudentLessonObjective
Assessments
ContentContent
Content
16
What is an example of Lesson adaptivity?
Khan Academy
ACTIVITY:Go to https://www.khanacademy.org/exercisedashboard
Understand adaptive systems & terminology
17
How is Content adapting to the student?
Learning Loop
Lesson #1 Lesson X
AlgorithmAssessmentAssignment
Agency
Understand adaptive systems & terminology
(or Resource, Material, Text, Video, Exercise, Problem, etc.)
Pre-Assessment
Content Content
AlgorithmAssessmentAssignment
Agency
AlgorithmAssessmentAssignment
Agency
18
Vendors often use a mix of all four models:
Understand adaptive systems & terminology
What is an example of Content adaptivity?
Algorithm
Assessment
Assignment
Agency
19
Progress can depend on performance! NOT JUST THE SCHEDULE
Understand adaptive systems & terminology
What does all this mean to the student?
ASU MAT142August to December 2012Concepts completed by each student Median
20
Get the MOST out of this session:
ACTIVITIES:Complete Khan Academy initial analysis
Think of questions to ask tomorrow
21
Getting Started with Adaptive Learning
Understand adaptive systems & terminology
Building and Maintaining Adaptive Learning
Identify challenges and opportunities with adaptive systems Analyze the costs and benefits of adaptive systems
Seminar Learning Objectives
22
What are the major challenges and decisions?
Pedagogy System Course Development Faculty Facilities Evaluation
Identify challenges and opportunities with adaptive systems
ACTIVITY:Build your plan as we go…
23
How can the system support your Pedagogy?
Identify challenges and opportunities with adaptive systems
Lecture
Flipped
Online
Other?
- homework and assessment
- content delivery and assessment
- content, homework and assessment
24
Student learning
What other pedagogical decisions matter?
Self-paced or Synced
Complex tracking Lesson alignment
Flexible exam dates Fixed exam dates
Independent Interactive
Competency Completion
Faculty need to make these decisions first.
Identify challenges and opportunities with adaptive systems
25
Options we’ve tried:
Online and Self-paced
Flipped and Self-paced
Flipped and Synced
Our best results have been with “Flipped and Synced”
What pedagogical strategy works for us?
Bloom’s Taxonomy
Adaptive Learningbefore class
Active Learning in class
Identify challenges and opportunities with adaptive systems
26
ADAPTIVE SYSTEM
ACTIVE CLASS
4. ASSIMILATEWrite essay, solve problems, take quiz, etc.
Prove
ParticipateProve
Prepare
1. ACQUIRE INFORead textbook, watch video, do simulation, etc.
2. ANALYZEDo practice problems, take quiz before class
3. APPLYSolve an applied problem (case study) with classmates.
How does this pedagogy work in practice?
Identify challenges and opportunities with adaptive systems
27
Identify challenges and opportunities with adaptive systems
What does a course structure look like?
Adaptive and Active learning are symbiotic in the educational ecosystem.
28
Adaptive
What are the major System components?
Student UX
ReportingLesson
Instructor UX
GroupingReporting
Content Curation
AssociateEdit
InformationArchitecture
ContentDatabase
Course Creation
SequenceEdit
Analytics
Identify challenges and opportunities with adaptive systems
29
Construct or Configure
Time consuming Quick
Flexible Constrained
Costly Cheap
Riskier Safer
Consider the commitment before committing.
What are some key system selection issues?
Identify challenges and opportunities with adaptive systems
Course Creation
30
How are different systems implemented?
Knewton Knowledge Graph
The market is moving toward modularity.
Identify challenges and opportunities with adaptive systems
Course Creation – Construct
Lesson 1
L-3
L-5
L-4
L-6
L-7 L-x
L-2
Configure
31
Identify challenges and opportunities with adaptive systems
How does the system handle different subjects?
Course Creation – Subject Matter
Math Model Assess, learn, assess Predicated on 12 years of prior math education
Other Disciplines More traditional presentation Learn, assess, remediate, assess
Adaptive logic may not be the same for all subjects.
32
Identify challenges and opportunities with adaptive systems
How can you manage content in the system?
Content Curation
Vendor Resources Prebuilt courses Library of materials
Other Resources Faculty produced content Open Educational Resources (OER)
Will the vendor allow outside content?
33
How can you manage Course Development?
Identify challenges and opportunities with adaptive systems
Adaptive tech is a team sport, so plan accordingly.
Faculty Technologists VendorPersonnel
Instructional Designers
Faculty Technologists VendorPersonnel
Instructional Designers
Faculty Technologists VendorPersonnel
Instructional Designers
Faculty Technologists VendorPersonnel
Instructional Designers
GraphicDesigners
Librarians Others?Video Producers
GraphicDesigners
Librarians Others?Video Producers
GraphicDesigners
Librarians Others?Video Producers
GraphicDesigners
Librarians LeadersVideo Producers
34
Strong provost and departmental
support (and $)
Faculty leadership for each course
Instructor peer mentoring for training
“Guide on the Side” is not for everyone
Be patient, it’s a learning process
The technology success depends on the teacher!
What can help Faculty succeed with AL?
Identify challenges and opportunities with adaptive systems
35
Think the project all the way through.
What types of classroom Facilities will you need?
How many rooms?
How big do they need to be?
How long will they take to build?
How much will they cost?
Who is going to pay for them?
How will you support them?
Identify challenges and opportunities with adaptive systems
36
Macro indicators• Persistence (lower student withdraw rate)• Performance (higher student pass rate)• Satisfaction
• Student• Instructor• Administrator
• Financials• Money saved or spent
Micro indicators• Assessment results (lesson or exam level)
Identify challenges and opportunities with adaptive systems
What factors could you Evaluate?
37
Identify challenges and opportunities with adaptive systems
How does ASU evaluate our GenEd efforts?
MISSION: Enable Student Success
OBJECTIVES:
Improve critical thinking & problem solving
Increase student subject mastery
Increase student retention
Improve instructor insight
Achieve 90% freshman retention
Do Applied Concept Exercise every class
Help 90% of students get C or better
Re-enroll 90% of students next semester
Identify struggling students by week 2
38
• Biology 100 – The Living World (300 non-majors)
Persistence Performance Satisfaction
Financials
How has it worked at ASU?
• Biology 100 – The Living World (300 non-majors)
Persistence – withdraws fell to 1.5 % Performance – C or better increased to 94% Satisfaction
Student course evaluations improvedProfessors said, “I’m never lecturing again.”
FinancialsWe saved money from not having repeat students.We spent money to put more instructors in the classroom.
Analyze the costs and benefits of adaptive systems
39
What do the students think?
Analyze the costs and benefits of adaptive systems
40
• Rio Salado College student survey results…•
98% of the students liked the adaptive course
96% felt the system adapted to their needs
89% stated the adaptive course was more helpful than traditional textbooks
**N = 138 students enrolled in ENG 081 and 091 (69% response rate)
What do the students think?
Analyze the costs and benefits of adaptive systems
41
What is this going to cost?
Implementation• Construct
• Configure
Operation
Implementation• Construct ~ $50,000
• Faculty time ~ 9 months • Staff time ~ 12 months• Systems integration ~ 1 month• Content development and licensing
• Configure ~ $5,000• Faculty time ~ 1 month• Staff time ~ 3 months
OperationStudent license $35 - $100Faculty training – every semester
Analyze the costs and benefits of adaptive systems
42
• Dale Johnson
• Dale.Johnson@asu.edu
• 480-884-1927
Thank You for participating!