Artificial Intelligence as an Effective Classroom Assistant Benedict du BoulayUniversity of SussexUK
Bloom (1984) and the 2-sigma challenge
2-sigma
Contents1. What is an AIED system2. An example: not just about cognition3. A range of meta-reviews since 2011
What is an AIED system?
AIED systems: Adaptivity• Reason about Pedagogy
• What task to set next• How best to react to the leaner’s cognitive, affective and
motivational state• Reason about the domain of instruction
• Follow learner’s problem-solving step by step• Reason about how much is being learned
• Focused feedback, help and support • Adjust learning tasks dynamically
• Use interface to model domain• Both system and learner can reason in the interface
AIED (ITS) system: 4 components
Pedagogical model
Student model
Interface
Domain model
Betty’s Brain (Leelawong & Biswas, 2008)
Johnson (2010) Tactical Iraqi
Recent META-REVIEWSComparing AIED/ITSs to whole class teaching and one-to-one tutoring
VanLehn (2011)Various ITSs
No. of Studies
Effects And . . .
88 “just as effective as adult, one-to-one tutoring for increasing gains in STEM topics”
“ITSs should be used to replace homework, seatwork and perhaps other activities but not to replace a whole classroom experience”
Ma, Adesope, Nesbit, Liu (2014)Various ITSs
No. of Studies
Effects And . . .
107 Good effect sizes against teacher-led, large group instruction;textbooks;non-ITSs.
No difference against individualised human tutoring or small group instruction
Steenbergen-Hu & Cooper (2013)Various College level ITSs
No. of Studies
Effects But . . .
39 “Have demonstrated their ability to outperform many instructional methods or learning activities in facilitating college-level students’ learning of a wide range of subjects . . .
. . . although they are not as effective as human tutors”
Steenbergen-Hu & Cooper (2013)ITSs K-12 Maths
No. of Studies
Effects But . . .
34 Very small positive effect sizes < 0.1
Better for ordinary pupils than low achievers;Better if intervention lasted < 1 year
Kulik & Fletcher (2015)Various ITSs
No. of Studies
Effects But . . .
50 “ITSs can be very effective instructional tools”
Degree of improvement depended on whether standardised or local tests used
Nesbit, Adesope, Liu (2014)Computer Science ITSsNo. of Studies
Effects But . . .
22 “A significant advantage of ITS over teacher-led classroom instruction and over computer-based instruction”
No system combined procedural with conceptual tutoring
Cognitive Algebra Tutor: Koedinger et al.
Pane, McCaffrey & Karam (2014)Cognitive Tutor Algebra
No. of Schools
Comparison Effects But . . .
73 high 74 middle
Across 7 USA states
Post-tests using traditional teaching
First year not significant.Second year high schools significant. Effect size 0.20
No direct monitoring of how the system actually used
-0.19
EffectSize
0.47
Conclusions• Described AIED systems• Offered an example concerning Motivation and Metacognition with partially successful evaluation
• Summarised recent meta-reviews• Generally positive about ITSs vs most forms of
traditional teaching for STEM subjects• including vs CAI systems
• Blended learning model favoured• But degree of overlap in coverage between meta-
analyses not checked
An Example: not just about cognition
Alison Hull - SQL-Tutor* - 2014• Based on SQL-Tutor (Mitrovic)• +Metacognitive & Motivational help and feedback• Invites learner to assess his or her confidence • Tracks self-efficacy and its accuracy• Provides metacognitive and motivational feedback at start/end of session and at end of problem
• Refers back to similar previous situations
Accuracy of self-efficacy
Performance
High Mid LowPessimistic “Even though you have
not found the problems easy, you really have made good progress in successfully completing problems . . .”
. . . Effort Poor: “Think about focussing on problems with complexity rating of 1 or 2 at the beginning of the next session to firmly establish your core knowledge before expanding on it . . .”Effort Good: . . .
Realistic . . . . . . . . .
Optimistic “You have made really good progress this session, but have the problems challenged you enough? Have a think whether you are ready to try more difficult problems next time . . .”
. . . Effort Poor: “You have not made much progress this session, but that doesn't mean the next session has to be the same . . .”Effort Good: . . .
Accuracy of self-efficacy
Performance
MiddleRealistic You have made steady progress this session
Your performance matched your confidence this session. How confident do you feel about answering SQL queries? Please indicate below.
Evaluation• Northumbria University: one semester.• Basic SQL-Tutor vs enhanced SQL-Tutor* (n=39) [+ vs no ITS ]
• Log files, assessment scores, questionnaire.
Results1. Does providing M&M feedback
lead to any measurable learning gains?
1. Does providing M&M feedback lead to any measurable gains in learner focus?
2. Do the learners perceive any benefit from using an ITS to aid their learning?
3. What are the effects of including M&M feedback based on the learner’s past states and experiences?
11 Yes study group move slightly up class rankings; both groups solved same number of problems, but Control group took more attempts. Both groups better than no Tutor
11 Control group bit more persistent on unsolved problems study group made bit better use of the system
11 Yes for both groups, study group slightly more positive
11 Some positive trends study group students typically not follow up references to past similar situations
Top Related