Slide 1
Kirsten Butcher
Elaborated Explanations for Visual/Verbal Problem Solving:
Interactive Communication ClusterJuly 24, 2006
Slide 2
Visual & Verbal Information in Geometry
Geometry Cognitive Tutor: Angles and Circles Units.
Slide 3
Research Goals
To understand how coordination between & integration of visual and verbal knowledge influences robust learning
To explore the potential transfer of laboratory-identified multimedia principles to classroom context
To inform the design of effective educational multimedia for classroom use
Slide 4
Relevant Learning Research
Learning with Multimedia Contiguity Effect (e.g., Mayer, 2001) Diagrams support inference-generation & integration of
information (Butcher, 2006)
Self-explanations & Cognitive Tutors Self-explanations promote learning (e.g., Chi et al., 1994) Simple (menu-based) self-explanations support Geometry
Learning (Aleven & Koedinger, 2002)
Slide 5
Hypotheses: Sense-making Scaffolds
Contiguity Work & receive feedback in diagram
Integrated Hints Apply verbal hints to visual problem situation
(diagram) Elaborated Explanations
Visual “explanations” to justify problem-solving
Slide 6
Hypotheses: Sense-making Scaffolds
Contiguity Work & receive feedback in diagram
Integrated Hints Apply verbal hints to visual problem situation
(diagram) Elaborated Explanations
Visual “explanations” to justify problem-solving
Slide 7
Connections to PSLC Theory Sense-making
Coordinative Learning: Integrate results from multiple inputs & representations. Verbal information Visual information
Scaffolds change the format of the interface to promote coordinative learning. Contiguous representation: reduces mapping & supports
inferences made directly from diagram Integrated hints: reduce mapping & support recognition of
critical visual elements
Slide 8
Hypotheses: Sense-making Scaffolds
Contiguity Work & receive feedback in diagram
Integrated Hints Apply verbal hints to visual problem situation
(diagram) Elaborated Explanations
Visual “explanations” to justify problem-solving
Slide 9
Connections to PSLC Theory Sense-making
Interactive Communication: Tutor prompts explanations Students “explain” geometry principles that justify problem-
solving steps Students receive feedback and hints on explanations
Scaffold: Elaborated explanations require student to “explain” the application of geometry principles Rationale for explanations are visual in nature Diagram Condition: Visual format for explanation Table Condition: Verbal format for explanation
Slide 10
Existing Tutor: Explanations are verbal-only
Slide 11
Elaborated Explanations Tutor
Slide 12
Elaborated Explanations Tutor
Slide 13
Elaborated Explanations Tutor
Demo of the Geometry Cognitive Tutor with
Elaborated Explanations
New & Improved!
Now with more
explanations!
Slide 14
Connections to PSLC Theory What are the relevant knowledge components?
(Verbal) Geometry principles.
E.g., Inscribed Angle Theorem means that the measure of the
angle is half the measure of the intercepted arc.
(Visual) Geometry elements.
E.g., Recognizing angles, arcs, and their relationships.
(Integrated) Geometry inferences E.g., Recognizing that an arc, which is associated with a
known (or found) inscribed angle, can be found via the
Inscribed Angle Theorem
Slide 15
Knowledge Components vs. Overall Visual Match
Slide 16
Knowledge Components vs. Overall Visual Match
Slide 17
Mapping Given Information to Elements
Slide 18
Integration of Principles and Elements
Slide 19
Superficial Strategies of Integration: Close = Connected
Slide 20
Robust Knowledge: Relationships connect Elements via Principles
Slide 21
Difficulty Factors Analysis (DFA): Problem Format & Explanation Type
3 Problem Formats Diagram Quadrant Table
2 Explanation Types Simple Explanations (Reasons Only) Elaborated Explanations (Reasons + Application)
Slide 22
DFA: Diagram Problem Format with Simple Explanations
Slide 23
DFA: Diagram Problem Format with Elaborated Explanations
Slide 24
DFA: Quadrant Problem Format with Elaborated Explanations
Slide 25
DFA: Table Problem Format with Elaborated Explanations
Slide 26
DFA Results: Given Information
Linear trend for Explanation Type, F (1, 88) = 3.8, p = .055
Performance by Problem Format & Explanation Type
50
55
60
65
70
75
80
85
90
95
100
Diagram Quadrant Table
Problem Format
Perc
en
t C
orr
ect
Simple Explanations
Elaborated Explanations
Slide 27
DFA Results: Problem Solving
Performance by Problem Format & Explanation Type
0
5
10
15
20
25
30
35
40
45
50
Diagram Quadrant Table
Problem Format
Perc
en
t C
orr
ect
Simple Explanations
Elaborated Explanations
Linear trend for Explanation Type, F (1, 88) = 2.9, p = .09Quadratic effect for Problem Format, F (1, 88) = 3.8, p = .053Trend for interaction, F (1, 88) = 3.0, p =.088
Slide 28
Preliminary Results: Process
Observational pilot data Longer latency of responses in table condition BEFORE entering
quantities Longer latencies AFTER quantities entered when elaborated
explanations are required
Classroom Feedback Teachers report student preference for diagram tutor Students report no perceived differences in the “amount of work”
for the elaborated explanations Students adapt quickly to the elaborated explanations, but
performance far from ceiling even after successful completion of tutor with simple explanations.
Slide 29
Next Steps
Log files??????!!!! Think-aloud protocols with elaborated explanations
Summer 2006
Lab testing of elaborated explanations Summer 2006
In-vivo testing with the elaborated explanations & contiguous interface (2 X 2) Late Fall 2006
Slide 30
Research Team Vincent Aleven: Research Scientist, CMU HCII Kirsten Butcher: Research Postdoc, Pitt LRDC Shelley Evenson: Assoc Prof, CMU School of Design Octav Popescu: Research Programmer, CMU HCII Andy Tzou: Masters Student: CMU HCII Honors Program Carl Angiolillo: Masters Student: CMU HCII Honors Program Grace Leonard: Research Associate, CMU HCII Thomas Bolster: Research Associate, CMU HCII
Slide 31
Questions?
Slide 32
Extra Slides
Slide 33
Existing Tutor: Multiple Verbal Inputs
Slide 34
Existing Tutor: Multiple Visual Inputs
Slide 35
Table Condition = Noncontiguous
Slide 36
Diagram Condition = Contiguous
Slide 37
Methods: Contiguity (Study 1)
Geometry Cognitive Tutor: 2 conditions Table (noncontiguous) Diagram (contiguous)
Procedure Pretest (in class) Training (classroom use of tutor, grade-matched pairs
randomly assigned to conditions within classes) Posttest (in class)
Slide 38
Assessment: 3 types of items
Answers
Slide 39
Assessment: 3 types of items
Reasons
Slide 40
Assessment: 3 types of items
Transfer
Slide 41
Preliminary Results: AnswersHigher and Lower Ability Students'
Performance on Answers (Solvable)
0
10
20
30
40
50
60
Pretest Posttest
Test Time
% C
orr
ect
Table Low
Table High
Diagram Low
Diagram High
Main effect of test time: F (1, 38) = 29.5, p < .01
Slide 42
Preliminary Results: ReasonsHigher and Lower Ability Students'
Performance on Reasons
0
10
20
30
40
50
60
Pretest Posttest
Test Time
% C
orr
ect
Table Low
Table High
Diagram Low
Diagram High
Main effect of test time: F (1, 38) = 65.7, p < .01
Slide 43
Preliminary Results: TransferHigher Ability Students: Transfer Performance
0
5
10
15
20
25
30
35
40
Pretest Posttest
Test Time
% C
orr
ect
Table HighDiagram High
Lower Ability Students: Transfer Performance
0
5
10
15
20
25
30
35
40
Pretest Posttest
Test Time
% C
orr
ect
Table LowDiagram Low
3-way interaction: Test Time * Condition * Ability: F (1, 38) = 4.3, p < .05