Tsovaltzi etal ectel2010

Learning from Online ErroneousExamples

Dimitra Tsovaltzi, Erica Melis, Bruce McLaren,

Ann-Kristin Meyer, Michael Dietrich, Goerge Goguadze

DFKI- University of Saarland - [email protected]

Summary

• Research background

• Research questions

• Studies

• Summary of Results

• Discussion of results

• Conclusion

Research Background

• Erroneous Examples (EE): worked out solutions with errors

• Novel learning opportunities, reflection, inquiry (Borasi ’95, Müller ’03, Oser&Hascher ’97)

• Benefit of self-explaining correct and incorrect solutions (Siegler ’02, Siegler&Chen ’08)

• Evidence for erroneous examples with feedback (Kopp et al ’08)

Research Background

BUT

• May be more beneficial to students with favourable prior knowledge (Grosse&Renkl ’07)

• „why“ self-explanations indispensible (Siegler ’02,

Grosse&Renkl ’08), but to the detriment of principle-based explanations (Grosse&Renkl ’08)

Help for self-explanation of errors and principle-based explanations

Adaptation to counterbalance prior knowledge differences

Summary



• Studies



• Conclusion

Research Questions

Online Learning of Fractions with Erroneous Examples and Self-explanation

When?• Do advanced students, gain more from online erroneous examples?

How?• Can online EE improve:

cognitive skills?conceptual understanding?transfer abilities improve?

• Can online EE improve error detection and error correction? • Does adaptive help play a role?

Research Questions

• Hypothesis 1: Presenting erroneous examples with help to students will lead to deeper, more conceptual learning and better error-detection (i.e., metacogntive) skills, which will help improve their cognitive skills and will promote transfer.

control group (problem solving) vs. erroneous examples and erroneous examples with help

• Hypothesis 2: The learning effect of erroneous examples is conditional on whether students are supported in finding and correcting the error with additional help.

Help vs. no help

Summary



• Studies



• Conclusion

• Similar design: – NOEE (control): standard exercise (standard feedback,

correct answer)

– EEWOH: standard exercise and erroneous examples without extra help (standard feedback, correct answer)

– EEWH: standard exercise and erroneous examples with extra help

• Different levels: 6th vs. 7th-8th vs. 9th-10th

• Presentation of erroneous examples

Studies

• Design – Familiarisation

– Pre-questionnaire

– Pretest

– Intervention

– Posttest

– Post-questionnaire

• Participants– paid volunteers in lab studies - German 6th grade

– EEWH (8), EEWOH (7), NOEE(8)

Study 1: Method

• Familiarisation

• Intervention– Standard exercises

– Erroneous examples

• Sequences (6):

– NOEE: SE – SE - SE

– EEWH: SE – EE – SE help

– EEWOH: SE – EE – SE no help

Study 1: Materials

Please write all individual thinking steps as if youwere thinking aloud. Add more steps whenever youneed to.

Add steps

Results

• Familiarisation

• Intervention– Standard exercises

– Erroneous examples

• Sequences (6):

– NOEE: SE – SE - SE

– EEWH: SE – EE – SE help

– EEWOH: SE – EE – SE no help

Study 1: Materials

Feedback consists of:

• Correct(√) / incorrect(X) (All conditions)

• error-awareness (EEWH): “The result cannot be smaller than one“, if two fractions larger than one are added

• self-explanation (EEWH): “Why is this step wrong?”, “How should Paul add?”

• error-correction (EEWH):“How does one add fraction with like denominators”

• error-specific (EEWH): “You did not expand the fractions”

• Worked-out correct solution (All conditions)

Feedback in ActiveMath

• Pretest and posttest

– Similar exercises plus erroneous examples with conceptual questions: “What did Paul not understand?”

• Questionnaires

motivation, self-efficacy, learning orientation,

cognitive load, error-awareness, critical thinking

Study 1: Materials

Study 1: Results

Condition EEWH N=8 EEWOH N=7 NOEE N=8

Score Subscore mean(sd)% mean(sd)% mean(sd)%

Cognitive

Skills

Pretest 80.2(26.7) 85.7(17.8) 86.5(12.5)

Post-pre-diff -2.1(33.6) 1.2(21.7)^ 2.1(23.9)+

Metacognitive

Skills (EE)

EE-find 91.7(15.4)+ 76.2(31.7)^ 66.5(35.6)

EE-correct 80.2(12.5)+ 75.0(21.0)^ 68.7(25.9)

EE-ConQuest* 64.6(25.5)+ 60.2(33.3)^ 41.7(21.2)

EE-total 75.3(16.8)+ 67.9(27.5)^ 54.7(23.0)

Total-time-on-postEE 16.9(6.2)^ 13.8(5.5)+ 18.0(5.1)

Transfer Transfer 75.0(46.2)+ 71.4(48.8) 75.0(46.3)^

• Metacognitive for finding error: EEWH>NOEE (t(20)=2.37, p<.05 , d=1.06)

• Metacognitive for total EE:

– Main (t(20)=2.34, p<.05 , d=1.01)

– EEWH>NOEE (t(20)=2.96, p<.05 , d=1.32)

• Conceptual questions: Main (t(20)=2.48, p<.05 , d=1.11)

• EEWH lower cog load (F(2,13)=7.76, p=.006, n2=0.54)

• Design as in Study 1, but one modeling exercise 7 sequences

• Participants

– paid volunteers in lab studies – German 7th and 8th

grades

– EEWH (8), EEWOH (8), NOEE (8)

Study 2: Method

Study 2:Materials

“2 groups of students get a pizza each. In thefirst group there are 3 students, 2 of whomare girls. In the second group there are 5students, 4 of which are girls. The pizza issplit equally within every group. Karl is tryingto calculate what part of the pizza the girls ofboth groups got together. His result is ¾ of apizza. Karl has made an error. Find the errorin Karl’s calculations. Choose the firsterroneous step.”

Study 2: Results


Score Subscore mean(sd)% mean(sd)% mean(sd)%

Cognitive

Skills

Pretest 73.7(26.7) 71.2(19.7) 77.9(12.4)

Post-pre-diff 2.4(24.4)^ -4.3(26.6) 6.9 (17.9)+

Metacognitive

Skills (EE)

EE-find 68.7(34.7) 75.0(13.4)^ 90.6(12.9)+

EE-correct 57.8(26.7)^ 54.7(21.1) 65.6(20.8)+

EE-ConQuest* 55.2(46.5) 62.5(12.6)+ 61.5(19.4)^

EE-total 59.3(37.1) 63.7(11.9)^ 69.8(15.0)+

Total-time-on-postEE 8.1(4.3)+ 11.5(4.2)^ 15.5(4.8)

Transfer Transfer 45.2(45.8)^ 38.0(36.0) 67.3(28.5)+

Conc. Underst. Modelling 36.4(42.2)^ 19.8(35.0) 40.8(48.6)+

• Term-grade sig. covariate for conc. questions (F(1,21)=4.49, p=.047, n2 =.18)

• More students could find the error than correct (t(23)=4.89, p<.05 , d=0.59):– EEWH (t(7)=2.19, p>.05 , d=1.64)– EEWOH (t(7)=4.83, p<.05 , d=1.15)– NOEE (t(7)=4.32, p<.05 , d=1.44)

• NOEE more cog. load drop than EEWOH (t(13)=2.52, p<.05, d=1.9)

• Design as in Study 1 and 2, but – conceptual sequences: “addition as increasing”, “part

of whole”

– Transformation exercises: 3/5+1/4

– More elaborated “how” questions

– Classroom for ecological validity

– Order of sequence: SE – SE- EE

• Participants– German school kinds in 9th and 10th grades

– EEWH (18), EEWOH (20), NOEE(19)

Study 3: Method

Study 3: Materials

Error Detection Phase

Students find the error

Error-awareness feedback“The result, walking distance=5 1/30, cannot be correct. Travel with the bus is already 4/5 of the total distance, so the walking distance

must be less than 1/5”

Step #: walking distance=…path

Study 3: Results - Cognitive Skills


Type of score Type of Subscore mean(sd)% mean(sd)% mean(sd)%

Cognitive

Skills

Pretest 74.5(14.2) 66.4(21.1) 64.9(17.2)

Diff-post-pre-total 8.9(12.8)+ 1.4(23.5) 4.9(18.8)^

Transform 16.2(23.0)+ 4.9(33.2)^ -10.2(45.4)

[A1]E.i. postEE[A2]In results file was named Add-subtr-total

•Cog. Skills: EEWH vs. EEWOH (t(30)=2.13, p<.05 , d=0.58)

•Transform: •main (t(30)=2.42, p<.05 , d=0.66)

•EEWH vs NOEE: (t(23)=2.87, p<.05 , d=0.97)

•More cog load reduction:•EEWH vs. NOEE (t(30)=2.22, p<.05, d=0.24)

•EEWH vs. EEWOH (t(28)=2.05, p=.05, d=0.14)

Study 3: Results - Metacognitive Skills

Condition EEWH N=18EEWOH

N=20NOEE N=19


Metacognitive

Skills (EE)

EE-find 61.1(28.7)+ 50.0(28.1) 60.5(28.0)^

EE-correct 40.3(28.0)+ 21.3(30.6) 30.3(33.9)^

EE-ConQuest* 50.9(20.7)+ 50.4(24.9)^ 47.8(25.1)

EE-total 50.8(22.1)+ 44.5(24.0) 46.8(24.7)^

Total-time-on-EE 5.9(3.2)+ 4.1(3.1) 5.9(3.9)+

• Significantly less students found the error than could correct it t(56)=, p<.001 , d=0.87

• Also within individual conditions• EEWH t(20)=3.83, p<.05 , d=0.66

• EEWOH t(19)=5.88, p<.001 , d=0.98

• NOEE t(18)=5.75, p<.001, d=0.97

Study 3: Results - Transfer

Condition EEWH N=18EEWOH

N=20NOEE N=19


Transfer

Cog-transf-total 32.0(30.1)+ 20.0(34.3) 29.0(34.6)^

Conc-transf-total* 46.8(34.7)+ 30.4(29.3)^ 29.5(30.30)

Transfer-total 39.4(20.3)+ 24.3(26.8) 26.5(28.6)^

[A1]E.i. postEE[A2]In results file was named Add-subtr-total

• EEWH better, but• No significant results

Study 3: Results – ConceptualKnowledge



Conceptual

Understanding

Part-of-whole 11.1(47.3)+ -5.0(59.4)^ -9.9(44.6)

Addition-as-incr 65.3(44.7)+ 56.3(48.6)^ 30.5(46.4)

Subtr-as-decreas 52.9(49.9)+ 27.5(44.4) 34.2(47.3)^

Rel-part-of 22.2(42.8)^ 7.5(24.5) 23.7(42.1)+

Modelling-total 54.5(30.5)+ 33.1(24.6) 35.6(27.4)^

• Significant results include: Modelling in generalo EEWH vs EEWOH (t(30)=2.10, p<.05 , d=0.58)

Modelling “addition as increasing” o Main (t(54)=2.32, p<.05 , d=0.63)

o EEWH vs NOEE (t(23)=2.35, p<.05 , d=0.64)

• Problem with “part of a whole”




Conceptual

Understanding

Part-of-whole 11.1(47.3)+ -5.0(59.4)^ -9.9(44.6)

Addition-as-incr 65.3(44.7)+ 56.3(48.6)^ 30.5(46.4)

Subtr-as-decreas 52.9(49.9)+ 27.5(44.4) 34.2(47.3)^

Rel-part-of 22.2(42.8)^ 7.5(24.5) 23.7(42.1)+

Modelling-total 54.5(30.5)+ 33.1(24.6) 35.6(27.4)^

• Significant results include: Modelling in generalo EEWH vs EEWOH (t(30)=2.10, p<.05 , d=0.58)

Modelling “addition as increasing” o Main (t(54)=2.32, p<.05 , d=0.63)

o EEWH vs NOEE (t(23)=2.35, p<.05 , d=0.64)

• Problem with “part of a whole”

Summary



• Studies



• Conclusion

Cognitive Skills

Problem Solving Ceiling effect Ceiling effectEEWH >EEWOH

TransformationMain effect

EEWH >NOEE EEWH >EEWOH

MetacognitiveSkills

Finding error EEWH >NOEE

Correcting error

total-EEMain effect

EEWH>NOEE

Finding vs. correctingAcross conditions

NOEE>EEWOHAcross conditions

NOEE, EEWOH>EEWH

Conceptual

UnderstandingMain

EEWH >NOEE Term grade covariate

Modeling [not done] EEWH >EEWOH

Model. “addition as increasing”

[not done] [not done]Main effect

EEWH >NOEE

Summary



• Studies



• Conclusion

Hypothesis 1: cog., metacog., transfer, concept. for EEWH

• Maybe conceptual knowledge also promotes cognitive skills

• No effects for other levels but maybe due to ceiling effect, or due to less conceptual material

• No transfer, but maybe basic-concept should be made explicit

• Metacognitive skills dependent on level

• Dissociation between declarative vs. procedural knowledge (Ohlsson ‘96), but students learned

Discussion

Hypothesis 2: Help vs. no Help• Microadaptation: more effects for help• EE with help to self-explain errors take advantage of

learning opportunities (Ohlsson ‘96)• Conceptual, principled-based help is useful (van Gog et

al ‘04)• Correcting the error may not be important lower

costOther results: When• Macroadaptation: EE after practice with SE• Despite adaptive help, class level may be important

Discussion

Summary



• Studies



• Conclusion

• EE can be beneficial

• Previous results on EE and WE in other domains transfer to only EE and in fractions (Siegler ’02; Siegler&Chen ’08; Grosse&Renkl‘07)

• Analogues to aptitude-treatment from Große&Renkle(07), grade-level importance

• Like Kopp et al (08), help better

Conclusion

Learning from ErroneousExamples

Dimitra Tsovaltzi, Erica Melis, Bruce McLaren, Ann-Kristin Meyer, Michael Dietrich, Goerge Goguadze

DFKI- University of Saarland - [email protected]

Thank you!

mailto:[email protected]

Tsovaltzi etal ectel2010

Documents

Transcript of Tsovaltzi etal ectel2010