Post on 15-Jan-2016
description
Emily R. Fyfe & Bethany Rittle-JohnsonVanderbilt University
Marci S. DeCaroUniversity of Louisville
The Effects of Feedback During Exploratory Math
Practice
1
SREE Fall 2011
How do children learn best?
Two schools of thought have emerged
2
• Advocates of Direct Instruction(Kirschner, Sweller, & Clark, 2006)
• Advocates of Discovery Learning(Bruner, 1961; Kuhn, 1989; Piaget, 1973)
An Integrated Perspective
No need for this strict dichotomy
3
“A mixture of guidance and exploration is needed” (Mayer, 2004)
“Proponents of constructivism and direct instruction…each have something to learn from the other” (Rieber, 1992)
“Characterizing discovery and direct instruction as diametrically opposed…has done a disservice to both approaches” (Wilson, et al, 2010)
“There’s a place for both direct instruction and student-directed inquiry” (Kuhn, 2007)
Combining Instruction and Discovery
Exploration prior to instruction facilitates learning(DeCaro & Rittle-Johnson, 2011; Schwartz & Bransford,
1998)
Hints or coaching during problem solving is better than pure problem solving alone
(Mayer, 2004)
Recent meta-analysis indicates that guided discovery is better than unassisted discovery or direct instruction
(Alfieri et al, 2010)
4
Questions remain…
Which elements of instruction are most suitable to incorporate within exploration?
For whom is guided exploration most advantageous and why?
5
Feedback During Exploration
Feedback represents one element of instruction that may be particularly effective in combination with exploration
What is Feedback?• Any information that the learner can use
to confirm, reject, or modify prior knowledge
• Accuracy information, hints, etc.
6
Types of Feedback
Outcome Feedback• Provides information
about learner’s answer
• Used extensively
• Related to strong, positive effects in past work compared to no feedback
Strategy Feedback• Provides information
about how answer was obtained
• Only examined in a few previous studies
• Better than outcome feedback in terms of strategy selection
7(Ahmad, 1988; Kluger & DeNisi, 1996; Luwel et al., 2011)
Why feedback?
May reduce disadvantages of discovery by guiding the learner’s search for information
Helps identify errors and encourages search for plausible alternatives (e.g., new strategies)
(Hattie & Timperley, 2007; Mory, 2004)
But…• Past research indicates variable efficacy
(Kluger & DeNisi, 1996)
• May only benefit a subset of learners8
What about prior knowledge?
Expertise reversal effect• Instructional technique is effective for novices,
but loses its benefits for high-knowledge learners
(Kalyuga, Ayres, Chandler, & Sweller, 2003)• Example: worked examples vs. problem solving
Low knowledge learners benefit from more external guidance; high knowledge learners benefit from less
Perhaps feedback during exploration only helps children who have low prior knowledge
9
Why the reversal?
Due to existing cognitive resources(Paas, Renkl, & Sweller, 2003)
• Novices lack schemas; need external guidance to reduce cognitive load
• High-knowledge learners have schemas; additional guidance creates more cognitive load
10
Goals of this study
Examine the effects of feedback during exploratory math practice for children with varying levels of prior knowledge
Specifically• Compare feedback vs. no feedback • Compare outcome vs. strategy feedback• Look at effects of prior knowledge
11
Hypotheses
12
Strategy Feedback > Outcome Feedback
Feedback > No Feedback
Feedback better for children with low prior knowledge
Ho 1:
Ho 3:
Ho 2:
Domain: Mathematical Equivalence
Concept that two sides of an equation represent the same amount and are interchangeable• Commonly represented by equal sign
(=)
13
3 + 7 + 8 = 3 + _3 + 7 + 8 = 3 + _ 6 + 4 = _ + 86 + 4 = _ + 8
Why Study Math Equivalence?
Fundamental concept in arithmetic and algebra
Very difficult for children in U.S.• Interpret equal sign as an operator symbol that
means “the total” as opposed to relational symbol
(McNeil, 2008; Rittle-Johnson & Alibali, 1999)
• In one study, only 24% of U.S. children in 3rd and 4th grade solved math equivalence problems correctly
(McNeil & Alibali, 2000)
14
Participants
Worked with 91 children (2nd & 3rd grade)- M age = 8 yrs, 7 mo- 53 females, 38 males- 45% white, 40% black, 15% other- 47% receive free or reduced lunch
15
Design and Procedure
Session 1: Pretest (~25 minutes)• Excluded if score >80% on pretest
measures
Session 2: Intervention & Posttest (~50 minutes)
Session 3: Two-week Retention Test (~25 minutes)
16
Tutoring Intervention
Exploratory Practice• Attempt to solve 12 math equivalence
problems• Randomly assigned to 1 of 3 conditions
• No Feedback (n = 31)• Outcome Feedback (n = 32)• Strategy Feedback (n = 28)
MidtestBrief conceptual instruction
(DeCaro & Rittle-Johnson, 2011)
17
Exploratory Practice
Find the number that goes in the blank.3 + 4 + 8 = 3 + ☐
How did you solve that problem?
No Feedback: “OK, let’s move on to the next problem.”
Outcome Feedback: “Good try, but that’s not the correct answer. The correct answer is 12.”
Strategy Feedback: “Good try, but that’s not a correct way to solve that problem.”
18
Assessment of Math Equivalence
Procedural Knowledge• Use correct strategy to solve problems
Conceptual Knowledge• Understand concept of equivalence
19
7 + 6 + 4 = 7 + _7 + 6 + 4 = 7 + _
What does the equal sign mean?
What does the equal sign mean?
4 + 8 = 8 + 4True or False?
4 + 8 = 8 + 4True or False?
(Rittle-Johnson, Matthews, Taylor, & McEldoon, 2011)
Used at Pretest, Midtest, Posttest, & Retention Test
6 - 4 + 3 = _ + 36 - 4 + 3 = _ + 3
Analysis & Results
Contrast-based ANCOVA (West, Aiken, & Krull, 1996)
• Two contrast-coded condition variables- Feedback (no feedback vs. two feedback conditions
combined)
- Feedback Type (outcome feedback vs. strategy feedback)
• Two condition x prior knowledge interactions
• Three covariates
Interaction follow-up• Categorize as low vs. high knowledge
(median split)
• Simple main effects of condition 20
Procedural Knowledge
21
Repeated Measures ANCOVA: Midtest, Posttest, and Retention Test.
Procedural Knowledge
22
Overall feedback x prior knowledge interaction, F(1, 83) = 7.05, p = .01Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 3.84, p = .05High Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.61, p = .04
*
*
Procedural Knowledge Results
Expertise reversal effect
Feedback during exploratory math practice is more beneficial than no feedback, but only for children with low knowledge
For children with high prior knowledge, the reverse is true; they benefit more from no feedback
23
Intervention Activities
Subjective Cognitive Load (NASA-TLX, Hart & Staveland, 1988)
• I had to work hard to solve those problems.• I was stressed and irritated when I had to
solve those problems.• Mean rating on agreement scale from 1 to 5.
Problem-Solving Strategy Use• Variability of correct & incorrect strategies
24
Cognitive Load
25
Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 1.15, p = .29High Knowledge: Feedback vs. No Feedback, F(1, 83) = 6.05, p = .02
*
Cognitive Load
Supports expertise reversal effect explanation
Feedback may have hurt high-knowledge learners’ performance because of increased cognitive load
An effect not found for low-knowledge learners
26
Strategy Coding Scheme
27
Strategy Sample explanation (4 + 5 + 8 = __ + 8)
Correct Strategies
Equalize I added 4, 5, and 8 and got 17. And 9 plus 8 is 17.
Add-Subtract I added 4, 5, and 8 and got 17. And 17 minus 8 is 9.
Grouping I took out the 8’s and I added 4 plus 5.
Incorrect Strategies
Add-All I added the 4, 5, 8 and 8.
Add-to-Equal I just added the first three, the 4, 5, and 8.
Carry I saw a 4 here, so I wrote a 4 in the blank.
Ambiguous I used 8 plus 8, and then 5.
Perseveration
28
* P = .01
*
Perseveration = Using the same incorrect strategy on all the problems.
Incorrect Strategy Variability
29
*
Ranged from 0 to 5.Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 5.66, p = .02High Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.56, p = .04
*
Correct Strategy Variability
30
*
Ranged from 0 to 3.Low Knowledge: Feedback vs. No Feedback, F(1, 83) = 4.76, p = .03High Knowledge: Feedback vs. No Feedback, F(1, 83) = 0.16, p = .90
Strategy Variability
For children with low prior knowledge, feedback prevented perseveration and led to the generation of diverse strategies
- May explain why these children learned more when they received feedback than when they did not
31
For children with higher prior knowledge, feedback led to the generation of more incorrect, but not correct strategies
- May help explain why feedback had negative impact
Summary
Feedback led to higher procedural knowledge of math equivalence than no feedback, but only for children with low prior knowledge
For children with high prior knowledge, no feedback was better
No differences between outcome feedback and strategy feedback
32
Implications
Theoretical• Extends expertise reversal effect to
feedback• Clarifies research on discovery learning
Practical• Pay more attention to when you give
feedback during tutoring and teaching
33
Thank You
Children’s Learning Lab
Bethany Rittle-Johnson
Marci DeCaroLaura McLean
Maryphyllis CreanLucy Rice
34
Funding SourcesExpERT Training Grant, through IES
to FyfeNSF CAREER grant to
Rittle-Johnson
Instruct vs. Discover
Direct Instruction• Explore the problems on your
own with no guidance
• Find target information and new strategies independently
• But…• Overwhelms working
memory• Might never locate target
info or invent correct strategy
Discovery Learning• Told or shown how to
solve the problems
• Provides structure and reduces task ambiguity
• But…• Limits self-discovery• Might limit learner
engagement
35(Kirschner, Sweller, & Clark, 2006; Mayer, 2004; Sweller, 1988)
Exploratory Practice
36
Children’s performance on 12 problems during intervention (Percent Correct)
Instruction
Let’s take a look at this problem.3 + 4 = 3 + 4
There are two sides to this problem, one on the left side of the equal sign and one on the right side of
the equal sign…
The equal sign means that the left side of the equal side is the SAME AMOUNT AS the right side of
the equal sign. That is, things on both sides of the equal sign are equal or the same.
37
Conceptual Knowledge
38
Feedback type x prior knowledge interaction, F(1, 83) = 4.82, p = .03.Low knowledge: No effect of feedback type, F(1, 83) = 0.51, p = .48High knowledge: Main effect of feedback type, F(1, 83) = 8.21, p = .005
Intervention – Strategy Use
39
*
*
*
Note. Differences are between no feedback condition and two feedback conditions combined: * p < .05
Incorrect Correct
Strategy Variability
40
Note. Difference is between no feedback condition and two feedback conditions combined: * p < .001
* *