The Effectiveness of Corrective Feedback on Chinese EFL Writers’

Grammatical Accuracy Improvement

Dongmei Cheng

Northern Arizona University

Motivation

No general conclusion on the efficacy of grammatical corrective feedback (e.g. Truscott, 1996, 2007; Ferris, 1997, 1999, 2004, 2006)

Motivation

No consensus on how explicit feedback should be: – More explicit feedback: Providing metalinguistic

information helps (Bitchener, 2008; Bitchener et al, 2005; Lalande, 1982)

– Less explicit feedback: Learners only need guidance in locating the errors (Chandler, 2003; Lee, 1997; Robb, Ross & Shortreed, 1986)

Motivation

Few studies has truly adopted “a sustained, systematic, replicable manner that would allow for comparisons across similar or different contexts and student populations” (Ferris, 2004, p.55).

Motivation

There’s a need to adopt a controlled design, which has a comparable selection of– corrective feedback types– targeted error types– Writing tasks

Research Questions

RQ: Does two types of indirect corrective feedback help to reduce the amount of targeted grammatical errors in students’ argumentative writings over time? If both types of feedback help, which one is more effective? • Feedback type 1: Highlighting the targeted errors • Feedback type 2: Highlighting the targeted errors

and providing abbreviated error codes

Research Questions

Time (Within two months)– Time 1: First Draft of Essay 1 (Week Three)– Time 2: Revised Draft of Essay 1 (Week Five)– Time 3: First Draft of Essay 2 (Week Eight)

Participants

• Total participants N=95

(Female N=60; Male N=35)• Participants came from three College English

classes taught by the same EFL instructor in a big public university in Northern China.

(original participant pool N=120)

Participants

• All college freshmen from three major areas: Administrative Management, Economics and International Accounting.

• Have no regular English writing practices and multiple-draft writing experiences.

• Are generally competent English readers but relatively weak in speaking and writing.

Participants

Age College Entrance Exam Score

Years of English Learning

Mean 18.69 125.19 8.39

Standard

Deviation

0.75 12.76 1.87

Design

Two experimental groups + A control group All learners in the three classes (initial participants’

pool N=120) were randomly assigned to three different types of feedback:

– Highlighting the targeted errors (Highlight)– Highlighting the targeted errors with abbreviated codes

(Code)– No grammatical feedback (None)

Procedure

All participants submitted their first essays electronically through the school online course platform.

The EFL instructor downloaded the essays and sent them to the researcher.

The researcher provided grammatical feedback using the “highlighter” and “comment” function on Microsoft Word.

The researcher sent back the grammatically commented essays to the EFL instructor.

The EFL instructor added in brief content and organization feedback for all participants (including the control group)

Variables

Independent variables:– Feedback type (Highlight, Code vs. None)– Time (Essay 1; Revised Essay 1 vs. Essay 2)

Dependent variable: Targeted error score (per essay)=

occurrence of targeted error(s)*100

total word

Argumentative Writing Tasks

Two essay topics from the IELTS Task Two Writing prompt pool.

Selection criteria: – Participants are familiar with both topics. – The two prompts fit in the “agree vs. disagree”

type, which provides a claim and asks the participants to what extent they agree or disagree with the given claim.

Argumentative Writing Tasks

Essay 1:

More and more women go out to work. Is it the government’s responsibility to subsidize them and provide free staff and facilities to care for their children? To what extent do you agree or disagree to this idea?

Argumentative Writing Tasks

Essay 2:

The government is responsible for protecting a nation’s cultural identity. Thus, some people believe new buildings should be built in traditional styles. To what extent do you agree or disagree with this opinion?

Targeted Errors—Type I

• Type I: • Noun endings (NE)

e.g. This phenomenons become more and more common among office ladies.

More and more woman go out to work.

Targeted Errors—Type II

Type II: Verbs– Subject-Verb Agreement (SV)

The government have responsibility to subsidize them. Culture exist in every one’s spirit.

– Verb Tense (VT) As the society developed, more people want their lives

to be more comfortable and convenient. To protect or rebuilt old buildings of traditional styles is

more meaningful than build some ones.

Targeted Errors—Type II

Type II: Verbs– Subject-Verb Agreement (SV)– Verb Tense (VT)– Verb Form (VF)

If new buildings be built in traditional styles, people will pay more cost and can’t adapt to live in these buildings.

In order to protect the nation’s culture, what we really need to do is to protect the treasure we have already own, like old buildings, old paintings and so on.  

Targeted Errors—Type III

Type III: Sentence Structure– Sentence Fragment (SF)

Here^ my reasons. So the government should subsidize the women. Because now people have a lot of pressure of modern

life. And finding job is easier.

Targeted Errors—Type III

Type III: Sentence Structure– Sentence Fragment (SF)– Run-on Sentences (RO): (including comma

errors) There is a woman go to work. Some problems appeared, who will care for their

children. A series of traditions are left which have a profound

effect on us.

Targeted Errors—Type IV

Type IV: Word Form – Wrong word formations including nouns,

adjectives and adverbs (WF) The construct which represent a nation’s history are of

vital importance. Mothers always show their children with patient. When they get home, they also need to work hard to

keep the room cleanly.

Reliability Check

Intra-rater reliability: 25% of the essays were coded again by the researcher after two months. Pearson Correlation r= .97

Inter-rater reliability: 25% of the essays were coded by a second coder.

Pearson Correlation r=.91

Results—Descriptive Statistics

Raw Error Score

Essay 1 Revised Essay 1

Essay 2

Highlight M=5.74

SD=2.51

M=2.74

SD=2.39

M=5.29

SD=3.33

Code M=5.74

SD=2.90

M=1.19

SD=1.38

M=3.48

SD=2.32

None M=7.23

SD=3.43

M=6.30

SD=3.23

M=7.30

SD=3.98

Raw Error Occurrences

Results—Descriptive Statistics

Normed Error Score

Essay 1 Revised Essay 1

Essay 2

Highlight M=3.69

SD=1.88

M=1.57

SD=1.40

M=3.29

SD=2.79

Code M=3.54

SD=1.62

M=0.70

SD=0.86

M=2.07

SD=1.46

None M=3.03

SD=1.52

M=2.72

SD=1.29

M=3.05

SD=1.48

Normed Error Scores

0

0.5

1

1.5

2

2.5

3

3.5

4

Essay1 RevEssay1 Essay2

Highlight

Code

None

Results—Group One (Highlight)

Alpha was set as .05. Test: Within-subject repeated measure

ANOVA RQ1: Does one type of indirect feedback

(Highlight) help to reduce the error scores in students’ essays over time?

Results—Group One (Highlight)

F(2)=14.97, p<.05 There is a significant time effect on the

highlighting group’s error scores. Results from pairwise comparisons:

– Essay1 vs RevEssay1: p<.05– Essay1 vs Essay2: p>.05

Results—Group Two (Code)

• RQ2: Does another type of indirect feedback (code) help to reduce the error scores in students’ essays over time?

• Results from repeated measure ANOVA:

F(2)=33.58 p<.05

Results—Group Two (Code)

There is a significant time effect on the error scores of participants’ who received coded feedback.

Pairwise comparison results:– Essay1 vs. RevEssay1: p<.05– Essay1 vs. Essay2: p<.05

Results—Control Group

Repeated ANOVA results:

F(2)=.788, p>.05• There is no significant time effect on control

group’s error scores.

Results—Between-subject

Test: Between-subject repeated measure ANOVA

Results:

F(2)=4.40, p<.05• There is a significant difference in the error

scores over times among the three groups.

Results—Between-subject

Pairwise comparisons:– Highlight vs. Code: p<.05– Highlight vs. None: p>.05– Code vs. None: p<.05

There’s a significant difference on error scores over time between the two experimental groups.

The “Highlight” group showed NO significant difference on error scores over time compared to the control group.

The “Code” group showed a significant difference on error scores over time compared to the control group.

Error Scores from the Three groups Over time

Conclusion

Both types of indirect feedback helped to reduce the targeted errors in students’ argumentative essays significantly in the short term (Essay1RevEssay1)

The more explicit feedback (Code) helped to reduce the targeted errors significantly from Essay 1 to Essay 2 (in a longer term)

Highlighting the targeted errors and providing abbreviated codes had a more significant effect (compared to only highlighting the errors) on error reduction in students’ essays over time.

Pedagogical Implications

EFL students could potentially benefit from more explicit corrective feedback on their argumentative writings.

Providing grammatical codes helps the students to notice and understand the nature of their grammatical errors in the essay.

A mutual understanding on the meanings of the error codes needs to be established beforehand. (e.g. an error chart)

Mean Error Scores for Noun Ending Errors

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Essay1 RevEssay1 Essay2

Highlight

Code

None

Mean Error Scores for Verb Errors

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Essay1 RevEssay1 Essay2

Highlight

Code

None

Mean Error Scores for Sentence Structure Errors

0

0.5

1

1.5

2

2.5

Essay1 RevEssay1 Essay2

Highlight

Code

None

Mean Error Scores for Word Form Errors

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Essay1 RevEssay1 Essay2

Highlight

Code

None

Results for Word Form Errors

Within-subject repeated measure ANOVA– Highlight: F(2)=.039, p>.05no sig time effect– Code: F(2)=4.34, p<.05sig time effect– Control: F(2)=18.54, p<.05sig time effect

Between-subject repeated measure ANOVA– F(2)=2.51, p>.05no sig different difference

among the effect of the three feedback types on the word form error scores

Implications of the unexpected findings on word form errors

Learners perhaps have difficulties correctly interpret the feedback on word form errors.

Compared to the other three types of targeted errors, word form errors might be relatively easier to be ignored by the learners, despite the provision of feedback.

More explicit classroom instruction needs to be done to address the correct use of word forms in context (vocabulary instruction).