1

The Use of Feedback in Self-Regulated Strategy Development: A Systematic Review

A Thesis Submitted in Partial Fulfillment of theRequirements of the Renée Crown University Honors Program at

Syracuse University

Jessica Claar

Candidate for Bachelor of Science Degreeand Renée Crown University Honors

May 2020

Honors Thesis in Psychology

Thesis Advisor: _______________________ Dr. Tanya Eckert, Associate Professor

Thesis Reader: _______________________ Dr. Bradley Seymour, Assistant Teaching Professor

Honors Director: _______________________ Dr. Danielle Smith, Director

2

© Jessica Claar 2019-2020

3

Abstract

Writing is a foundational skill taught to all students across the nation. Despite this, many students cannot write according to expectations set for them. Foundational aspects of writing, such as spelling, grammar, and punctuation, are taught in early elementary school. Students who are not proficient in these areas early on, will be more likely to struggle as their education continues (National Commission of Education, 2003). Self-Regulated Strategy Development (SRSD) is one writing instructional approach that attempts to address students’ writing difficulties by teaching planning, revising, and editing skills associated with written compositions. One component often included in SRSD is performance feedback, and research suggests that student success increases when provided with performance feedback (Butler & Winne, 1995). This systematic review analyzes the use of performance feedback in SRSD studies among elementary-aged students. A total of nine (n = 9) studies that utilized SRSD were included in this systematic review. A series of one-way analysis of variance (ANOVAs) were conducted to examine the impact of providing performance feedback within the context of SRSD studies and examined several conditions, including (a) use of performance feedback, (b) feedback type, (c) method of administering feedback, and (d)education type. There was found to be no statistically significant differences between any of the conditions, with each condition having a large effect size (range: 0.25 to 3.19) regardless of whether SRSDs studies included performance feedback or did not include performance feedback. The study concludes by discussing how SRSD can be effectively implemented in different ways across a wide range of classroom settings.

Keywords: SRSD, performance feedback, writing instructional interventions

4

Executive Summary

Literacy rates among adolescents are declining in the United States. Experts point to

elementary school as one of the most critical points for literacy development. Effective

communication and understanding through writing will set individuals up for success later on in

their education and will carry through into their contribution to the work force. This project is a

systematic review of empirical studies that focus on improving writing outcomes of elementary

school students. In particular, one writing intervention was found to be especially helpful in

improving writing scores for children. This intervention is entitled Self-Regulation Strategy

Development (SRSD) and relies on a process of internal thought and outside editing to help

students develop a successful “writer’s mind”. The intervention models the way students should

internally think through a writing assignment in order to accurately and effectively communicate

their ideas. There has been previous research examining the effectiveness of SRSD in improving

students’ writing performance, with many meta-analyses demonstrating that it is highly effective.

However, there has not been extensive research into the role of performance feedback within the

SRSD model. Performance feedback can be defined as an outside source evaluating a student’s

writing and providing suggestions on how to improve. Within the SRSD model, feedback can be

administered by various outside sources (including teachers, research assistants, and computers)

and in various ways (oral or written). This project examines the efficacy of performance

feedback when used within SRSD.

One meta-analysis conducted by Graham et. al (2012) identified SRSD as the most

effective writing intervention of several that were examined. For the purposes of this systematic

review, studies were extracted from the Graham et. al (2012) meta-analysis that used SRSD as a

writing intervention method and were published in a peer-reviewed journal. The process of

5

finding studies from the meta-analysis involved sorting through all of the studies included and

determining which ones were published in peer-reviewed journals. Next, the selected studies

were analyzed to see which ones used SRSD as their writing intervention process. There was a

total of nine studies (n=9) from the meta-analysis that fulfilled the inclusionary criteria.

Statistical analyses (ANOVAs) were used to with the studies to determine if there was a a

statistically significant difference between four conditions. The first condition was whether the

study operationally defined and used performance feedback or not. The second condition was

whether the study utilized oral feedback or a combination of written and oral feedback. The third

condition was whether feedback was administered by a teacher or a research assistant. The fourth

condition was whether the interventions were conducted with students eligible for special

education, students not eligible for special education, or both.

The results of the statistical analyses found that there were not statistically significant

differences between any of the conditions. All of them were found to have very high effect sizes,

indicating that they all were very effective in improving writing performance of elementary

school students. It is interesting to note that studies that utilized performance feedback within an

SRSD model were not more effective than studies that did not implement performance feedback

within an SRSD model. Both conditions were highly effective. The role of performance feedback

in SRSD cannot be concretely determined by this finding, but one can argue that performance

feedback is largely not an integral part of SRSD, based on this systematic review. The results did

strongly support previous research that SRSD is a highly effective writing intervention

technique. The new information that this systematic review provides is the efficacy of SRSD

across a broad array of education types, feedback types, and methods of administering feedback.

6

This contributes to the field of psychology by showing educators that SRSD can be implemented

in virtually any setting and produce positive results.

7

Table of Contents

Abstract……………………………………….……………….………….. 3

Executive Summary………………………….……………….………….. 4

Preface……………………………………….……………….…………… 9

Acknowledgements…………………………………………… 9

Advice to Future Honors Students………………………….. 10

Chapter 1: Introduction ……………………………………………… 11

The Significance of Writing ……………………………….…....... 11

Student’s Writing Outcomes in the United States………...………. 11

Theoretical Conceptualization of Writing……………….…………. 13

Writing Instructional Practices……………………………………... 14

Self-Regulated Strategy Development………………...…….……. 15

Performance Feedback…………………………………….………... 17

Purpose of the Present Study……………………….…………….. 19

Chapter 2: Method ……………………………………………… 21

Protocol and PRISMA Guidelines.………….………………………. 21

Eligibility Criteria …….………….…………………………………. 21

8

Variables………….…………………………………………………. 22

Descriptive Measures……………………………. 22

Outcome Measures………………………………. 22

Chapter 3: Results ……………………………………………… 23

Descriptive Findings ………….……………………………………. 23

Inferential Results …………………………………………………. 23

Chapter 4: Discussion ……………………………………………… 25

The Role of Performance Feedback in SRSD Research……………. 26

Limitations …………………………………………………………. 29

Implications …………………………………………………………. 29

Areas for Future Research…….……………………………………. 30

Conclusions……………..…….……………………………………. 30

References.……………………………………………………………… 31

Appendices………………………………………………………………… 35

9

Acknowledgements

First and foremost, I’d like to thank my thesis advisor, Dr. Tanya Eckert. Thank you so much for

your continued support on this project. You have successfully and carefully guided me through

this entire thesis, and I quite literally could not have done it without you. I’d also like to thank

Professor Seymour for agreeing to read my whole thesis and help with the editing process. You

have been one of my favorite professors and I am incredibly thankful to learn from you. Finally,

to my friends and family, thanks for not only putting up with me, but being the best cheerleaders

while I figured this whole thesis thing out. My undergraduate career certainly ended differently

than I expected, but still I want to say thank you to Syracuse University for all the wonderful

experiences these last four years!

10

Advice to Future Honors Students

Congratulations on making it this far into your undergraduate career! Try to see this

thesis as an opportunity to perform research into a topic of personal interest. When conducting

research and writing a thesis for the honors program, choose something that you are interested in.

There is a plethora of disciplines that students’ write their Honors’ Thesis on. Although it seems

obvious, you are going to put a lot of time and energy into this undergraduate thesis. If your

primary passion is researching declining literacy rates in the United States and your thesis is

about studying a protein in the eye of an earthworm, it’s going to be a long two years. Choose an

advisor who you connect with and who can help make your project the best it can be. Finally,

work on it every day or at least a couple times a week. Fix small things, like formatting and

citations, early on. Enjoy the process while keeping up to date with deadlines. This project will

undoubtably provide a deeper understanding into your chosen field and provide a talking point

for future interviews and applications!

11

Chapter 1

Introduction

The Significance of Writing

Writing is one of the most important forms of communication. It provides people with the

opportunity to connect across distance and time and is necessary for success in the development

of literacy (Graham & Peris, 2007). Specifically, writing allows for self-expression,

comprehension of ideas, and forming connections between those ideas. Although writing is a

foundational skill taught at schools across the nation, many students cannot write. The problem

lies in students’ inability to write well enough to fulfill expectations set for them. Students are

generally able to be understood through their writing. However, they lack the skill set to produce

extended or complex thought (The National Commission on Writing, 2003). Additionally,

adolescents who cannot write well are more likely to struggle through school than their peers.

They are also less likely to attend college, because universities focus on writing as an integral

part of the application process (Graham & Peris, 2007). Furthermore, when entering the work

force, adults who cannot communicate through writing (i.e., creating emails, memos), may

struggle to obtain jobs that pay a living wage (Graham, Harris, & Hebert, 2012). In summary,

writing bears importance in everyday life, and those who cannot write well are at an overall

disadvantage.

Students’ Writing Outcomes in the United States

Research completed at the national level indicates that a large portion of students are

unable to write according to their grade specific guidelines. Assessments often analyze the ability

of the writer “to persuade, to explain, and to convey experience” through their writing (The

12

National Center for Education Statistics [NCES], 2012). In 2011, the National Assessment of

Educational Progress (NAEP) examined the writing abilities of students in the United States. A

staggering 72% of eighth-grade students and 74% of twelfth-grade students were found to be

functioning below the proficient level (NCES, 2012). Although more than half of the students in

both grade levels were able to complete basic writing requirements, they failed to demonstrate

appropriate mastery of the writing skills for their respective grade level. These findings are

indicative of a nationwide problem and require further examination. These students have been

attending school for several years and are still unable to fulfill writing expectations.

These findings have important implications regarding instructional efforts for

elementary-aged children. Foundational aspects of writing are formed in early education.

Spelling, grammar, and punctuation are taught in elementary school. If students are not proficient

in these areas, then they will likely struggle later in their education (National Commission of

Education, 2003). In 2002, the NAEP investigated the writing performance of fourth-grade

students. Within this study, an examination was made of the percentage of students achieving

proficiency for their grade level, which was defined as being able to provide a response that

demonstrated understanding of the writing task, with minimal spelling and grammar errors.

Results indicated that only 28% of fourth-grade students in the United States were at or above

the Proficient level (Persky, Daane, & Jin, 2003). In this same study, the results were consistent

for students in eighth and twelfth grade, with proficiency estimates ranging between 24% and

31%. These findings illustrate how critical elementary education writing is for students’ success.

Without these foundational skills, elementary aged students remain unable to keep up with

expectations set for them as they progress through middle and high school.

13

Theoretical Conceptualizations of Writing

The act of writing requires complex planning and cognitive reasoning. It is not simply the

physical action of dragging a pen across a paper or typing on a keyboard. Conceptualizing

human thought into characters that are understood by others without spoken word is

immeasurably complex. A few theoretical conceptualizations have attempted to decipher this

process. One is Hayes’ and Flowers (1980) model, which explains how adults think to

themselves while writing. In this model, adult writers go through different steps in order to

produce their intended written information. The steps are (a) planning, (b) translating, and (c)

reviewing. The model states that the steps are not necessarily sequential. They work in

conjunction with one another and interact with each other at given points in time. Specifically, an

adult writer may progress from one step to another, and then revert to the first step with new

information (Abbott & Berninger, 1993). However, this model applies to older writers, beginning

in sixth grade and continues into in adulthood.

When considering this model, it is important to remember that children are not just

smaller versions of adults. Beginning writers' brains do not process information in the same way

as advanced writers. The way that children conceptualize writing, and the process accompanied

with it, varies greatly from adult writers. As a result, elementary writing should not be viewed as

a less intense version of skilled adult writing (Abbott & Berninger, 1993). To account for this,

Berninger and Graham (1997) created a model that expands on Hayes’ and Flowers (1980) ideas

but considers the minds of developing writers. The Simple View of Writing attempts to describe

the conceptual thought process that beginning writers undergo. The Simple View of Writing

(Berninger & Graham, 1997) can be exemplified by a triangle divided into three sections (see

Figure 1). The two base components of the triangle are transcription skills (i.e., ability to

14

represent language as symbols that can be understood by others) and self-regulation executive

functions (i.e., planning, reviewing, and revising). These two base components symbolically

support the top of the triangle, which contains the third component of the model: text generation

(i.e., using working memory involving language representation).

There is empirical support for this model among emerging writers. In first and second

grade, foundational aspects of writing, such as spelling ability, take precedence. If students do

not learn these initial skills, they are unable to write using higher order “thinking” processes

(Juel, Griffith, & Gough, 1986). For example, when students are unable to spell a word or

construct a sentence, the resulting composition is not accurate in representing the idea in their

mind. On the other hand, when students spell words correctly and accurately construct sentences,

they are positively reinforced as their writing is reflective of their thought process (Berninger et

al., 2002). Additionally, higher cognitive processes should begin by the end of second grade as

students are able to make connections and infer ideas in their writing (Juel et al., 1986).

Writing Instructional Practices

General education writing instruction is highly variable across the nation. The attention a

teacher places on individual aspects of writing can change across school districts, and even vary

in classrooms within the same school. Teachers’ writing instruction generally emphasizes basic

writing skills, such as spelling and grammar. For example, based on a survey of general

education teachers, Cutler and Graham (2008) reported that up to 75% of teachers provide

instruction in basic writing skills every day. Teachers reported that sentence structure and

paragraph skills were taught multiple times a week, but not every day. Further, while the

majority of teachers emphasized basic writing skills, they reported utilizing a number of different

15

strategies to assist their written instruction, with the majority of practices not supported by

empirical evidence.

Additionally, teachers often overestimate the extent to which they alter their instructional

practices for struggling writers in their classrooms. In another survey of general education

teachers (Graham, Harris, Fink-Chorzzempa & MacArthur, 2003), 45% reported engaging in

more writing instruction adaptations for underperforming students than were actually observed.

Specifically, one in every five teachers made no adaptations to help their students who were

having a hard time keeping up with their classmates writing skills. These findings are

disconcerting because accommodations for underperforming students are an integral part of

writing instruction as a whole. Furthermore, children in urban schools, primarily from low

income households, benefit when they are exposed to methodical writing instruction (Graham &

Harris, 2005).

In summary, while students continually fail to meet grade-level writing expectations,

developing instructional practices and policies to address this need must come from an

understanding of current writing instruction, as experienced by students in schools across the

country (Cutler & Graham, 2008). These instructional practices and policies will need to

incorporate new procedures but must also account for how frequently these procedures are

applied.

Self-Regulated Strategy Development

One instructional approach that attempts to address students’ writing difficulties is Self-

Regulated Strategy Development (SRSD). This is a writing instructional practice that teaches

planning, revising, and editing written compositions for students. In a full SRSD model, there are

five stages through which students progress. The first stage is entitled Develop and Activate

16

Background Knowledge. This refers to students identifying basic elements of a prewritten story.

The second stage, Discuss It, is when instructors discuss with the students (a) the goals of the

assignment, (b) why these goals are important, and (c) how to incorporate the individual parts of

the story to accomplish these goals. In this stage, it is pertinent that instructors emphasize the

need for students’ effort in accomplishing their goals. The third stage is Model It. This is when

instructors model the thought process students should undergo when attempting their goals (e.g.,

‘What is my next step?’). Memorize It, stage four, is when students begin memorizing the

strategy steps, often using mnemonics. Finally, the fifth stage is Support It. This stage involves

using the strategy developed in the previous steps to plan and write a story (Harris & Graham,

1999). The students are expected to utilize the goals they made for themselves as a guide

throughout all the stages.

An example of an SRSD instructional practice includes a student reading a story and

identifying the main character, setting, and aspects of the plot. Then, the instructor will say to the

student, “The goal of this assignment is to write a one-page story with a main character, setting,

and clearly defined plot.” The instructor will periodically check in with the student and model for

them to ask themselves “What should I do next in order to accomplish my goal?” Students will

then memorize their strategy steps to accomplish their goal. When they are finished writing they

will compare their final product to the goals they made. These students will ask themselves “Did

I write a one-page story with a setting?” and “Do I have a main character?”. They will compare

their written performance with their initial goal setting.

Graham and Perin (2007) performed a meta-analysis that examined the effectiveness of

123 studies that utilized various writing instructional practices. A total of 20 of the 154 total

effect sizes calculated were based on SRSD instructional practices. The results of this meta-

17

analysis indicated that SRSD was found to be the most effective method for teaching writing

strategies to students in grades 4-8 (mean weighted effect size = 1.14). Also, SRSD was found to

be a particularly beneficial method for struggling writers, in addition to the general population of

students. As noted previously, SRSD includes many instructional components, such as planning,

goal setting, and revision. An additional component embedded in SRSD is providing students

with performance feedback regarding their writing performance.

Performance Feedback

Feedback has generally been defined as information presented by an outside party, such

as a teacher, peer, or parent, that evaluates ones’ performance. Examples include corrective

information, encouragement, and alternative solutions (Hattie & Timperley, 2007). There are

several early theories within the field of psychology that examined feedback. These theories,

most notably Thorndike’s Law of Effect (Thorndike, 1931) and the Social Cognitive Theory

(Bandura, 1991), were seminal in the development of contemporary models of feedback.

Thorndike’s Law of Effect initially included two components. The first was the law of

reward, the idea that with a reward, a person had an increased probability of performing the

desired response. The second component was the law of punishment, the idea that punishment

decreased the probability of performing the desired response (Marks, 2011). In 1931, Thorndike

removed the second component, the law of punishment, but kept the law of reward. His new Law

of Effect related an increase in the probability of a particular behavior in a specific environment

to a rewarding response (Thorndike, 1931). Simply put, Thorndike found that a reward (e.g.

positive feedback) increased desired behavior.

Moreover, Bandura’s Social Cognitive Theory of Self-Regulation (1991) suggests that

humans are proactive in achieving goals and regulate their behavior with feedback. This theory

18

explains human behavior as strongly influenced by a self-monitoring mechanism. Bandura

argues that the amount of self-monitoring depends heavily on whether attention is spent on ones’

successes or failures. Desired behavior increases when monitoring successes, while attention to

failures results in little change, and can even lower performance (Bandura, 1991). Similar to

Thorndike’s findings, Bandura discovered that attention to positive self-feedback produces better

results, and that attention to negative self-feedback produces worse or similar results. Further,

when self-motivation is combined with external feedback, individuals can perform at their best.

A negative feedback loop occurs when a person seeks to lessen the discrepancy between their

internal goals and their outward performance (Carver & Scheier, 1982). This creates a closed

loop that eventually stops producing results. Comparative feedback from an outside party is

necessary to regulate motivation. However, individuals often set goals before receiving feedback

regarding their efforts. The discrepancy-reducing negative feedback loops assists individuals in

accomplishing preestablished goals, but occasionally individuals must self-monitor to break free

from the loop and begin a new challenge (Bandura, 1991).

Butler and Winne (1995) proposed that a broad model of self-regulation that is relevant to

student learning and is theoretically related to feedback. In their work, Butler and Winne argued

that self-monitoring is an integral part of a self-regulated task, such as writing, and this

monitoring generates internal feedback. Additionally, external feedback affects a student’s

confidence in their ability to complete a task and their final performance on the task. As a result,

in their work, Butler and Winne posit that self-monitoring through self-regulation creates the

space for performance feedback and helps to optimize student success.

19

Most recently, Hattie and Timperley (2007) performed a review of meta-analyses

regarding feedback. This meta-meta-analysis consisted of 12 meta-analyses which focused on

feedback in the classroom. They reviewed 6,972 effect sizes from 196 different studies. The

average effect size from feedback was 0.79, which reflects a large impact on student

performance. In their study, they reported that the studies with the largest effect sizes included

feedback regarding how students should complete a task more effectively. Additionally, they

found that smaller effect sizes were associated with task-irrelevant components, such as

compliments and awards. Further, Hattie and Timperley reported that feedback was most

effective in the classroom when it was intentionally task directed and not focused on punishment.

With this information, Hattie and Timperley (2007) developed a model of feedback. The

model consists of three consecutive questions. The first question is Where Am I Going? This

question refers to goal setting (e.g., passing a test). The second question is How Am I Going?

This question is typically answered by a teacher or a peer. It holds constant the idea that students

need information about their progress and how to proceed. The third and final question is Where

to Next? Students should develop this on their own, after receiving feedback from the second

question. They will self-regulate their goals and develop something new (i.e. enhanced

challenges). Within this model, self-regulation plays a prominent role. Students are taught to

monitor their progress through received feedback.

Purpose of the Present Study

Because current research supports the claim that student success increases with their

attention to performance feedback (Butler & Winne, 1995) and one component frequently

included in SRSD is performance feedback, the purpose of the present study is to conduct a

systematic review of the extent to which SRSD focusing on elementary-aged students writing

20

incorporates performance feedback. Although prior meta-analyses have examined the efficacy of

SRSD, none have examined how inclusion or exclusion of various model components of SRSD

impacts students’ outcomes. As a result, this synthesis examined whether SRSD studies that

incorporated performance feedback resulted in improved students’ writing outcomes. In addition,

SRSD studies that incorporated performance feedback and reported strong or moderate evidence

were reviewed to examine associated study characteristics and student demographics, which

included: (a) use of feedback (i.e., whether the method section of the original study operationally

defined feedback or not), (b) type of feedback (i.e., written or oral), (c) method of administering

feedback (i.e., teacher, research assistant, computer), (c) student grade level (i.e., elementary,

middle, or high school), and (d) student educational status (i.e., eligible for special education vs.

not eligible for special education).

To address the primary aims of the study, four research questions were developed:

(1) Was there a statistically significant difference between effect sizes of SRSD studies that

incorporated feedback and those that did not?

(2) Was there a statistically significant difference in effect sizes of SRSD studies that used

different types of feedback (i.e., written or oral)?

(3) Was there a statistically significant difference in effect sizes of SRSD studies that used

different methods of administering feedback (i.e., teacher, research assistant, or

computer)?

(4) Was there a statistically significant difference in effect sizes of SRSD studies based on

special education eligibility?

21

Chapter 2

Method

Protocol and PRISMA Guidelines

For the purpose of this systematic review, a review protocol was created (see Appendix

A). Although prior registration of systematic review protocols is recommended to increase

transparency and reduce reporting bias (Moher et al., 2009), the protocol was not formally

registered because (a) this review was based on studies included in a prior meta-analysis (i.e.,

Graham et al., 2012) and (b) this review was completed to fulfill an undergraduate honor’s

program thesis requirement. However, the procedures for this review were aligned with PRISMA

guidelines (Moher et al., 2009), which provides evidence-based reporting procedures to improve

the quality of systematic reviews.

Eligibility Criteria

There were eligibility requirements that governed which studies were selected to be

analyzed in this systematic review. The first inclusionary criterion was that the study was

included in the Graham et al. (2012) meta-analysis on the effectiveness of writing instructional

practices in elementary grades. In the Graham et al. (2012) meta-analysis, the specified search

procedure was done in four parts. First, there were electronic searches (including up until

October 2010) on the following databases: ERIC, PsycINFO, Education Abstracts, ProQuest,

and Dissertation Abstracts. Additionally, eighteen psychology and research journals were hand

searched, previous meta-analyses were examined, and the reference pages of the included articles

were searched. In order for a study to be included in the meta-analysis, the criteria included the

study was a true experiment, the participants were students in elementary school (some schools

22

included grade level up to sixth grade), the study performed a writing intervention within a

treatment group that included a posttest measure, was presented in English, and contained

necessary statistics to compute an effect size. A total of 115 articles met the inclusionary criteria.

Using the studies identified in the meta-analysis conducted by Graham et al. (2012),

additional inclusionary criteria were applied. Specifically, to be included in the present

systematic review studies were required to be (a) published in peer-reviewed journals (i.e.,

studies published as dissertations or conference presentations were excluded), and (b) included

SRSD as an instructional practice.

A total of 115 articles were used in the meta-analysis conducted by Graham et al. (2012).

However, only 108 articles were identified from the reference section, and were included for this

review. Following the removal of studies that were not published in peer-reviewed journals (n

=53) or did not include SRSD as an instructional practice (n =46), a total of nine articles met the

predetermined inclusionary criteria. Figure 2 outlines the search and inclusionary criteria and

Table 1 provides a summary of the nine included studies.

Variables

Descriptive measures. All study variables were defined and summarized (see Appendix

A). I coded study features (i.e., general or special education classroom), participant

characteristics (i.e., grade level), and SRSD intervention characteristics (i.e., type of feedback

and method of administering feedback).

Outcome measure. The primary outcome measure that was extracted from the Graham

et al. (2012) meta-analysis was the postintervention effect size, which provides a standardized

metric for comparison across studies. This meta-analysis computed average weighted effect sizes

for writing interventions that were tested in four different investigations in their selected articles.

23

SRSD was one of these interventions. They formula used in this meta-analysis to compute effect

size was to subtract the treatment group mean score at the posttest from the control group mean

score at the posttest and then divide this by the pooled standard deviation of the two groups

(Graham et al., 2012).

Chapter 3

Results

Descriptive Findings

The demographic characteristics of the participants from the studies included in this

systematic review appears in Table 2. There were seven studies (77.8%) that conducted SRSD

interventions with elementary school students (i.e., aged 5-10 years), and four studies (44.4%)

involving middle school students (i.e., aged 11-13 years). There were no studies involving high

school students (i.e., aged 14 – 18 years). The discrepancy in the total percentage for age range is

because a single study could have conducted an SRSD intervention with both elementary and

middle school students.

A total of six studies (66.7%) included students eligible for special education and eight

studies (88.9%) included students that were not eligible for special education. Again, the total

percentages were calculated knowing that the studies could include both students who are

eligible for special education and students who were not eligible for special education.

Inferential Results

To address the first research question, a one-way between subjects analysis of variance

(ANOVA) was conducted, which examined whether there was a statistically significant

difference between effect sizes of SRSD studies that incorporated feedback and those that did

24

not. Results indicated that there was not a statistically significant difference between effect sizes

of SRSD studies that incorporated performance feedback and those that did not, F (1,7) = .0001,

p = 0.98. Large effect sizes were reported for SRSD studies that incorporated performance

feedback (M = 1.28; SD = .965) and the single SRSD study that did not incorporate performance

feedback (M = 1.26).

To address the second research question, a one-way between subjects ANOVA was

conducted, which examined whether there a statistically significant difference between effect

sizes of SRSD studies that used feedback that was oral or a combination of oral and written.

Results indicated that there was not a statistically significant difference between the effect sizes

of SRSD studies with feedback incorporating feedback as oral or a combination of oral and

written F (1,7) = .0001, p = 0.98. Large effect sizes were reported for SRSD studies that used

oral feedback (M = 1.148; SD = .711) and studies that used a combination of oral and written

feedback (M = 1.50; SD = 1.46).

To address the third research question, a one-way between ANOVA was conducted,

which examined whether there was a statistically significant difference between effect sizes of

SRSD studies that administered feedback by teacher, research assistant, or a combination of

both. Results indicated that there was not a statistically significant difference between the effect

sizes of SRSD studies that administered feedback by teacher, research assistant, or used a

combination of both, F (2,5) = .353, p = 0.71. Large effect sizes were reported for SRSD studies

that administered feedback by teacher (M = 1.72; SD = 2.07), research assistant (M = 1.23; SD

= .592), and a combination of both (M = .640).

Finally, to address the fourth research question, a one-way ANOVA was conducted,

which examined whether there a statistically significant difference between effect sizes of SRSD

25

studies conducted with general education students, special education students, or a combination

of general and special education students. Results indicated that there was not a statistically

significant difference between the effect sizes of SRSD studies in general education, special

education, or a combination of both, F (2,6) = .160, p = 0.85. Large effect sizes were reported for

SRSD studies that included only general education students (M = 1.54; SD = 1.50), special

education students (M = 1.26), or a combination of general and special education students (M =

1.12; SD = .650).

Chapter 4

Discussion

SRSD is an instructional writing practice that has been proven as effective in a broad

range of settings. It has been taught in schools across various grades and education types. Within

the Graham et al. (2012) meta-analysis, SRSD was found to be the most effective treatment

approach for writing interventions among elementary-aged children. An integral part of SRSD is

the incorporation of performance feedback, as students are provided with information about

areas where they have been successful and areas where improvement is needed. However,

previous SRSD research has not separately examined the contributions of performance feedback

across various classroom and student types. As a result, the purpose of this study was to conduct

a systematic review of the extent to which SRSD focusing on elementary-aged students’ writing

incorporated performance feedback. In the remaining sections of this discussion, I summarize the

results of analyzing performance feedback in SRSD instructional interventions across classroom,

student, and education style, which address the primary aims of the study.

26

The Role of Performance Feedback in SRSD Research

In this systematic review, a total of nine studies were identified that indicated

incorporating performance feedback as part of SRSD based on the Graham et al. (2012) meta-

analysis. To address the first research question, I examined whether there was a statistically

significant difference between effect sizes of SRSD studies that incorporated feedback and those

that did not incorporate feedback. Results indicated that there was no statistically significant

difference in the effect sizes for those SRSD studies that included performance feedback and the

single SRSD study that did not. Regarding performance feedback, Bandura’s Social Cognitive

Theory of Self- Regulation (1991) explains that motivation in writing is often stunted when

students write on their own but can reappear when an outside source provides positive feedback.

Students create a negative feedback loop when they desire to decrease the discrepancy between

their goals and their performance (Carver & Scheier, 1982). This systematic review found that

when students (both those eligible for special education and those not eligible for special

education) were provided with feedback (both oral and written) by an outside source (either

teacher or research assistant), they showed improvement in their overall written performance.

While students improved overall, as indicated by the large effect sizes, it is debatable whether

performance feedback is an essential part of the effectiveness of SRSD. This fails to support the

ideas of Bandura, Carver, and Scheier in their previous research, because SRSD with

performance feedback was not found to be significantly more effective than SRSD without

performance feedback. However, it is important to note here that only one study did not indicate

use of performance feedback, so the conclusions drawn based on the sample of studies examined

is limited.

27

To address the second research question, I examined whether there was a statistically

significant difference between effect sizes of SRSD studies that used oral feedback and those that

used both oral and written feedback. Results indicated that there was no statistically significant

difference in the effect sizes for those SRSD that used oral feedback and those that used both

oral and written feedback. The Hattie and Timperley (2007) review of 12 meta-analyses

analyzing feedback in the classroom found that the largest study effect sizes were from those that

utilized feedback for students on how to most effectively complete an assigned task. The results

from the present systematic review corroborate the their results reported in the review of meta-

anaalyses. SRSD studies that incorporated performance feedback as either oral or oral and

written were found to have large effect sizes and proved to be beneficial for students. Regardless

of feedback type, students writing improved tremendously.

To address the third research question, I examined whether there was a statistically

significant difference between effect sizes of SRSD studies that used teacher administered

feedback or research assistant administered feedback. Results indicated that there was no

statistically significant difference in the effect sizes for those SRSD that used teacher

administered feedback and those that used research assistant administered feedback. As a result,

these findings suggest that the benefits of improving writing among young students cannot be

improved using a variety of feedback sources. This particular result should come as a comfort to

teachers nationwide. The National Commission of Education (2003) found that if students are

unable to write well early on, they are much more likely to have a more difficult time in the

classroom as they continue their education. Teachers are attempting to prepare students to

continuing their education and become successful contributing members of society. SRSD has

proven effectiveness when different people administer the feedback. Teachers could use an

28

assistant in the classroom to help them administer SRSD, and it would be just as effective. This

should take some pressure off of teachers in elementary classrooms. Students’ writing improves

when SRSD is implemented, and that is the ultimate goal.

To address the fourth and final research question, I examined whether there was a

statistically significant difference between effect sizes of SRSD studies for students eligible for

special education and students not eligible for special education. Results indicated that there was

no statistically significant difference in the effect sizes for those SRSD studies that included

students eligible for special education and students not eligible for special education. As a result,

these findings suggest that all students can benefit from SRSD instructional practices in their

classrooms. Cutler and Graham (2008) found that instructional practices, in order to be effective,

needed to display an awareness of how students in classrooms actually experience current

writing instruction. The results from this systematic review strongly support SRSD’s

effectiveness in real-life settings. The studies in this systematic review were conducted in actual

schools across the country. There were large effect sizes for both students eligible for special

education and students not eligible for special education. As a result, it can be concluded that

SRSD successfully works within the real-life experience of young writers and improves their

writing outcomes.

In summary, the results of this systematic review confirm that large effect sizes were

associated with SRSD interventions, regardless of whether performance feedback was

incorporated into the SRSD intervention. In addition, no differences were found in the magnitude

of effect sizes for those SRSD studies that included performance feedback, as a function of

feedback type or educational status of the students.

29

Limitations

The most notable limitation of this systematic review is the small sample size (n = 9) of

studies that were included from the Graham et al. (2012) meta-analysis. This resulted in some

analyses only containing one or two studies. Thus, it is difficult to confirm results based on a few

studies. It is possible that different findings would have been observed if more studies were

included in the systematic review. In addition, one recommended practice for systematic reviews

is to conduct interscorer agreement across coded categories (e.g., effect sizes, classifications). No

interscorer agreement was collected, which may have resulted in limited reliability in the coding

of studies included in the review. Finally, the effect size estimates were based on computations

reported in the meta-analysis conducted by Graham and colleagues (2012). Without external

verification, it is impossible to assert that there were no computational errors in the calculation of

effect sizes used for the purposes of this systematic review.

Implications

First, the greatest implication of this current systematic review is that SRSD’s use of

performance feedback can be extrapolated to a wide range of classroom styles, age groups, and

methods of administering feedback in the classroom. Teachers hoping to improve students’

literacy skills can be encouraged that students will benefit equally whether their performance

feedback is administered orally or written. Further, both students eligible for special education

and those not eligible for special education can benefit from the performance feedback aspect of

SRSD. Finally, the method of administering feedback, whether by teacher or assistant, has been

shown to be equally effective. It can be inferred that utilizing performance feedback in SRSD

with any of these conditions will improve writing outcomes of elementary aged children.

30

Areas for Future Research

It would be interesting to consider the implications of SRSD in all of these areas for an

extended analysis. The results of this study suggested that SRSD is effective in a broad array of

settings and teaching styles, but further research is needed to confirm these results. It would also

be interesting to consider the effects of SRSD across different areas of the United States. Further

research could potentially compare the efficacy of SRSD in different demographics and

socioeconomic statuses.

Conclusions

As declining literacy rates in the United States are of growing concern, SRSD is a bright

spot of hope for the future citizens of this country. SRSD is a highly effective method of teaching

writing practices to students. The results of this systematic review showed no statistically

significant differences in SRSD effectiveness across classroom type, feedback type, and method

of feedback administration. It can be taken to mean that SRSD is highly effective as a writing

instructional practice for elementary students. This includes students who are struggling writers

and fail to meet grade level expectations set for them. Teachers in elementary classrooms can

definitely implement SRSD practices into their own teaching styles and if done correctly, will

most likely see positive results among their students.

31

References

Abbott, R. D., & Berninger, V. W. (1993). Structural equation modeling of relationships among

developmental skills and writing skills in primary- and intermediate-grade

writers. Journal of Educational Psychology, 85(3), 478–508. doi: 10.1037//0022-

0663.85.3.478

Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior and

Human Decision Processes, 50(2), 248–287. doi: 10.1016/0749-5978(91)90022-l

Berninger, V. W., Vaughan, K., Abbott, R. D., Begay, K., Coleman, K. B., Curtin, G., Hawkins,

J. M., & Graham, S. (2002). Teaching spelling and composition alone and together:

Implications for the simple view of writing. Journal of Educational Psychology, 94(2),

291–304. doi: 10.1037//0022-0663.94.2.291

Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical

synthesis. Review of Educational Research, 65(3), 245–281. doi:

10.3102/00346543065003245

Carver, C. S., & Scheir, M. F. (1981). Attention and self-regulation: a control-theory approach

to human behavior. New York, NY: Springer-Verlag.

Cutler, L., & Graham, S. (2008). Primary grade writing instruction: A national survey. Journal

of Educational Psychology, 100(4), 907–919. doi: 10.1037/a0012656

*Glaser, C., & Brunstein, J. C. (2007). Improving fourth-grade students’ composition skills:

Effects of strategy instruction and self-regulation for adolescent students. Journal of

Educational Psychology, 99, 445–476. doi:10.1037/0022-0663.99.3.445

Graham, S., Berninger, V. W., Abbott, R. D., Abbott, S. P., & Whitaker, D. (1997). Role of

mechanics in composing of elementary school students: A new methodological

32

approach. Journal of Educational Psychology, 89(1), 170–182. doi: 10.1037//0022-

0663.89.1.170

Graham, S., & Harris, K. R. (2005). Improving the writing performance of young struggling

writers. The Journal of Special Education, 39(1), 19–33. doi:

10.1177/00224669050390010301

Graham, S., Harris, K. R., Fink-Chorzempa, B., & Macarthur, C. (2003). Primary grade teachers’

instructional adaptations for struggling writers: A national survey. Journal of

Educational Psychology, 95(2), 279–292. doi: 10.1037/0022-0663.95.2.279

Graham, S., Harris, K., & Hebert, M. (2012). Informing writing: The benefits of formative

assessment. New York, NY: Carnegie Corporation of New York.

*Graham, S., Harris, K. R., & Mason, L. (2005). Improving the writing performance, knowledge,

and self-efficacy of struggling young writers: The effects of self-regulated strategy

development. Contemporary Educational Psychology, 30, 207–241.

doi:10.1016/j.cedpsych.2004 .08.001

Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students.

Journal of Educational Psychology, 99, 445–476. doi 10.1037/0022-0663.99.3.445

Harris, K. R., & Graham, S. (1999). Programmatic intervention research: Illustrations from the

evolution of Self-Regulated Strategy Development. Learning Disability Quarterly, 22(4),

251. doi: 10.2307/1511259

*Harris, K. R., Graham, S., & Mason, L. H. (2006). Improving the writing knowledge and

motivation of struggling young writers: Effects of self- regulated strategy development

with and without peer support. American Educational Research Journal, 43, 295–340.

doi:10.3102/ 00028312043002295

33

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational

Research, 77(1), 81–112. doi: 10.3102/003465430298487

Juel, C., Griffith, P. L., & Gough, P. B. (1986). Acquisition of literacy: A longitudinal study of

children in first and second grade. Journal of Educational Psychology, 78(4), 243–255.

doi: 10.1037//0022-0663.78.4.24

*Lane, K., Morphy, P., . . . House, E. (in press). The effects of self-regulated strategy

development for second grade students with writing and behavioral difficulties: A

randomized control trial. Journal of Research on Educational Effectiveness.

*MacArthur, C. A., Schwartz, S. S., & Graham, S. (1991). Effects of reciprocal peer revision

strategy in special education classrooms. Learning Disabilities Research and Practice, 6,

201–210.

Marks, L. E. (2011). Neurobiology of sensation and reward. Boca Raton, FL: CRC PRESS.

Moher, D. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The

PRISMA Statement. Annals of Internal Medicine, 151(4), 264. doi: 10.7326/0003-4819-

151-4-200908180-00135

National Assessment Governing Board. (2012). Writing 2011: National assessment of

educational progress. Washington, DC: U.S. Dept. of Education.

National Commission on Writing. (2003). The neglected "R": The need for a writing revolution.

New York, NY: The College Board.

Persky, H. R., Daane, M. C., & Jin, Y. (2003). Writing 2002. Washington, D.C.: U.S. Dept. of

Education, Institute of Education Sciences, National Center for Education Statistics.

34

*Sawyer, R. J., Graham, S., & Harris, K. R. (1992). Direct teaching, strategy instruction, and

strategy instruction with explicit self- regulation: Effects on the composition skills and

self-efficacy of students with learning disabilities. Journal of Educational Psychology,

84, 340– 352. doi:10.1037/0022-0663.84.3.340

Thorndike, E. L. (1931). Human learning. New York, NY: The Century.

*Torrance, M., Fidalgo R., & Garcia J.N. (2007). The teachability and effectiveness of cognitive

self-regulation in sixth-grade writers. Learning and Instruction, 17, 265–285.

doi:10.1016/j.learninstruc.2007 .02.003

*Tracy, B., Reid, R., & Graham, S. (2009). Teaching young students strategies for planning and

drafting stories: The impact of self-regulated strategy development. Journal of

Educational Research, 102, 323–331. doi:10.3200/JOER.102.5.323-332

*Wong, B. Y. L., Hoskyn, M., Jai, D., Ellis, P., & Watson, K. (2008). The comparative efficacy

of two approaches to teaching sixth graders opinion essay writing. Contemporary

Educational Psychology, 33, 757–784. doi:10.1016/j.cedpsych.2007.12.004

35

Table 1. Summary of Studies Included in Systematic Review

CitationNumber of Participants

Grade Levels

TargetedEducation

TypeEffectSize

Harris, K. R., Graham, S., & Mason, L. H. (2006). Improving the writing knowledge and motivation of struggling young writers: Effects of self- regulated strategy development with and without peer support. American Educational Research Journal, 43, 295–340. doi:10.3102/ 00028312043002295

66 2nd General and Special

Education

1.89

Graham, S., Harris, K. R., & Mason, L. (2005). Improving the writing performance, knowledge, and self-efficacy of struggling young writers: The effects of self-regulated strategy development. Contemporary Educational Psychology, 30, 207–241. doi:10.1016/j.cedpsych.2004 .08.001

73 3rd General and Special

Education

1.78

Tracy, B., Reid, R., & Graham, S. (2009). Teaching young students strategies for planning and drafting stories: The impact of self-regulated strategy development. Journal of Educational Research, 102, 323–331. doi:10.3200/JOER.102.5.323-332

127 3rd General

Education

0.25

Glaser, C., & Brunstein, J. C. (2007). Improving fourth-grade students’ composition skills: Effects of strategy instruction and self-regulation for adolescent students. Journal of Educational Psychology, 99, 445–476. doi:10.1037/0022-0663.99.3.445

113 4th General

Education

1.19

Sawyer, R. J., Graham, S., & Harris, K. R. (1992). Direct teaching, strategy instruction, and strategy instruction with explicit self- regulation: Effects on the composition skills and self-efficacy of students with learning disabilities. Journal of Educational Psychology, 84, 340– 352. doi:10.1037/0022-0663.84.3.340

56 5th & 6th General and Special

Education

0.63

36

CitationNumber of Participants

Grade Levels

TargetedEducation

TypeEffectSize

Torrance, M., Fidalgo, R., & Garcia, J.-N.(2007).The teachability and effectiveness of cognitive self-regulation in sixth-grade writers. Learning and Instruction, 17, 265–285. doi:10.1016/j.learninstruc.2007 .02.003

71 6th General

Education

3.19

Wong, B. Y. L., Hoskyn, M., Jai, D., Ellis, P., & Watson, K. (2008). The comparative efficacy of two approaches to teaching sixth graders opinion essay writing. Contemporary Educational Psychology, 33, 757–784. doi:10.1016/j.cedpsych.2007.12.004

57 6th General and Special

Education

0.64

Lane, K., Morphy, P., . . . House, E. (in press). The effects of self-regulated strategy development for second grade students with writing and behavioral difficulties: A randomized control trial. Journal of Research on Educational Effectiveness.

44 2nd General and Special

Education

0.68

MacArthur, C. A., Schwartz, S. S., & Graham, S. (1991). Effects of reciprocal peer revision strategy in special education classrooms. Learning Disabilities Research and Practice, 6, 201–210.

29 4th & 5th Special Education

1.26

37

Table 2. Descriptive Characteristics of Participants

Participant Type Percentage (n)

Elementary School Students 77.8 (7)

Middle School Students 44.4 (4)

High School Students 0 (0)

Students Eligible for Special Education 66.7 (6)

Students Not Eligible for Special Education 88.9 (8)

38

Figure 1. Simple View of Writing (Berninger et al., 2002)

39

Figure 2. Search and Inclusionary Criteria for Selected Articles

Articles that included SRSD as an instructional practice

(n=9)

Articles from Graham et al. meta-analysis (n=108)

Articles published in a peer-reviewed journal

(n=55)

Articles not published in a peer-reviewed journal

(n=52) Excluded

Articles that did not include SRSD as an

instructional practice (n=46) Excluded

40