The Instructional Practice Implications of a School Reform Model versus No School Reform Model on

Reading Achievement

A Doctoral Dissertation DefensePresented by

Adebimpe A. Odunjo

Dissertation Committee:

Rose McNeese, PhD. Tammy Greer, PhD. David Lee, PhD. Wanda Maulding, PhD. Chair Member Member Member

October 2011March M

Chapter I

•Development of Educational Reform began in the late 70’s – early 1980s. The main purpose: to address the quality of education.

•Most programs avoid measuring instructional practice when explaining student achievement outcomes. (Rowan, Correnti, Miller, Camburn, 2009)

Introduction

•Clear identification and measure of the extent of a reform program’s instructional practice implementation and its effect on accelerating or improving the learner’s achievement outcome regardless of confounding variables.

Statement of the Problem

•A closer look at the instructional practices that affect an increase in student achievement.

Purpose of the Study

1. Is there a difference in reading growth scores between students

participating in the America’s Choice reading program compared to students

who are not participating in the America’s Choice reading program?

Research Questions

2. Is there a relationship between teaching conditions and teacher’s

perception of autonomy in instructional practice?

• 1. There was a difference between reading growth

scores of students who are in the America’s Choice program and students who are not in the America’s Choice program.

• 2. There was no relationship between teaching conditions and teacher’s perceptions of autonomy in instructional practice.

Hypotheses

Theoretical FrameworkStudent Achievement

Impact of Program EffectsHistorical Background of School

Reform

Implementation Practices

Leadership Practices that Affect Teacher Autonomy

Instructional practices

To provide:

More evidence from which school districts and states can make more

informed decisions.

Clear identification of which interventions to provide more funding.

Documentation of proven instructional practice that actually impact

student achievement.

Ways to implement institutional change that will produce sustained and

increased student achievement.

Significance of Study

Chapter II

Review of Literature

There is a need to ensure that American children can effectively participate in the future job market. According to 2009 national report of 4th Graders reading ability:

What obtains

34%

33%

25%

8%

Below Basic At Basic Proficient Advanced

•The U.S. ranks 27th out of 29 wealthy countries in the proportion of college students with degrees in science or engineering •The U.S. 48th out of 133 developed and developing nations in quality of math and science instruction •In American graduate schools, nearly half of students studying the sciences are foreigners

Achievement Gap Our 15-year-olds trail nearly all other OECD countries math and science.

Up until the end of third grade, most children are learning to read. Beginning in 4th grade, they are reading to learn. •Up to half of the printed fourth-grade curriculum is incomprehensible to students who read below that grade level. •High school graduation, can be predicted with reasonable accuracy by knowing someone's reading skill at the end of third grade. A person who is not at least a modestly skilled reader by that time is unlikely to graduate from high school.

Achievement Gap By 4th grade, the majority of Black and Latino students struggle to read a simple children's book. This has devastating consequences for their future.

Source: 2009 data, National Center for Education Statistics, NAEP Data Explorer, http://nces.ed.gov/nationsreportcard/nde; Early Warning! Why Reading by the End of Third Grade Matters (Annie E. Casey Foundation).

Source: National Report Card on Higher Education, http://measuringup.highereducation.org.

The U.S. is among the leaders in college participation but ranks 16th – in the bottom half – in college completion. We only earn ½ a degree per college student, whereas in Portugal, for example, it's 1:1.

• • Close to half of the students who enter college need remedial courses: • -At Cal State, the system admits only students with at least a B average in high school, yet

37% of the incoming class last year needed remedial math, and 45% needed remedial English

• According to scores on the 2006 ACT college entrance exam, only 21% of students applying to four-year institutions are ready for college-level work in all four areas tested: reading, writing, math and biology

• Lack of preparedness leads to nearly half of all students beginning higher education by attending a community college, which has negative consequences:

• -One study showed that 73% of students entering community college hoped to earn four-year degrees, but only 22% had done so after six years

• -The Pew Charitable Trusts recently found that three-quarters of community college graduates were not literate enough to handle everyday tasks like comparing viewpoints in newspaper editorials or calculating the cost of food items per ounce

Source: At 2-Year Colleges, Students Eager but Unready, New York Times, 9/1/06.

Lack of Preparationstudents – the ones who go to college – are alarmingly unprepared.

School of Improvement by

Design

Implementation of Practices

Program

Effectiveness

Schools of Improvement by Design: build a network of educational laboratories and research development centers to promote the dissemination and utilization of innovative designs for teaching practice (Rowan, Correnti, Miller, Camburn, 2009).

Limited availability of rigorous, scientific or independent evidence CSR designs effectiveness in either implementation quality or most critically raising student achievement (Ross, Nunnery, Goldfeder, McDonald, Rachor, Hornbeck & Fleischman, 2004).

Overarching Goals of CSR

Findings on Instructional

Practice

Effects of Leadership

Teacher Autonomy

Correnti (2005) •teacher practice precedes student achievement in the causal chain of events

•Evaluating program effectiveness: case studies research overly- rely on results from standardized tests.

•Attending to the technical core of schooling only may be insufficient to change pedagogy.

Schools have failed to effectively use their institutional incentives to improve teaching practice. (Sunderman and Nardini (1994)

Goals of CSR

Student Achievement

Instructionism Constructionism

Rote Learning Meaningful Learning

Balanced Literacy Approach

Instructionism :educational

practices that are teacher-

focused, skill-based, product-

oriented, non-interactive,

and highly prescribed.

Constructivism: educational

practices that are student

focused, meaning-based,

process-oriented, interactive,

and responsive to student

interest (Johnson, 2009, pp.

90).

Student AchievementInfluencing factors

Chapter III

Methodology

Research Design

USM & District IRB

Teacher Surveys and school

demographics

Archived data: 3rd and 5th grade ITBS scores in Reading

Mixed Model AnalysisReport findings

• Archived 3rd and 5th grade ITBS average school scores made available on the school districts website for elementary schools that currently utilize the America’s Choice program and elementary schools that do not use the America’s Choice program were obtained. District demographics were extracted from the Georgia Department of Education website. 3rd and 5th grade Teachers were asked to complete surveys that contained items to assess the level of implementation of the America’s Choice program and that measure teacher autonomy in instruction

Participants

• Implementation Survey developed by Supovit and May (2004) was designed to examine the relationship between implementation and the impact of America’s Choice comprehensive school reform.

• Teacher Autonomy Scale developed by (W.W. Charters (1974) was designed to measure perceptions of educators’ autonomy of practice.

• The scores from district administered standardized reading assessment tests of the Iowa Test of Basic Skills (ITBS) , obtained from the district’s website, were used to determine growth in reading achievement due to the type of reading reform implemented at the school. Administration of the ITBS assessment allowed the district to compare the performance of its students to that of other students across the nation who took the same test at the same time of year.

Instrumentation

America’s Choice Implementation Survey

• Cronbach’s Alpha for the subscales of the instruments

were as follow: “(1) Writer’s Workshop Preparation

Scale ((1) “Writers Workshop Preparation Scale (α

= .96)a"; (2) “Readers Workshop Preparation Scale (α

= .94)a”; (3) “All Students Can Learn Scale (α = .60)b”;

and, (4) Same Standards Should Apply to All Students

Scale (α = .74)b” (Supovit and May 2004). See Appendix

A for a complete list of subscale items. This scale has

been shown to relate to teacher’s attitudinal

characteristics, teacher’s acceptance, experience,

receptiveness and teacher self reported preparation to

teach.

TAS

Cronbach’s Alpha (Pearson and Hall,

1993), for 171 cases was determined

with the total scale (18 items) internal

consistency coefficients ranging

from .80 to .83. The reliability

coefficients for the subscales

indicating general autonomy and

curriculum autonomy ranged .80

to.85. The correlation between the

general and curriculum autonomy

subscales is moderate (r = .49)

(Moomaw, 2005).

ReliabilityOf Survey Instruments

Data Analysis This study is a quasi-experimental study with mixed design. Quantitative data analysis procedures will be used to assess the

hypothesis of this study.

• 1. In order to determine whether there was a difference in reading growth scores between students participating in the America’s Choice reading program compared to students who are not participating in the America’s Choice reading program a mixed model ANOVA was conducted with type of test (vocabulary and comprehension) scores on the ITBS as a repeated measures variable and participation in the America’s Choice curriculum (yes, no) and grade ( 3 and 5) as grouping variables.

Process 1

• 2. In order to determine whether there was a relationship between teaching conditions and teacher perceptions of autonomy in practice, Pearson’s correlation was computed between scores on the Teacher Autonomy Scale and Teaching Conditions Survey.

Process 2

Chapter IV Results

ResultsQuestion 1

Results from the analysis of simple interactions, America’s Choice Participation x Grade, for each type of test were not significant. Tukey’s HSD pairwise comparison of means, however, indicated significant pairwise differences for each AC versus non AC school (HSD = 2.59 for grade 3 and 2.43 for grade 5 verbal and comprehension scores) with both verbal and comprehension scores from the non AC school significantly higher for both third and fifth graders

• Test X grade interaction• Vocabulary and Comprehension • 3rd grade (F (1,73) = 339.15, p<.001) • 5th grade (F (1,85) = 91.15, p<.001). • Mean differences between vocabulary and comprehension scores • 3 grade (MVocab = 36.16 (1.22) versus MComp = 45.51 (1.24), mean difference = 9.35), • 5th grade (MVocab = 37.40 (1.22) versus MComp = 43.63 (0.97), mean difference = 6.23).

• Type of test main effect• - (F (1,312) = 689.14, p<.001)• -Verbal (M = 36.34, SD = 9.96)• -Comprehension scores (M = 44.02, SD =8.65).

• Main effect of AC status • (F(1,312) = 6269.39, p< .001) • composite vocabulary and comprehension score averaged across grades 3 and 5 • AC schools (M = 36.25, SD =14.41) • Non AC schools (M = 4.12, SD=10.66)

ResultsQuestion 1

ResultsQuestion 1

Grade 3 Grade 5 Grade 3 Grade 50

10

20

30

40

50

60

ØACAC

• In the analysis to determine if there is a direct relationship between Teaching conditions and teacher perceptions of autonomy, it was indicated that there was no significant relationship.

•

(62) = -.092, p< .01

Question 2

Chapter V

• Of the eight schools petitioned and that initially agreed to participate in this study, only two AC schools were willing participants.

• The sample population for this study was 62.• The Socio economic status of the population

selected for this study was below the poverty level.• Access to teachers in AC schools was restricted in all

but two schools.• There were noticeable inconsistencies in the type of

data requested by the researcher that warranted using archival data rather than live data.

• The Area superintendent for the schools used in this study was unwilling to approve the study in a timely manner which affected the collection of time sensitive data.

Limitations

Response rate

Population 312

Total Sample surveyed 135Responders 62 46%

Non responders 73 54%

Population 312

Total sample 135

Responders 62 46%

Non responders 73 54%

• Research studies exploring the effects of other school reform initiatives on the education of at-risk students have also suggested that higher levels of implementation are associated with greater student performance gains.

Relationship to Earlier studies

As bad as these numbers are, they're far worse in many cities. The black male dropout rate is 80% in Indianapolis and Detroit, 69% in Baltimore and Buffalo, and 66% in Atlanta and Cleveland.

Source: The Graduation Project, 2006.

What it meansThere are large racial gaps in high school graduation rates.

Sources: U.S. Dept. of Education, National Center for Education Statistics, Common Core of Data, State-level Enrollment and Degree Attainment Data. U.S. Census Bureau, 2003 Current Population Survey, Educational Attainment in the United States, June 2004. Slide courtesy of Education Trust.

What it also meansFew black and Latino students make it to college and even fewer graduate.

• If U.S. students had met the educational achievement levels of higher-performing nations between 1983-1998, America's GDP in 2008 could have been $1.3 trillion to $2.3 trillion higher.

• •We are paying higher and higher taxes for the increasing cost of our public schools, yet they are failing

to deliver improved performance• • • To compensate for underprepared workers, U.S. industry spends about $25 billion on dropouts yearly

on remediation • Illiteracy costs American businesses more than $60 billion each year in lost productivity and health and

safety issues

• • High school dropouts: • -Are more likely to be unemployed, earn lower wages, and have higher rates of public assistance

• • -Cost our society $260,000 each in lost earnings, taxes, and productivity • -Are more likely to be single parents and have children at a young age

• •75% of America's state prison inmates and 59% of federal inmates are high school dropouts • •63% of prison inmates are functionally illiterate • •52% of African-American men who fail to finish high school end up in prison by their early 30s

The bigger pictureThe failure of so many of our schools costs our society enormously.

• Teacher preparation/selection• Set high expectations and standards,

benchmarked against international standards

• Use proven curricula • Hire and train great leaders and then

empower them • Measure results • Doug Reeves: Hold people accountable

DiscussionHow do we fix the problem?

• To find an empirical link between levels of implementation of a school reform program and instructional practices that do affect student achievement outcomes.

Recommendation for future study

• Schools that do not utilize a school reform program, but are able to develop a culture of network learning, a laser focus on the instructional core and a plan for improving instructional practice on a higher level, have a greater advantage in increasing student achievement outcomes in reading, than do prescribed programs that do not allow for much flexibility in instructional practice, are less likely to be implemented with fidelity and are designed to produce a finite type of learner.

Conclusion

• Correnti R, (2005) Literacy instruction in CSR Schools: Consequences of Design Specification on Teacher Practice. URL: http://proquest.umi.com/pqdlink?did=982790911&Fmt=2&clientId =79356&RQT=309&VName=PQD

• Charters, W. W. (1974). Sense of teacher work autonomy: Measurement & finding. Eugene: University of Oregon, Project MITT, Center for Educational Policy and Management.

• Moomaw, W. (2005) Teacher Perceived Autonomy: a Construct Validation of the Teacher Autonomy Scale.

• Rowan, B., Correnti, R., & Miller, R. J., Camburrn E. (2009). School Improvement by Design: Lessons from a Study on Comprehensive School Reform Programs

• Sunderman, G. L., & Nardini , G. (1999) Institutional Constraints on Implementing School reform: Lessons from Chicago. Paper presented at the Annual Meeting of the American Educational Research Association ( Montreal, Québec, Canada. April 19-23, 1999)

• Supovit, J.A., & May, H. (2004). A study of links between implementation and effectiveness of the America’s Choice Comprehensive School Reform Design. Journal of Education of Students Placed at Risk, 9(4), 389 – 419.

• U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service, Evaluation of the Comprehensive School Reform Program Implementation and Outcomes: Third-Year Report, Washington, D.C., 2008.

References

• Tilson, Whitney (2010) A Right Denied: A critical need for Genuine School reform

• U.S. Dept. of Education, National Center for Education Statistics, Common Core of Data, State-level Enrollment and Degree Attainment Data. U.S. Census Bureau, 2003 Current Population Survey, Educational Attainment in the United States, June 2004. Slide courtesy of Education Trust.

• Harlow, C.W. (2003). Education and correctional populations, bureau of justice statistics special report. Washington, DC: US Department of Justice.

• Western, B., Schiraldi, V., & Zienberg, J. (2004). Education and incarceration. Washington, DC: Justice Policy Institute, p. 1; Early Warning! Why Reading by the End of Third Grade Matters (Annie E. Casey Foundation).

References

…Putting the pieces together for our children’s future.Thank you .