1

Public School Reform

Developing a Student Achievement Business Model

Askew Institute

February 6, 2004

• Meta Analysis• Cost Benefit• Academic Business Plan• Pay for Performance• Data Warehouse• AAA Plan• Online Assessment• Return on Investment

Developing a Student Achievement Business Model

School District of Palm Beach County

Meta-analysis presentation

Fall 2003

Factors that Accelerate Achievement

Student Home environment

Learned intelligence / background knowledge Motivation

Teacher Instructional strategies

Classroom management Curriculum design

Class Class size

Paraprofessionals Peer performance

School Guaranteed and viable curriculum

Challenging goals and effective feedback Parental and community involvement

Safe and orderly environment Collegiality and professionalism

Marzano, R. J., 2003

School Factors Guaranteed and Viable Curriculum

Opportunity to Learn

Has a rigorous, well-articulated curriculum

Addresses the content in assessments used to make judgments about student achievement

Monitors the extent to which teachers actually cover the articulated curriculum

Time

Allocates instructional time

Engages students during instructional time

Ensures students are successful at the tasks in which they are engaged

Marzano, R., 2003

2

4

8

10

11

12

15

31

0 5 10 15 20 25 30 35

Cooperation

Leadership

School Climate

Parental Involvement

Pressure to Achieve

Monitoring

Time

Opportunity to Learn

* The average gain in percentile points of the average student in the experimental group compared to the average student in the control group.

Marzano, R., 2000; Borman, G.D.; Hewes, G.M. et al., 2000

School Factors

Guaranteed and Viable Curriculum: Opportunity to Learn

Percentile Gain*

Opportunity to Learn affects student achievement more than double any other school factors.

School Factors

Leadership

Research on principal leadership responsibilities has revealed the following attributes to be among the most important for a site-based

leader to possess.

10

10

10

10

11

11

11

12

12

12.5

13

0 2 4 6 8 10 12 14

Ideals/Beliefs

Affirmation\

Resources

Order

Monitoring / Evaluation

Outreach

Culture

Change agent

Input

Intellectual stimulation

Situational awareness

Waters, T., Marzano, R., and McNulty, B., 2003

Percentile Gain

Definitions of leadership attributes located on notes page.

Lea

der

ship

att

rib

ute

s

In Dallas, teacher effectiveness accounted for a 34-50 percentile difference after three years for students who started at similar achievement levels.

27

42

76 76

0

15

30

45

60

75

90

Pe

rce

ntil

e

Jordan, Mendro & Weerasinghe, 1997

After 3 years of very ineffective teachers After 3 years of very effective teachers

60

60 = Beginning Percentile Score

Ending Percentile Score

Math

Math

Reading

Reading

Teacher Effectiveness

Teacher Assignment EffectivenessIn Tennessee, African-American students are more likely than White students to have less effective teachers.

14%

27%22%

16%

0%

10%

20%

30%

40%

50%

Least Effective Teachers Most Effective Teachers

Per

cent

with

Inef

fect

ive

Tea

cher

s

African American Students White Students

Sanders and Rivers, 1996

Teacher Assignment ExperienceNationally, poor and minority students are more often taught by teachers with less experience.*

20%

11%

21%

10%

0%

10%

20%

30%

40%

50%

% te

ache

rs w

ith >

3 ye

ars

expe

rienc

e

High-poverty schools Low-poverty schools

High-minority schools Low-minority schools

*Teachers with three or fewer years of experience. “High” and “low” refer to top and bottom quartiles.National Center for Education Statistics, “Monitoring Quality: An Indicators Report,” December 2000.

Teacher Assignment ExperienceFY2002 School District of Palm Beach CountyHigh-poverty, high-minority elementary schools generally have less experienced teachers.

FY2002 % Receiving Free/Reduced Lunch

100806040200

FY

20

02

Te

ach

er A

ve

ra

ge

Ye

ars E

xp

erie

nce

20

18

16

14

12

10

8

6

4

r = -.28

Florida School Indicators Report and School Accountability Report data analyzed by DREA

25%

14%19% 16%

40%

20%

31%

18%

0

20

40

60

80

100

Math Science English Social SciencePe

rce

nt o

f un

de

r-q

ua

lifie

d te

ach

ers

Less than 20% free/reduced lunch Greater than 49% free/reduced lunch

*Teachers who have less than a minor in the field. National Commission on Teaching and America’s Future, 1996, as customized by The Education Trust, Inc.

Teacher Assignment Subject Matter KnowledgeNationally, classes in high-poverty, high-minority high schools are more often taught by under-qualified* teachers.

Teacher Assignment Advanced DegreesFY2002 School District of Palm Beach CountyHigh-poverty, high-minority high schools tend to have asmaller percent of teachers with advanced degrees.

Florida School Indicators Report and School Accountability Report data analyzed by DREA

FY2002 % Receiving Free/Reduced Lunch

100806040200

FY

20

02

% T

ea

ch

er

Ad

v.

De

gre

es

70

60

50

40

30

20

10

r = -.44

Teacher Vacancies

Districts often find it difficult to hire teachers for their poorest schools.

Applications for teacher vacancies in Maryland's poorest and wealthiest districts.

1996-1997

826 665

1800

6109

01000

20003000

40005000

60007000

Num

ber

Vacancies Applications

Prince, 2002: “The Challenge of Attracting Good Teacher and Principals to Struggling Schools.”

Baltimore

(poorest district in state)

Montgomery County

(wealthiest district in state)

Teacher VacanciesFY2003 School District of Palm Beach County

r= .40

Teacher TurnoverFY2002 School District of Palm Beach CountyHigh-poverty, high-minority elementary schools experienced higher teacher turnover than more affluent schools

FY2002 % Receiving Free/Reduced Lunch

100806040200

% T

each

er T

urno

ver

30

20

10

0

r = .48

FY2002 FSIR and SACR, as analyzed by DREA.

Parent and Community InvolvementSchool District of Palm Beach County 2002High-minority, high-poverty schools tend to have fewer volunteer hours contributed

Cost Effectiveness Analyses

FY2001 Elementary Schools – Highest Total Budget

FY2001 Elementary Schools – Lowest Total Budget

Middle School of the Arts

FY2001 Middle Schools – Highest Total Budget

FY2001 Middle Schools – Lowest Total Budget

FY2001 High Schools – Highest Total Budget

Dreyfoos School of the Arts

FY2001 High Schools – Lowest Total Budget

FY2001 Elementary Schools’ Budgets & Federal Lunch

r = .76

FY2001 Middle Schools’ Budgets & Federal Lunch

r = .82

FY2001 High Schools’ Budgets & Federal Lunch

r = .69

FY2001 ELEMENTARY SCHOOLS’ BUDGETS & READING GAINS

FY2001 Relationship Between Cost per Student and FCAT SSS Reading Gain

Elementary Schools

r= .14

FY2001 ELEMENTARY SCHOOLS’ BUDGETS & MATH GAINS

FY2001 Relationship Between Cost per Student andFCAT SSS Math GainElementary Schools

r = 0.0

FY2001 MIDDLE SCHOOLS’ BUDGETS & READING GAINS

FY2001 Relationship Between Cost per Student and FCAT SSS Reading Gain

Middle Schools

R = .22

FY2001 MIDDLE SCHOOLS’ BUDGETS & MATH GAINS

FY2001 Relationship Between Cost per Student and FCAT SSS Math Gain

Middle Schools

r = .14

FY2001 HIGH SCHOOLS’ BUDGETS &

READING GAINS

FY2001 Relationship Between Cost per Student and FCAT SSS Reading Gain

High Schools

r = .37

FY2001 HIGH SCHOOLS’ BUDGETS &

MATH GAINS

FY2001 Relationship Between Cost per Student and FCAT SSS Math Gain

High Schools

r = .10

FY2003 Relationship Between Cost per Student and Percent Receiving Free/Reduced Lunch Percent

Elementary Schools

r = .83

FY2003 Relationship Between Cost per Student and Percent Receiving Free/Reduced Lunch Percent

Middle Schools

r = .76

FY2003 Relationship Between Cost per Student and Percent Receiving Free/Reduced Lunch Percent

High Schools

r = .56

District Average

SDPBC Budget Department, as analyzed by DREA

r = .21

FY2003 Relationship Between Cost per Student and FCAT Reading GainElementary Schools

District Average

r = .39

FY2003 Relationship Between Cost per Student and FCAT Math Gain

Elementary Schools

District Average

SDPBC Budget Department, as analyzed by DREA

r = .21

FY2003 Relationship Between Cost per Student and FCAT Reading Gain

Middle Schools

SDPBC Budget Department, as analyzed by DREA

District Average

r = .44

FY2003 Relationship Between Cost per Student and FCAT Math GainMiddle Schools

District Average

SDPBC Budget Department, as analyzed by DREA

r = .03

FY2003 Relationship Between Cost per Student and FCAT Reading Gain

High Schools

District Average

SDPBC Budget Department, as analyzed by DREA

r = .10

FY2003 Relationship Between Cost per Student and FCAT Math Gain

High Schools

www.palmbeach.k12.fl.us/

Academic Business Plan

0

20

40

60

80

100P

erc

en

t

All 39.5 43.4 39.3 45.3 47.1 49.5 54.1 58.7 63.3 67.9 72.5 77 81.6 86.2 90.8 95.4 100

Black 14.9 16.5 15.5 20.8 23.4 26.5 33.2 39.9 46.5 53.2 59.9 66.6 73.3 80 86.6 93.3 100

Hispanic 28 33.1 28.3 34.4 37 39.7 45.2 50.7 56.1 61.6 67.1 72.6 78.1 83.6 89 94.5 100

White 53.1 58.4 54.6 61.8 63.5 66.2 69.3 72.3 75.4 78.5 81.6 84.6 87.7 90.8 93.9 96.9 100

FRL NR NR NR 27.3 30 33 39.1 45.2 51.3 57.4 63.5 69.5 75.6 81.7 87.8 93.9 100

ESE NR NR NR 11.4 12.6 14.3 22.1 29.9 37.7 45.5 53.3 61 68.8 76.6 84.4 92.2 100

LEP NR NR NR 12.4 14.1 17 24.5 32.1 39.6 47.2 54.7 62.3 69.8 77.4 84.9 92.5 100

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

As additional data becomes available, it will be added and projections will be adjusted to reach the FY2014 goal of 100%.

Only grades 4, 8, and 10 tested in FY98-00.

Base- Annual Progress Benchmarks Goal line

No Child Left Behind Goals

Key Result 3Percent of Students Scoring Level 3 and Higher on FCAT Reading

Data Warehouse

Building an Information Foundation in a Large District

School District of Palm Beach County

• Dr. Art Johnson, Superintendent• Katheryn Gemberling, Consultant

Excerpted from presentation made at the Council of Greater City Schools

October 24, 2003

Algebra Initiative: All Grade 9 Student Successfully Complete Algebra

• Class size of 20• Certified Math Teachers• Support for High Risk Students

– Students in Stanines 1-3– Second period of math: Intensive Math

• Other Considerations– All Algebra 1 students take the same first semester

exam and the same end-of-year exam.– All teachers use district grading scale for exams.

EDW Upfront Web Screen

Students taking Algebra at High Poverty SchoolClass Size

Students taking Algebra at Low Poverty SchoolClass Size

High Risk High School Students’ Performance on Algebra I ExamTaking Intensive Course

High Risk High School Students’ Performance on Algebra I ExamNOT Taking Intensive Course

Grade 9 Students taking Algebra I Semester Exam Grade at Each High School

Grade 9 Students Algebra I Who Entered Proficient (Level 3 – FCAT Math)Semester Exam Grade at Each High School

Factors that Accelerate Achievement

Student Home environment

Learned intelligence / background knowledge Motivation

Teacher Instructional strategies

Classroom management Curriculum design

Class Class size

Paraprofessionals Peer performance

School Guaranteed and viable curriculum

Challenging goals and effective feedback Parental and community involvement

Safe and orderly environment Collegiality and professionalism

Marzano, R. J., 2003

Grade 9 Students taking Algebra I Who Entered Proficient in MathSemester Exam Grade at One High School for Each Math Teacher

School District of Palm Beach County Distribution of Gain for Elementary FCAT SSS Reading -- Teacher/Class Combinations*FY2001 to FY2002

42%

9%

1%1%

9%

38%

%

10%

20%

30%

40%

50%

Class Average Gain in LNCE*

Per

cent

of T

each

er/C

lass

es

-17 to -10 -10 to -5 -5 to 0 0 to 5 5 to 10 10 to 19

Increasingly Negative Gain Increasingly Positive Gain

DREA

* An LNCE (Local Normal Curve Equivalent) gain of +15 would move a class from the 25 th to the 50th percentile.

Source ofhigh

performing teachers

Lagging and Leading Lagging and Leading IndicatorsIndicators

Lagging Leading

End of Year

Delayed Feedback

General Direction

During the Year

Immediate Feedback

Specific Direction

Can change end of year outcome

Can’t change end of year outcome

www.palmbeach.k12.fl.us/

Accelerated Academic Achievement Plan

FY2004 Fall & Winter Diagnostic TestMath Predicted Levels

Compared to SelfFY2003 & FY2004 Fall & Winter Diagnostic Test &

FY2003 FCAT Math LevelsAll Grades

Looking Good

Compared to StandardFY2003 & FY2004 Fall & Winter Diagnostic Test &

FY2003 FCAT Math LevelsAll Grades

70A

NCLB Group PerformanceFY2004 Fall and Winter Diagnostic Test

NCLB Counts (N= 30)

30

NCLB Group PerformanceFY2004 Fall and Winter Diagnostic Test

Percents

38Lvl 3+

On line Assessment

Public Education: The best investment we make