School Report Cards A-F

Steven Crawford – CCOSA

Ryan Owens - USSA

Grading Schools A-F

• In May of 2011 the Oklahoma Legislature passed H.B. 1456 to grade school performance on an A through F scale.

• The law is passed to easily communicate complex school performance data to parents and patrons.

Initial Challenges

• In January of 2013 researchers with the education policy centers at OU and OSU release a report urging policy makers to scrap the system and start again.

“Accountability systems are only useful if their measures are credible and clear. Despite good intentions, the features of the Oklahoma A-F grading system produce school letter grades that are neither clear, nor comparable; their lack of clarity makes unjustified decisions about schools. Further, A-F grades are not productive for school improvement because they do not explain the how or why of low performance.”- Excerpt from Jan. 2013 report

H.B. 1658

• In response to criticism and low levels of buy-in, a “tweak” is passed in May of 2013.

• H.B. 1658 is passed without input from educators!

Researchers Respond to H.B. 1658

“We also examined recent legislative changes to the A-F reporting system and found that the fundamental problems contributing to inaccurate results and invalid interpretations have been compounded, not resolved. Despite initial missteps in the design of an effective accountability system, our objective, which we share with State leaders, is to get accountability right so all students have access to learning opportunities that prepare them for academic and personal success.”

Current A-F Challenges

Corrupted Data Inputs in 2013

• Spring 2013 Testing Debacle• Cut Score Manipulation• Missing Testing Booklets• Writing Assessments Re-graded

Multiple Grade Calculation AdjustmentsLead to Preliminary Grades

Changing 5 to 7 Times

Renewed Criticism

• In October of 2013 researchers with the education policy centers of OU and OSU released a second report concerning Oklahoma’s A-F report card system.

• Researchers reviewed over 15,000 student test scores in 63 schools to determine the effectiveness of Oklahoma’s school accountability system.

Areas of Concern

• Small Differences Between Letter Grades– “Three to six correct responses separated A

schools from F”

– “Many of the achievement differences between letter grades were likely due to chance; even when they reached statistical significance they were of questionable practical utility”

Areas of Concern

• Classification Error– “Our analysis showed that math performance in

some “D” and “F” schools was higher than that in some “B” and “C” schools”

– “To be meaningful, the letter grade would have to represent a school’s performance pattern, but it turns out that within-school variation across subject areas fluctuates a great deal. Thus, it is never clear what an A is or what an “F” is”

Areas of Concern

• Achievement Gaps– “Consistently across the three subject areas

(reading, math, and science), minority and poor children tested highest in “D” and “F” schools and lowest in “A” and “B” schools”

– “A” and “B” schools are the least effective for poor and minority children

CCOSA and USSA Concerns

The A-F Committee

• In 2013 CCOSA, USSA and OSSBA formed an A-F Committee composed of educators to review the new school accountability system, the concerns of researchers, and any proposed legislative solutions in an effort to advise the associations on the proper position(s) to support.

• The A-F Committee’s membership consists of superintendents as well as assessment and accountability experts from districts across the state.

Accountability Systems Must Be:

• Valid• Reliable• Useful• Timely• Nationally Normed / Comparable

School Grades Capture Little Data

At any high school, A-F grade is calculated on 7 tests (school performance) and student testing gains (growth) on Algebra I and English 2 (2 components: all student gains and bottom 25% of student performance gain).

• 50% of the high school's grade is based on how students perform in 7 of 100 or more subject offerings. At Clinton Public Schools half of the high school grade is based on how students do in only 6% of classes!

• The other 50% of the grade is based on

student testing gains (25% all student growth and 25% bottom quartile growth) in 2 subjects.

With only 6% of the total classes making up 100% of the grade and 8% of the students making up 33% of the grade, A-F in it's current format is a terrible way to measure the quality of the teachers or the overall accountability of a school.

Position of the A-F Committee

• We will not seek a delay because the system is broken!

• We seek a nationally comparable accountability system that reflects student and teacher effectiveness and is not subject to manipulation by politicians.

• We will focus our attention on data that is relevant to students (ACT, graduation rates, drop out rates, etc.)

THANK YOU