Slide 1

Know Your Accountability 2013Marianne Mottley, Assistant Director, Office of Accountability June 28, 20131Todays Topics New Report Card Calculations

Graduation Rate

Achievement

Gap Closing

Progress

K-3 Literacy

Prepared for Success

New Report Card Calculations

Graduation Rate

Achievement

Gap Closing

Progress

K-3 Literacy

Prepared for Success

2Todays Topics New Report Card Screen Shots

New Secure Data Center (SDC) Reports

Five Ranking Reports

Dropout Recovery Report Card

Closure Requirements

New Report Card Screen Shots

New Secure Data Center (SDC) Reports

Five Ranking Reports

Dropout Recovery Report Card

Closure Requirements3

Ohios New Report Card

A better tool for parentsOhios new accountability system, enacted through House Bill 555 and drafted in our states ESEA waiver application, is designed to be more user friendly to parents and the public. Districts and schools no longer will be rated with designations such as Excellent or Effective.

4Measures Phased InInstead, they will receive letter grades for measures that will be phased in over time.

In 2013 districts and schools will have up to 9 measures depending on the grades they serve.

In 2014 there will be up to 16 measured; in 2015 up to 17 measures and in 2016 and beyond up to 18 measures.

Not every school will have all measures as some are specific to certain grades or grade configurations.

But --- EVERY school SHOULD have at least one measure.5

Report card measures grouped into 6 components in 2015.

The 6 components make up the overall grade in 2015.New Report Card Based on Letter GradesBeginning in 2015, the measures will be grouped into six components and the six components will be aggregated to one overall letter grade. House Bill 555 specifies which measures will be grouped into each of the six components, but it will be up to the State Board of Education to adopt rules that specify how the six component grades will be determined and also how the overall letter grade will be calculated.

6Prepared for SuccessK-3 LiteracyGraduation RateGap ClosingProgressAchievement Overall Grade(2015)Overall Grade & ComponentsThe six components that will eventually receive letter grades includeGraduation, Achievement Gap Closing Progress K-3 Literacy Prepared for Success

Lets look at each component in a little more detail

7Report Card ComponentsThe graduation rate component will have two measures that will receive grades beginning with the 2013 report card:

4-year graduation rate --- consisting of data for the graduating Class of 2012

5-year graduation rate consisting of data for the graduating Class of 2011

Eventually the two individual measure grades will be tied together into one component letter grade for graduation --- but as I said we dont know how that will work yet.8Graduation RateSame longitudinal cohort calculation used on 2012 report cards

Cohort: Students accountable to the school and/or district who were first-time 9th graders four years earlier

Accountable: Last school and/or district of accountabilityNumerator: Number of students in the adjusted cohort who graduated in four years or lessDenominator: Total number of students in the adjusted cohort for that class

As we look at the calculation itself, some things have not changed from last years report card.

For example --- the calculation itself is the same longitudinal cohort rate used on 2012 report cards

To refresh your memory, our business rules define:

Cohort: Students accountable to the school and/or district who were first-time 9th graders four years earlier

Accountable: Last school and/or district of accountability. For the 2013 report card, we will use the 2012 cohort for the 4-year rate and 2011 cohort for the 5-year rate.

Numerator: Number of students in the adjusted cohort who graduated in four years or less

Denominator: Total number of students in the adjusted cohort for that class

9Graduation Rate CalculationAdjustments to the cohort made for students who join or leave the school at grade level during the four-year period

Adjustments may not be made to reassign a student to a new cohort

Adjustments to the original cohort may be made for students who join or leave the school during that four-year period

What this means is that as students transfer in and out we will increase or decrease the denominator of the calculation.

HOWEVER!!! Adjustments may not be made to reassign a student to a new cohort

Note that when calculating the four year rate for the Class of 2012, the calculation period ends at the end of the summer 2012. What this means is that districts and schools are held accountable based on their data reported are of the summer 2012. Students who transfer sometime after the 12-13 school year begins are NOT included in the 2012 data.

10Graduation RateData lagged one year (four-year rate) or two years (five-year rate) to allow summer graduates to be counted

Final percentages for Classes of 2011 and 2012 available nowAlso as in the past, we will continue to lag the reporting of the graduation data by one year for the four year rate and two years for the five year rate. This allows schools to include students who graduate over the summer in both calculations.

Schools with a graduation rate were able to see their final rates for the Classes of 2011 and 2012 earlier this spring.

11Four-Year Graduation Calculation#FY12 Cohort Graduates (includes summer 2012 grads)# of FY2009 1st Time 9th Graders (grads & non-grads)

plus

# of FY09 1st Time 9th Grade Transfers In (grads & non-grads)

plus

# of Dropouts reported any year from FY2009-FY2012

minus

# of FY2009 First-Time Ninth Grader Transfers Out*

*NOTE: Transfers to Other Ohio Public Schools Must Be Picked Up By Another District or they are Reinstated Transfers that are returned to the withdrawing district/school This formula represents the calculation for the four year graduation rate.

The numerator includes those students who were first reported as 9th graders during the 2008-2009 school year who graduated sometime prior to the start of the 2012-13 school year. I say sometime before the start of the 12-13 school year because there are three different type of graduates that can be in a schools numerator. First --- if a student graduates early, maybe at the end of the third year of high school, we hold that students graduation record and report it when the rest of his cohort graduates the following year. The second type is what we think of as the typical student who graduates in may or June at the end of the fourth year of high school.

Finally, some schools will have summer graduates included. The key is that to be counted as being a four year on time graduate, a student who started the 9th grade during the 2008-09 school year MUST earn a diploma sometime prior to the start of the 2012-13 school year.

Now lets look at the denominator.

Obviously, the graduates are included in that number, but we also have any other students who started the ninth grade with your school in FY09 who didnt graduate either because they didnt have enough credits or because they failed the OGT.

We also add any students who were reported by some other public school as being in the 9th grade in FY09 who transferred to your school and are either graduates or non-graduates.

Also included are students who dropped out from your school who never enrolled in any other public school. Note that we do a check of dropouts and if a student was reported as being a dropout by your school, but they later enroll someplace else using the same SSID, we update your cohort to remove the student from your denominiator and place him in the receiving schools calculation.

Finally --- we remove students reported as transfers during that four year period.

Note that if a student is reported with a Code 41, which is the code to denote a transfer to another Ohio school, we do a check to make sure that the SSID really does show up elsewhere. If we dont fine that student as being enrolled anywhere, reinstate the student in your schools denominator.

124 Year Graduation Rate** Specified in legislation

Grad Rate Four Year Letter GradeGrad Rate Four Year PercentageA93% - 100% **B89% - 92.9%C84% - 88.9%D79% - 83.9%F< 79%Will be calculated for the Class of 2012 reported on the 2013 report card.

The table on the screen shows the percentages that are needed to obtain the various letter grades for the four year rate. As you can see, the legislature set the rate needed to get an A while the state board set the other ranges.

Since Ive had a lot of e-mails and calls from school about their cohort --- I want to again explain that when we were determining each schools four year graduation rate, we looked at the status of the student at the end of the summer of 2012.

What this means is that if a student ended the 2011-12 school year in your building, and then on the first day of school for the 12-13 school year he enrolled somewhere else, he will not be updated into the new schools cohort until we calculate the five-year rate for the Class of 2012 on the 2014 report card.

134 Year Graduation RateWe ran some simulations on the graduation rate letter grades and here are some pie charts that show how the schools and districts fared.

144 Year Graduation RateHere is one concentrating on urban students and community schools.

15Five-Year Graduation Calculation2011 graduating class updated to reflect mobility during the 2011-12 school year

Same business rules apply to the five-year rate that apply to four-year rate

Summer graduates from the fifth year are included in the numerator

We also will be reporting letter grades for the five year rate in 2013.16Five-Year Graduation Calculation#FY11 Cohort Graduates (includes summer 2012 grads)# of FY2008 1st Time 9th Graders (grads & non-grads)

plus

# of FY08 1st Time 9th Grade Transfers In (grads & non-grads)

plus

# of Dropouts reported any year from FY2008-FY2012

minus

# of FY2008 First-Time Ninth Grader Transfers Out*

*NOTE: Transfers to Other Ohio Public Schools Must Be Picked Up By Another District or they are Reinstated Transfers that are returned to the withdrawing district/school This formula represents the calculation for the four year graduation rate.

The numerator includes those students who were first reported as 9th graders during the 2007-2008 school year who graduated sometime prior to the start of the 2012-13 school year. I say sometime before the start of the 12-13 school year because again are three different type of graduates that can be in a schools numerator. First --- if a student graduates early, that student will be in the five year rate. The second type is what we think of as the typical student who graduated in the four year of high school, either in

Finally, some schools will have summer graduates included. The key is that to be counted as being a four year on time graduate, a student who started the 9th grade during the 2008-09 school year MUST earn a diploma sometime prior to the start of the 2012-13 school year.

Now lets look at the denominator.

Obviously, the graduates are included in that number, but we also have any other students who started the ninth grade with your school in FY09 who didnt graduate either because they didnt have enough credits or because they failed the OGT.

We also add any students who were reported by some other public school as being in the 9th grade in FY09 who transferred to your school and are either graduates or non-graduates.

Also included are students who dropped out from your school who never enrolled in any other public school. Note that we do a check of dropouts and if a student was reported as being a dropout by your school, but they later enroll someplace else using the same SSID, we update your cohort to remove the student from your denominiator and place him in the receiving schools calculation.

Finally --- we remove students reported as transfers during that four year period.

Note that if a student is reported with a Code 41, which is the code to denote a transfer to another Ohio school, we do a check to make sure that the SSID really does show up elsewhere. If we dont fine that student as being enrolled anywhere, reinstate the student in your schools denominator.

175 Year Graduation RateGrad Rate Five Year Letter Grade Grad Rate Five Year PercentageA95% - 100% **B90% - 94.9%C85% - 89.9%D80% - 84.9%F< 80%New Measure** Specified in legislationNew Measure. Data are lagged TWO YEARS.

2013 report card will have graduating Class of 2011 data.

Initial four-year rate for this graduating class was created at the end of summer 2011 (for the 2012 report card).

Each schools cohort was updated during the fifth year of high school in 2011-12 to reflect movement of students during that school year.

Cohort cut off was end of summer 2012.

18Supplementary EMIS FormsThe process of creating and adjusting graduation cohorts is critical.

Problems can arise when an LEA is unable to make necessary adjustments because the appropriate EMIS reporting period has closed.

This supplementary EMIS data submission process is NOT designed to give LEAs the opportunity to correct erroneous data submitted during a previous EMIS reporting period.

19Supplementary EMIS FormsTwo basic requirements:

During the appropriate EMIS reporting period, an LEA reported a withdrawal code and reason for a student that kept the student in the denominator of the graduation rate calculation. AND

After the appropriate EMIS reporting period had closed, the LEA obtained evidence indicating that a different withdrawal code and reason should have been used and that new code would have removed the student from the denominator of the calculation.

20Supplementary EMIS FormsLEA is responsible for retaining all supporting evidence for the supplementary EMIS data submission (e.g., transcript requests, record requests, court documents, medical documents, legal documents), which is subject to audit at any time.

Any approved cohort reassignments will impact only the Four-Year and/or Five-Year Adjusted Cohort Graduation Rate calculation(s). No other calculations or processes using the EMIS Withdrawal Reason Element will be affected by the supplementary data provided on this form.

21Supplementary EMIS FormsLEAs must complete a separate form for EACH student where an adjustment is being requested.

Deadline to submit requests fro the 2011 graduation rate data was: MAY 31, 2013

Any requests for changes are subject to audit.

Early 2014 we should know grad rate for 4 year and 5 year.Deadline for supplementary form will most likely remain the same year to year. 22Report Card ComponentsTwo measures --- Performance Index and Performance Indicators.

No longer tied together in driving overall rating

Each get own letter grade and eventually will be combined into an Achievement letter grade.23State Indicators24 Indicators

Grades 3-8 Reading (6 tests)Grades 3-8 Math (6 tests)Grades 5 & 8 Science (2 tests)10th Grade OGT (5 tests)11th Grade OGT (5 tests)

No Attendance or GraduationUntil new assessments are released in 2015, we will have 24 indicators ---- no Attendance rate and graduation was moved to its own component with two measures.24State IndicatorsSame Where Kids Count Rules (FAY applies)

Same FERPA masking rules (At least 10 accountable students to have an indicator reported)

25State Indicators2013 75% proficiency standard (85% required for 11th grade OGT)

2014 Proficiency standard increases to 80% (11th grade stays at 85%)

2015 New assessments implemented; do not know what new standard will be26State IndicatorsOne extra step in calculation

Once number of indicators met is calculated, turn the number into a percentage of the total indicators possible and assign a letter grade

Denominator is the number of indicators with unmasked (N >=10) data for the school or district it can be less than 2427State Indicators Current 75% Proficient StandardState Indicators Letter Grade State Indicators PercentageA90% - 100% **B80% - 89.9%C70% - 79.9% **D50% - 69.9%F< 50% **** Specified in legislationShows how many students have a proficient level of knowledge. Based on a series of 24 state tests.

28State Indicators Current 75% Proficient Standard29State Indicators 75% Proficient302014 State Indicators 80% ProficientState Indicators Letter Grade State Indicators PercentageA90% - 100% **B80% - 89.9%C70% - 79.9% ** D50% - 69.9%F< 50% **** Specified in legislationIn 2013-14 (this fall) the benchmark for meeting this performance indicator will increase to 80%. 80% of students will have to meet the indicator in order to receive credit for the indicator.

31State Indicators 80% Proficient32State Indicators 80% Proficient33Performance IndexMeasures the achievement of every student, not just whether or not they reach proficient. Schools receive points for every students level of achievement. 34Performance IndexSame Where Kids Count Rules (FAY applies) Same basic calculation of assigning points based on percentages in each performance category

Same weights for each existing category

One new weight35Performance Index New WeightStudents on formal Acceleration plans earn higher weights

Students in Proficient range = Accelerated weight

Students in Accelerated range = Advanced weight

Students in Advanced range = Advanced Plus weight of 1.3

K-8 onlyAcceleration Plan separate from gifted served. Developed by a team and decided how student will be served. Can still have a formal acceleration plan without having a gifted service plan. 8th grade do not take test if in second year of acceleration. Can end acceleration at any time. Retained students MUST retake retained grade level and current year score counts. 36Performance Index CalculationTest RangePercent of Tests in RangeWeightPointsUntested0.1%00Limited4.0%0.31.2Basic22.0%0.613.2Proficient45.0%145Accelerated25.0%1.127.5Advanced3.6%1.24.32Advanced Plus0.3%1.30.39Total PI Score Points91.637Performance Index CalculationPerfect PI Score = 120 Points

Calculate percentage of points earned

91.6/120 = 76.3%38Performance IndexPerformance Index Letter Grade Performance Index PercentageA90% - 100% **B80% - 89.9%C70% - 79.9%D50% - 69.9%F< 50% **** Specified in legislationMeasures the achievement of every student, not just whether or not they reach proficient. Schools receive points for every students level of achievement.

MUST HAVE 108 POINTS OR MORE TO EARN AN A

39Performance IndexPerformance Index Version 1

Public Districts Based on 2012 Data

40Performance IndexPerformance Index Version 1

Public Districts Based on 2012 Data

41Report Card ComponentsGap closing and AMOs are part of our new waiver. We submitted an updated template to the waiver a couple of weeks ago and are waiting to be told to submit a clean copy of the document to the USDOE.42Gap Closing (Annual Measurable Objectives)Measures the academic performance of specific groups of students

Replaces old AYP calculation

More realistic targets, but more aggressive improvement (gap closing) expected to earn top letter grades

43Gap Closing-Similarities to AYP

Same Where Kids Count Rules

Same 10 subgroups evaluated

Same N size of 30 for Proficiency

Same N size of 40 for Participation

Same Attendance Goal (93%)

Same Participation Rate Goal (95%)44Gap Closing Differences from AYPA single AMO for reading and math versus seven different AYP goals

Graduation AMO for 4-Year Rate Only

AMO MeasureBaseline AMO2012-20132013-20142014-20152015-20162016-20172017-20182018-2019Reading81.9%83.4%84.9%86.4%87.9%89.4%90.9%Math76.8%78.5%80.5%82.5%84.5%86.5%88.5%Four-Year Grad. Rate73.6%75.9%78.2%80.5%82.8%85.1%87.4%90.0%Waiver explains how AMOs set ---- goal to cut gap in half by 2018 or 2019 through annual AMO increases.

**ODE will need to resubmit their waiver in 2015 and % may be re-evalutated based on PARCC assessments. 45Gap Closing Differences from AYPAYP calculation yielded a Met or Not Met label.

AMO calculation yields a letter grade of A through F.

Final AMO grade of C or better analogous to meeting AYP

46Gap Closing Differences from AYPNo School or District Improvement StatusNo public school choice or supplemental educational servicesNo parent letters

Replaced by Priority (lowest 5%) and Focus (lowest 10%) schoolsWaiver requires implementation of school turnaround model

47Gap Closing Differences from AYPIf AMO is not met, points are awarded based on amount of improvement (Gap Closing) between prior year and current year.

No Safe Harbor provision

No Growth provision

48Gap Closing Differences from AYPAttendance Rate used as demotion tool

Participation Rate used as demotion tool

Low subgroup performance on reading/math/graduation used as demotion tool

49Gap Closing Points CalculationFour Rules

If the AMO is met assign 100 points

If the amount of improvement exceeds the current year gap assign 100 points

If improvement is made, but it does not exceed the gap, assign points between zero and 100

If the performance declines, and AMO is not met, assign zero points

50Gap Closing Points CalculationRule 1 - If the AMO is met assign 100 points

2013 AMOs

Reading = 83.4%

Math = 78.5%

Graduation 75.9%

51Gap Closing Points CalculationEquals

ImprovementCurrent Year Gap

Current Year AMO Current Year ProficiencyCurrent Year Proficiency Previous Year ProficiencyFor rules 2 through 4 use this calculation to determine which rule applies and number of points awarded.52Gap Closing Points CalculationCurrent Year GapImprovementX100=Points AwardedCalculation used to determine points awarded for Rules 2, 3 and 453Gap Closing Points CalculationRule 2 If the amount of improvement exceeds the current year gap assign 100 points

Calculation

2013 Percentage 2012 Percentage = Improvement

2013 AMO 2013 Percentage = Current Year Gap54Gap Closing Reading Example 12012 Proficiency = 70%2013 Proficiency = 80%2013 Reading AMO = 83.4%

80.0%-70.0% = 10%83.4% - 80% = 3.4%Note Fraction 10/3.4 is greater than 1.0 --- this means the amount of improvement (10%) exceeds the current year Gap (3.4%) and 100 points are assigned per Rule #255Gap Closing Points CalculationRule 3 - If improvement is made, but it does not exceed the gap, assign points between zero and 100

Calculation

2013 Percentage 2012 Percentage = Improvement

2013 AMO 2013 Percentage = Current Year Gap

56Gap Closing Reading Example 22012 Proficiency = 70%2013 Proficiency = 75%2013 Reading AMO = 83.4%

75.0%-70.0% = 5%83.4% - 75% = 8.4%Note Fraction 5/8.4 is less than 1.0 --- this means the amount of improvement (5%) was less than the current year Gap (8.4%) and points are assigned per Rule #3X100=59.5 Points Awarded57Gap Closing Points CalculationRule 4 - If the performance declines, and AMO is not met, assign zero points

Calculation

Compare 2013 percentage to 2012 percentage

If 2013 percentage is smaller; and,

If 2013 AMO is not met

Award ZERO points

58Gap Closing Reading Example 32012 Proficiency = 75%2013 Proficiency = 70%2013 Reading AMO = 83.4%

70.0%-75.0% = -5%83.4% - 70% = 13.4%Note Fraction -5/13.4 is a negative number -- this means there was no improvement and instead the gap grew so no points are assigned per Rule #4X100=0 Points Awarded59Gap ClosingTwo Final Rules

Points are capped at 100

Points are floored at ZERO

60Gap ClosingStudent SubgroupsSubgroup ProficiencySubgroup Points EarnedSub-Component Points2013 Reading AMO = 83.4%All Students92.40%100 Points520/600 = 86.67Amer. Indian/Alaskan NativeNC