2013 Accountability Overview

2013 Accountability Overview

WebinarJuly 9, 2013

What We Will Cover

Important Dates Slides 4 & 5 Accountability Changes for 2012 to 2013 Slides 7 & 18 District Accountability Overview Slides 8-22 District Accountability Data Release, 7/12 Slides 24-27 Appeals Slide 30 Questions

04/19/2023 2

IMPORTANT DATESAccountability 2013

04/19/2023 3

Important Dates-Districts

04/19/2023 4

Date Milestone Delivery Medium

June 29, 2013 Value-Added Data Released TVAAS Restricted Website

July 12, 2013 Accountability Data Released to Districts

Accountability Application

July 12, 2013 District Appeals Window BeginsEmail Accountability Appeals to:

[email protected]

July 19, 2013 District Appeals Window Ends

July 24, 2013Embargoed District Determinations Email to District Directors

August 13, 2013 Reward School Lists Email to Directors w/ Reward Schools

Important Dates-Public

04/19/2023 5

Date Milestone

Week of June 24, 2013 Public Release of State Level Data

Week of July 22, 2013 Public Release of District Achievement Scores

Week of July 29, 2013 Public Release of District Determinations

Week of August 12, 2013 Public Release of Reward Schools, School Achievement Scores, and TVAAS Public Site

July 24, 2013Embargoed District Determinations

August 13, 2013 Reward School Lists

ACCOUNTABILITY CHANGESAccountability 2013

04/19/2023 6

Accountability Model: What’s New?Change Description

Achievement Status: Achieve: Not Exemplary

Districts that meet a majority of Achievement AMOs, but do not improve in half of the measures cannot be given Exemplary Status

Gap Closure Safe Harbor

Districts obtain safe harbor for gap closure AMO if subgroup meets AMO without widening gap.

Graduation Rate Safe Harbor

Districts that hit 95 percent graduation rate automatically hit their AMO for graduation rate

Improvement Test for Algebra I/II & English

II/III

Algebra I/II and English II/III are treated as one measure – for subgroup improvement, districts must improve in both subjects

Gap Closure Measures Combine

Algebra I + Algebra IIEnglish II + English III

New measures in gap closure that combine Algebra I and Algebra II, English II and English III by establishing a weighted gap to set the AMO.

Grade 3 Value Added Districts using SAT-10 will have a Grade 3 Value Added composite; therefore, will have a TVAAS Safe Harbor for Grade 3 Math and Reading

04/19/2023 7

DISTRICT ACCOUNTABILITY REVIEWAccountability 2013

04/19/2023 8

Accountability Model- Two Parts

AchievementHow are All Students doing on tested

subjects or measures?

Gap ClosureAre

achievement gaps being closed in

tested subjects and measures?

04/19/2023 9

Districts Set the AMOs for SY 2012-13

04/19/2023

Achievement Gap Closure SubgroupFor the following subject/grade levels, targets will be set for all students:

For the following subject/grade levels: For the following subject/grade levels:

3-8 Reading3-8 Math3 Reading3 Math7 Reading7 Math9-12 Algebra I 9-12 Algebra II9-12 English II9-12 English III9-12 Graduation Rate

3-8 Reading3-8 MathAlgebra I + Algebra II*English II + English III*

*Takes the average by weighting each subject by the number of students in each subgroup.

3-8 Reading3-8 MathAlgebra IAlgebra IIEnglish IIEnglish III9-12 Graduation Rate

Gap targets will be set for these subgroup vs. comparison groups gaps:

Achievement targets will be set for these subgroups:

Black/Hispanic/Native American vs. All Students

Economically Disadvantaged vs. Non-ED

Limited English Proficient vs. Non-LEP

Students with Disabilities vs. Non- SWD

Black/Hispanic/Native American Economically DisadvantagedLimited English ProficientStudents with Disabilities

AsianBlackHawaiian Pacific IslanderHispanicNative AmericanWhiteNon-EDNon-LEPNon-SWD

Accountability Model - Achievement

Achievem

entH

ow are A

ll Stude

nts doin

g on tested subjects or m

easures?

Steps1. Determine if district met

participation rate.

2. Determine numbers of AMOs met after all Safe Harbors are applied.

3. Determine for how many measures the district improved.

4. Determine status for achievement side of the model.

04/19/2023 11

Achievement Tests

04/19/2023 12

Test Question

Participation Rate Eligibility Did the district/school have 30 or more students with valid tests in the current year for all students?

Participation Rate Did the district/school have a 95 percent participation rate for the All subgroup?

Achievement AMO Target Eligibility Test Did the district/school have 30 or more students with valid tests at the time the target was set and have 30 students or more in the current school year?

Achievement AMO Target Test Did the district/school meet the AMO target without any safe harbors?

Improvement Test (Total) Did the district/school improve in half or more of the eligible measures?

Improvement Test (Aggregate) Did the district/school improve in 3-8 Math, 3-8 RLA, and half or more of the high school measures- Algebra I, Algebra II, English II, English III, and Graduation Rate?

Achievement Safe HarborsSafe Harbor Question

Confidence Interval Safe Harbor

Did the district/school make the AMO when the upper bound of the 95% confidence interval is applied?

TVAAS Safe HarborDid the district/school 1) receive an index value of 1 or higher (represented by dark green

on TVAAS) for the current year if it is an elementary or middle school measure or 2) receive an index of 2 or higher (represented by dark green in TVAAS) in the current year if

it is a high school measure?

Reduction in Percent Below Proficient Safe

Harbor

Did the district/school reduce the number of students scoring basic/below basic by 10 percent over 1 year, or 19 percent over 2 years, or by 27 percent over 3 years?

Graduation Rate Safe Harbor

Did the district/school have a graduation rate that is greater than or equal to 95%?

04/19/2023 13

04/19/2023 14

Did the District Pass Participation Rate Test?

Did the District Pass the AMO Target Test?

Did the District Pass the Improve Test?

Achievement Status

PassParticipation Rate Test

PassAchievement AMO

Target Test

Pass Improvement Test ACHIEVE

Fail Improvement Test

Achieve: Not Exemplary

Fail Achievement AMO Target

Test (Including Safe Harbors)

Pass Improvement Test

MISS: Intermediate Possible


MISS: In Need of

Improvement

Fail Participation Rate Test

PassAchievement AMO

Target Test


MISS: In Need of

Improvement


MISS: In Need of

Improvement

Fail Achievement AMO Target Test (Including Safe Harbors)


MISS: In Need of

Improvement


MISS: In Need of

Improvement

Accountability Model- Gap Closure

Ga

p C

losu

reA

re

ach

ieve

me

nt

ga

ps

be

ing

cl

ose

d in

te

ste

d

sub

ject

s a

nd

m

ea

sure

s?

04/19/2023 15

Steps1. Determine if district met

participation rate.

2. Determine number of AMOs met after Safe Harbor is applied.

3. Determine for how many measures the district improved.

4. Determine status for gap closure side of the model.

Gap Closure & Subgroup Tests

04/19/2023 16

Test Question

Gap Closure Eligibility Test Did the district have at least 30 students in both the subgroup and the comparison group for the current

year and the prior year?

AMO Gap Test Did the district/school meet the Gap Closure AMO?

Subgroup Eligibility Did the district/school have 30 valid test scores at the subject/subgroup level in the current year?

1 Year, 2 Year, & 3 Year Participation Rate

Did the district/school meet the participation rate for the subject/subgroup for current year, two year, or

three years?

Subgroup Improvement Test Did the district/school improve the percent of students scoring Proficient/Advanced in the

subject/subgroup when compared to the prior year?


04/19/2023 17

To meet the gap closure safe harbor the district MUST:

-Have at least 30 valid tests for the aggregate measure: 3-8 Math, 3-8 Reading, Algebra I/II, English II/III. -The subgroup must hit its subgroup AMO for the measure.*-The gap between the subgroup and comparison group MAY NOT widen.

Note: Subgroup AMOs were set in Algebra I and II separately and English II and III separately. To get the gap closure safe harbor, district must hit subgroup AMO in both subjects.

District X Data 3-8 Math

2012 2013 AMO

ED 30.2% P/A 34.7% P/A 34.6%

Non-ED 45.4% P/A 49.4% P/A 48.8%

Gap 15.2% 14.7% 14.3%

District: -Hit ED AMO-Missed Gap AMO-Gap Didn’t Widen

Algebra I/II & English II/III Changes

04/19/2023 18

Achievement Gap Closure

Measures/Subjects Individual Measures Combined Measures

AMOs Used individually with standard methodology

Combined Weighted Gap

Improvement Test

Doesn’t apply to Achievement

• Eligibility depends on 30 valid tests in both subjects

• Both subjects must improve to count as improvement


• Eligibility depends on 30 valid tests combined

• Must hit subgroup AMO in both subjects

• Gaps must not widen

04/19/2023 19

Did the District Pass the Subgroup

Participation Test?

Did the District Pass the GAP AMO Target Test?

Did the District Pass the Subgroup Improve Test?

GAP Closure Status

Pass Subgroup Participation Test

Pass Gap AMO Target

Test

Pass Subgroup Improvement

TestACHIEVE:

Gap Closure

Fail Subgroup Improvement

Test

MISS: In Need of Subgroup

Improvement

Fail Gap AMO Target Test


TestMISS:

Intermediate Possible


Test

MISS:

In Need of Subgroup Improvement

Fail Subgroup Participation

Test

Pass Gap AMO Target

Test


Test

MISS:



Test

MISS:


Fail Gap AMO Target Test


Test

MISS:



Test

MISS:


Accountability Model - Making Final Determinations

AchievementHow are All

Students doing on tested subjects or

measures?

Gap ClosureAre achievement gaps being closed in tested subjects and measures?

04/19/2023 20

Steps1. Determine Achievement Status

2. Determine Gap Closure Status

3. Use Achievement Status and Gap Closure Status to determine Final Determination

04/19/2023 21

District ACHIEVEMENT Measures – How are determinations made?*Updated Summer 2013Achievement Measures:

For 2011-12, ALL Students are measured in the following areas (up to 9 per LEA):

- 3rd grade Math- 3rd grade RLA- 7th grade Math- 7th grade RLA- 3-8 grades Math- 3-8 grades RLA- HS: Algebra I- HS Algebra II- HS: English II- HS: English III- HS: Graduation Rate

AMOs are set to measure the required percent of annual growth in the % of students scoring proficient and advanced for All students.

AMOs are set so the number of students scoring below basic or basic is reduced in half over eight years.

Participation Rate Test: Is there a 95% Participation Rate in TCAP and EOC Assessments for ALL students? No

MISS Achievement Measures

IN NEED OF IMPROVEMENT

Achievement AMO Test: Did the district meet the majority of eligible achievement AMOs after all safe harbors have been applied?

Yes

Achievement Improvement Test: Did the district improve in half of the eligible subjects?

Achievement Aggregate Improvement Test: Did the district improve in all aggregate measures (3-8 Math, 3-8 RLA, and at least half of the high school measures)?

No

Yes

ACHIEVE: NOT EXEMPLARY

Achievement Measures



Yes

No

Yes

No


INTERMEDIATE POSSIBLE

ACHIEVE Achievement Measures

04/19/2023 22

District GAP CLOSURE Measures – How are determinations made?*Updated Summer 2013Gap Closure Measures

Subgroups are measured in the following measures: -3-8 Math-3-8 Reading/Language Arts-Algebra I + Algebra II -English II + English III

Gap Closure AMOs are set for the following comparison groups:-Black/Hispanic/Native American vs. All Students-Economically Disadvantaged vs. Non-Economically Disadvantaged-Students with Disabilities vs. Non-Students with Disabilities-English Language Learners vs. Non-English Language Learners

Additionally, each of the 9 individual subgroups below must improve in half of the following measures:

-3-8 Math-3-8 Reading/Language Arts-Algebra I/Algebra II-English II/III

Participation Rate Test: Is there a 95% Participation Rate in TCAP and EOC Assessments all individual subgroups with 30 or more students? No



Gap Closure AMO Test: Did the district meet a majority of eligible AMOs outright or with the gap closure safe harbor applied?

Yes

Yes

MISS In Need of Subgroup

Improvement

MISS In Need of Subgroup

Improvement

Yes

No

Yes

No

MISS Intermediate Possible

ACHIEVE Gap Closure Measures

Subgroup Improvement Test: Did the % of students scoring proficient or advanced

increase in half of the measures? N

o

Subgroup Improvement Test: Did the % of students scoring proficient or advanced

increase in half of the measures?

04/19/2023 23

District ACHIEVEMENT and GAP CLOSURE MeasuresWhat is the final outcome?

ACHIEVE

ACHIEVE: Gap Closure

MISS: In Need of Subgroup Improvement


Exemplary


Intermediate

ACHIEVE: Not Exemplary

MISS: In Need of Improvement











Intermediate

Intermediate

Intermediate


In Need of Improvement





Final Achievement Status

Final Gap Closure Status

Final Determinatio

n

DATA RELEASE Accountability 2013

04/19/2023 24

TDOE will release the accountability data files on July 12

04/19/2023 25

Districts Will Receive

Description

2013 Student Level File

Produced by SAS and includes the scores used for all students in accountability determinations; the file also outlines which scores are excluded or changed due to business rules (e.g. MAAS reassignment, multiple scores).

2013 Base Accountability File

Displays school, district, or state level testing data at the subject/grade/subgroup level for all tested grades and subjects for the past three years; these files are produced at the state, system, and school level.

2013 Numeric Accountability File

Displays data with all accountability rules applied and all data points used to make accountability determinations; these files are produced at the state, system, and school level

2013 Achievement Determinations File

Outlines why districts and schools arrive at a given accountability determination for achievement, gap, and subgroup AMO targets

2013 Gap Determination File

2013 Subgroup Determination File

Additional Files for Public Release

26

Districts Will Receive Description

2013 District Achievement Results

System and state level data that shows the following for each tested subject and grade: %P/A 2012, %P/A 2013, Gain from 2012-13

2012 District Gap Results

System and state level data that shows achievement gaps in tested subjects for 2012 and 2013. Also shows change in gaps from 2012 to 2013.

CORE Accountability Data Overview

27CORE Accountability Data Overview

Subject Grade Level Number of Valid Tests

Percent Below Basic

Percent Basic

Percent Proficient

Percent Advanced

Algebra I 9th through 12th 67765 15.5 24.2 30.6 29.7Algebra II 9th through 12th 64370 24 34 28.6 13.4Biology I 9th through 12th 68751 15.5 22.2 45.6 16.7English I 9th through 12th 72950 8.8 23.1 55.1 13English II 9th through 12th 70340 10.5 30 49.2 10.3English III 9th through 12th 58969 22.6 37.8 26.4 13.2US History 9th through 12th 57903 0 3.9 47.2 48.9Math 3rd through 8th 445329 15.6 33.7 30.3 20.4

2013 Statewide TCAP and End-Of-Course Achievement Results

28CORE Accountability Data Overview

2013 Statewide TCAP and End-Of-Course Gap Results

Subject Subgroup GradePercent

Proficient/Advanced (2012-13)

Percent Proficient/Advanced

(2011-12)Gap Size

(2012-13)

Algebra I Black/Hispanic/Native American 9th through 12th 46.2 40.3 14.1Algebra II Black/Hispanic/Native American 9th through 12th 24 15.9 18English II Black/Hispanic/Native American 9th through 12th 39.8 41.7 19.7English III Black/Hispanic/Native American 9th through 12th 21.8 19 17.8Math Black/Hispanic/Native American 3rd through 8th 36 31.3 14.7RLA Black/Hispanic/Native American 3rd through 8th 33 32.8 17.3Algebra I Economically Disadvantaged 9th through 12th 50.9 45.2 21Algebra II Economically Disadvantaged 9th through 12th 28.4 19.6 25.1English II Economically Disadvantaged 9th through 12th 45.2 46.3 28.1English III Economically Disadvantaged 9th through 12th 26.8 24.2 25.3Math Economically Disadvantaged 3rd through 8th 39.6 35.9 26RLA Economically Disadvantaged 3rd through 8th 37.8 37.5 29.2

Additional Resources

04/19/2023 29

Resource Why It’s Helpful

Updated ESEA Flexibility Waiver This is the primary governing document behind the implementation of our Accountability Model; includes all amendments and changes to date to USED.

SAS Accountability Protocol 2013 Outlines the technical methodology used to implement the district accountability model.

Accountability Frequently Asked Questions 2013

Easy-to-access, just the bare-bones guides to district and school accountability. Lists out commonly asked questions with responses.

FYI: Working with Accountability Files

Recorded presentation that walks through how to access and navigate accountability files using the new structure.

FYI: District Determinations A recording of this presentation that includes general overview of district determinations, key dates, and procedures.

FYI: School Determinations We will host a Webinar on school determinations that includes an overview, key dates, and procedures. We will post after the Webinar is held. Dates forthcoming

30

Accountability Application- Announcements Tab

31

Accountability Application- AMO Dashboard


32

Accountability Application- AMO Dashboard


Additional Support

04/19/2023 33

Regional Data Analysts• Based in CORE Offices• Provide support on accessing and interpreting

accountability data• Answers general accountability/data questions• Several have consortium meetings where regional

administrators talk about best practices• Find Yours: http://www.tn.gov/education/CORE.shtml

TNED.Accountabilty• Direct access to TDOE Accountability Support Team via

email• Will answer questions on accessing and interpreting

data, accountability methodology, etc.

Password Resets• Every School Director should have a DATA account to

access the accountability application .• To request a reset, the Director must email

TNED.Accountablity

APPEALSAccountability 2013

04/19/2023 34

Appeals

District finds issue with data

District organizes and drafts appeal by July 19, 2013 at 5 PM CSTMust clearly describe data error by referencing the

file name and data point being questioned

Director submits appealThe appeal must be submitted as an attachment to an email to [email protected]. Appeal must be submitted on district letterhead.

TDOE Responds to Appeal

School Improvement Budget Amendment Process 35

Additional Support

04/19/202336

Regional Data Analysts• Based in CORE Offices• Provide support on accessing and interpreting

accountability data• Answers general accountability/data questions• Several have consortium meetings where regional

administrators talk about best practices• Find Yours: http://www.tn.gov/education/CORE.shtml

TNED.Accountability• Direct access to TDOE Accountability Support Team via

email• Will answer questions on accessing and interpreting

data, accountability methodology, etc.

Password Resets• Every School Director should have a DATA account to

access the accountability application.• To request a reset, the Director must email

TNED.Accountability

2013 Accountability Overview

Documents

Transcript of 2013 Accountability Overview