Va 101 ppt

Value-Added Analysis: Update Information & Value-Added 101

Copyright ©2011. Battelle for Kids.

The Development of the State System

Resources to Support Value-Added Work in Ohio

Value-Added 101 Review

Understanding Key Reports

Presentation Overview


Value-Added in Ohio: The Development of the System


How We Got To Where We Are

2002—Project SOAR established for school improvement in 42 school districts Grades 3-8, math, reading, science, social studies Different testing regimen (URM model) Re-normed every year

2007—State System established for accountability Grades 4-8, reading and math Results were anchored to the 2007 student results (MRM model) System is re-normed every 3-4 years. Was re-normed in 2010

Both the URM & MRM are SAS® EVAAS® approaches to value-added modeling – Dr. Bill Sanders


Why Two Systems in One State?

The MRM approach: Is the preferred model when there is consistent,

consecutive testing from year to year Measures growth to a predetermined standard Allows gains to be averaged over time Uses new data to make prior estimates more precise

The URM approach: Accommodates multiple kinds of tests and differing

testing regimens; therefore, it is re-normed each year Provides a comparative measure of growth based on

the pool


NOW…

A unified system and expansion of teacher-level reports statewide

One Value-Added (VA) System for Ohio: Ohio VA EVAAS reporting

Transparency for LEA accountability on report cards School improvement

Expanded reporting LEAs in BFK projects receive additional reports (no

duplication of grades/subjects that ODE provides) Teacher-level reporting statewide


Report Enhancements for 2011

Addition of science in grades 5 and 8 (URM)

Inclusion of students' historical data across LEAs Previously not included in analysis due to migration across

LEAs Will increase school improvement utility for LEAs with high

mobility

Export functionality District & School Administrator Accounts District & School Value-Added Reports and Student

Projections


Report Enhancements for 2011 (cont’d)

Previously, only students who were used for accountability purposes were available in EVAAS

Report views are now available for both the tested students and the accountability subset of students


Recent Advances Continuation of student names for Ohio EVAAS

reports Districts and ITCs submit student names for the SSID

crosswalk file each year. An additional benefit of getting this done each year is that

teachers are able to view reports with re-rostered student information.

Projections for 1 & 2 years and pie charts

Integration of VA into the Ohio Improvement Process


Value-Added Analysis in the Ohio Accountability System & Evaluation


Value-Added & Ohio Accountability

Value-added analysis can affect your state designation on Ohio’s Local Report Card

AYP Growth Measure Projection information (SAS) Provides 4th opportunity to meet AYP No negative implications % of students who have greater than 50% probability

of meeting the AYP standard in the future


Two New Accountability Rules State Board of Education (SBOE) established a more rigorous

statistical definition for the growth standard, moving it from one to two standard errors of measurement.

Impact of value-added measures to Local Report Card ratings. Reduce the time period for value-added impact on the report card designation from the current “2 years plus (green) to go up, and 3 years minus (red) to go down” The rule changed to: One year to go up if VA is above Two consecutive years to go down if VA is below

2011 = Transition Year. Still need 3 years in the red to be negatively effected. ODE will use a 2 standard error, versus 1, when looking at the past 3 years.


Value-Added & Teacher Evaluation Race to the Top Senate Bill 5 House Bill 153: Requires LEAs to establish new policies

relative to teacher & principal evaluation Multiple measures to include student growth metrics as 50% of

evaluations (aligns with OTES) Implementation timeline is 2013-2014 school year Performance-based compensation is a major national

component of education reform. If it was NOT included as a component in LEA approved RttT scope of work, RttT LEAs are NOT required to implement, but are strongly encouraged to engage in this work as are all LEAs in Ohio

SBOE shall adopt a salary schedule for teachers based on performance


Resources to Support Value-Added in Ohio


Key Statewide Deliverables of RttT Teacher-Level Value-Added Reporting

30% of LEAs link in Year 1 RttT (reports received fall 2011) –primarily LEAs in Battelle for Kids’ expanded value-added report projects along with some SIG schools

60% of all LEAs link in Year 2 (represents all RttT LEAs) 100% of all LEAs in Ohio link in Years 3 & 4 Requires teacher linkage each spring to verify teacher

assignments and teachers’ instructional time with students

Professional development & resources will address the use of value-added for school improvement and implications of teacher-level reporting


Support Resources Value-Added Toolkits – 2 per central office; 1 per

building, grades 4-8 Includes URM examples Problem-based approach Key reports organized by audience

Refresh of Value-Added Online Courses Greater interactivity Condensed and practical

Annual webinar series on various key topics

Resource information found on the Ohio StudentProgress Portal: www.BattelleforKids.org/Ohio


Value-added 101: Why value-added?


In a perfect world: Students start at the same place Students progress at the same pace Achievement test scores are enough to

show growth

Stair Step Expectations


In reality: Students start at different places Students progress at different rates Need more than scores on a single test to show

a school’s effectiveness

Differentiated Reality


We must expect progress for ALL students

Jacob

Adam

Standard

Grade

Pro

fici

ency

Why Value-Added is Necessary


Achievement and Progress


School Performance and Poverty Level


School VA Gains and Poverty Level


How do value-added measures support what we know about schools?

The Power of Two:Achievement & Progress

Pro

gre

ssO

ne Y

ear’

s G

row

th

AchievementTest Results

Standard

Low ProgressLow Achievement

Low ProgressHigh Achievement

High ProgressLow Achievement

High ProgressHigh Achievement

• School A

• School B

• School C

• School D

• School E

• School F

• School H

• School K

• School G

• School J


What is a Growth Measure?

Growth = Observed Score – Baseline Score

2014

2013

2012

2011

2010


Tests are the most convenient way to measure students’ achievement levels, but: All measures have error Students don’t always score where they should

Guessing Cheating Other outside factors beyond a teacher’s control

Small numbers of students in some teachers’ classrooms make it even more difficult to produce a system that is fair

Why is it so difficult to measure growth?


Tests must possess the following three properties: They must be highly correlated to curricular

objectives They must have sufficient stretch to differentiate

student achievement levels at both the lower and higher ends

They must be sufficiently reliable

Required Test Properties


Follows individual students across time

Uses all available student test data

Students’ growth is compared to their own history Estimates the school’s influence or “school effect” on

a group of students

Growth expectations are a policy decision and can be fixed or normative

Statistical models can accommodate various testing regimens

EVAAS®: Value-Added Analysis Overview


MRM Example

Scale scores are converted to NCEs

294 = 26 Student 1 360 = 37

302 = 27 Student 2 327 = 30

284 = 19 Student 3 350 = 35

294 = 26 Student 4 372 = 41

310 = 28 Student 5 335 = 33

Mean Baseline = 25.2

Grade 6Baseline

Grade 7Observed

Mean Observed = 35.2

A crude measure of the growth for this group is 10.0 NCEs

Growth = Mean Observed – Mean Baseline

Growth = 35.2 – 25.2 = 10.0


What is a Normal Curve Equivalent (NCE)?

1 10 20 30 40 50 60 70 80 90 99

1 5 10 20 30 40 50 60 70 80 90 95 99

Normal CurveEquivalent

Distributionof Scores

Normal CurveEquivalents

PercentileEquivalents


URM Example Using student results from all tests past and

present, correlations between tests are calculated

For each student in each subject area, a predicted score is produced by using up to five years of prior test results

Predicted scores for a cohort are averaged Value-added result is the difference between

students’ average observed score and their average predicted score


Understanding Key Value-Added Reports


Key ReportsSchool Value-Added (MRM)


Key ReportsSchool Diagnostic (MRM)


Key ReportsSchool Value-Added (URM)

OAA Science

2011 School Value-Added Report

School-Glx in Snd School District


Key ReportsSchool Diagnostic (URM)

8th-Grade OAA Science


Questions?

Thank you!

Resource information found on the Ohio Student Progress Portal:

www.BattelleforKids.org/Ohio

Support Desk: [email protected] or (866) 543-7555

www.BattelleforKids.orghttp://twitter.com/BattelleforKids

Va 101 ppt

Education

Transcript of Va 101 ppt