Measuring Student Growth
description
Transcript of Measuring Student Growth
![Page 1: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/1.jpg)
Measuring Student Growth
![Page 2: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/2.jpg)
Feedback Loop
![Page 3: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/3.jpg)
Key Concept
All models are wrong, but some are useful. George Box
![Page 4: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/4.jpg)
Two Conflicting ModelsPtolemaic Copernican
![Page 5: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/5.jpg)
Key Problems
Theta
Neither teachers nor students are randomly assigned
Lack of Randomness
![Page 6: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/6.jpg)
Topics
What are the main types of growth models and their application
What are the major problems presented by the use of growth models
What does the actual application of growth models look like
Assessment and scoring
![Page 7: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/7.jpg)
Item AnalysisDifficulty Index
p=(number correct)/(number of responses)
Discrimination Index
d=(((number correct upper group)-(number correct lower group))/(number in each group))
![Page 8: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/8.jpg)
Raw Score v. Scale ScoreRaw ScoreTotal point value of correct responses to valid items
Scale ScoreEquating adjustment to ensure that any given assessment is comparable to previous assessments
Vertical ScaleEquating adjustment that requires a higher score in higher grades
![Page 9: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/9.jpg)
CRT v. NRTCRTMeasure of performance relative to a delimited domain of learning tasks
NRTMeasure of performance relative to an individual’s standing in a group
In PracticeThe primary difference is in the way the scores are interpreted
![Page 10: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/10.jpg)
Three MeasuresCenter
Mean, Median, and Mode
SpreadVariance and Standard Deviation
DistributionWhat does the data look like?
![Page 11: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/11.jpg)
Distribution
![Page 12: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/12.jpg)
The Mean Follows the Tail
![Page 13: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/13.jpg)
Grade 6 Math 2010
![Page 14: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/14.jpg)
Grade 6 Math 2010
![Page 15: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/15.jpg)
Main Types of Growth Models
Trajectory
Value/Transition Table
Projection/Linear
![Page 16: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/16.jpg)
Trajectory (Growth to Proficiency)
Begins with the current score and the needed score for proficiency in the future, dividing the required student gains needed to reach this score into annual targets
Usually dependent on assessments with a vertical scale
![Page 17: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/17.jpg)
Trajectory Model
![Page 18: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/18.jpg)
Arkansas Example
![Page 19: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/19.jpg)
Value Table/Transition Model
Creates subdivisions of performance and awards credit for moving students to higher levels.
A categorical approximation of a trajectory model, not dependent on a vertical scale
![Page 20: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/20.jpg)
Year 1 Level
Year 2 Level
Level 1ALevel 1B Level 2A Level 2B Proficient
Level 1A 0 150 225 250 300
Level 1B 0 0 175 225 300
Level 2A 0 0 0 200 300
Level 2B 0 0 0 0 300
Proficient0 0 0 0 300
Value Table/Transition Model
![Page 21: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/21.jpg)
Previous Year
Current Year1 1.5 2 2.5 3 3.5 4 4.5
1 0 .5 1 1.5 2 2.5 3 3.51.5 -.5 0 .5 1 1.5 2 2.5 32 -1 -.5 0 .5 1 1.5 2 2.52.5 -
1.5-1 -.5 0 .5 1 1.5 2
3 -2 -1.5
-1 -.5 0 .5 1 1.5
3.5 -2.5
-2 -1.5 -1 -.5 0 .5 1
4 -3 -2.5
-2 -1.5 -1 -.5 0 .5
4.5 -3.5
-3 -2.5 -2 -1.5 -1 -.5 0
Arkansas Example
![Page 22: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/22.jpg)
Projection (Linear) Model
Uses current and past scores to predict performance in the future
Such models can be quite complex and difficult for stakeholders to understand
![Page 23: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/23.jpg)
Projection Model
![Page 24: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/24.jpg)
EVAAS Example
![Page 25: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/25.jpg)
SGP Example
![Page 26: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/26.jpg)
GMPP Evaluation
![Page 27: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/27.jpg)
GMPP Participation
State Growth ModelDeleware Transition MatrixIowa Transition MatrixAlaska TrajectoryArizona TrajectoryArkansas TrajectoryFlorida TrajectoryNorth Carolina TrajectoryOhio ProjectionTennessee Projection
![Page 28: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/28.jpg)
Results by State
![Page 29: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/29.jpg)
Schools Meeting AYPOverall, the increase of schools making AYP when growth was included was 16%
Biggest rates were in Ohio, 50%, Arkansas, 13%, and Tennessee, 10%
If Ohio’s results are excluded, the overall rate is only 4%
![Page 30: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/30.jpg)
Growth Model Comparisons
![Page 31: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/31.jpg)
Two Viewpoints
![Page 32: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/32.jpg)
Classification Errors
False negative: an effective teacher is classified as a less-effective teacher
False positive: a less-effective teacher is classified as an effective teacher
![Page 33: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/33.jpg)
The Widget EffectThe vast majority of school districts presently employ teacher evaluation systems that result in all teachers receiving the same (top) rating…. In districts that used binary ratings more than 99 percent of teachers were rated satisfactory. In districts using a broader range of ratings, 94 percent received one of the top two ratings and less than 1 percent received an unsatisfactory rating.
![Page 34: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/34.jpg)
…student test scores alone are not sufficiently reliable and valid indicators of teacher effectiveness to be used in high-stakes personnel decisions, even when the most sophisticated statistical applications such as value-added modeling are employed.
Teaching is Complex
![Page 35: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/35.jpg)
Teaching is ComplexThe use of imprecise measures to make high stakes decisions that place societal or institutional interests above those of individuals is wide spread and accepted in fields outside of teaching…. nearly all selective colleges use SAT or ACT scores as a heavily weighted component of their admission decisions even though that produces substantial false negative rates (students who could have succeeded but are denied entry).
![Page 36: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/36.jpg)
Perverse Incentives…research and experience indicate that approaches to teacher evaluation that rely heavily on test scores can lead to narrowing and over-simplifying the curriculum, and to misidentifying both successful and unsuccessful teachers. These and other problems can undermine teacher morale, as well as provide disincentives for teachers to take on the neediest students. When attached to individual merit pay plans, such approaches may also create disincentivesfor teacher collaboration.
![Page 37: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/37.jpg)
Perverse Incentives
Much of the fear concerning growth is about “use.” Take that fear away, and what is left is something that is very useful and which teachers are interested in knowing as well.
At least in terms of SGP, the “neediest” students represent a teacher’s best chance to demonstrate superior growth.
![Page 38: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/38.jpg)
Data Requirements
All growth models are dependent upon the ability to track a large percentage of student over time
This is especially difficult at the teacher level, where a host of issues will probably never fully be resolved
![Page 39: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/39.jpg)
Terminology
Is the term “value-add” loaded? Is the term “growth” more palatable and perhaps more descriptive?
![Page 40: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/40.jpg)
Other PopulationsEthnicity Count SGP_Median
A 5028 60B 82290 43H 37857 50N 2808 51P 1575 48T 4707 52W 259692 52
GT Count SGP_MedianN 341949 48Y 52299 63
Sped Count SGP_MedianN 354723 51Y 39619 38
ESICode Count SGP_MedianAU 1539 43DB 8 42.5ED 467 34HI 408 37.5MD 214 36.5MR 2345 28OHI 7374 34OI 135 44SI 9396 44SLD 17515 37TBI 82 33VI 136 41
![Page 41: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/41.jpg)
G5 2008 Math G6 2009 Math IncreaseNoah 702 775 73Ben 425 527 102
Who Had More Growth?
![Page 42: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/42.jpg)
2Percentiles 16 50 84 98
The Normal Distribution
![Page 43: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/43.jpg)
The Normal Distribution
![Page 44: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/44.jpg)
Student Growth Percentiles
![Page 45: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/45.jpg)
Quantile Regression
Ben
Noah
![Page 46: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/46.jpg)
G5 2008 Math G6 2009 Math 2009 Math SGPNoah 702 775 54Ben 425 527 44
Who Had More Growth?
![Page 47: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/47.jpg)
Math Scale and Growth
![Page 48: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/48.jpg)
Aggregated Growth Percentiles
![Page 49: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/49.jpg)
![Page 50: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/50.jpg)
Goodness of Fit
![Page 51: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/51.jpg)
Distribution
![Page 52: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/52.jpg)
Distribution by Cohort
![Page 53: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/53.jpg)
Prior/Current Math Scale
![Page 54: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/54.jpg)
Math Scale and Growth
![Page 55: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/55.jpg)
Prior/Current Math Growth
![Page 56: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/56.jpg)
Density Plot
![Page 57: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/57.jpg)
Current Math/Literacy Scale
![Page 58: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/58.jpg)
Current Math/Literacy Growth
![Page 59: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/59.jpg)
Density Plot
![Page 60: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/60.jpg)
THOMAS KANEhttp://metproject.org/
Wide variation among teachers
Wide variation within schools
Little (if any) differences in teacher preparation
Teachers improve, but improvement flattens after the third year
No schools where all teachers are highly effective
Metproject
![Page 61: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/61.jpg)
![Page 62: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/62.jpg)
Evaluating Teachers
![Page 63: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/63.jpg)
Evaluating Teachers
![Page 64: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/64.jpg)
Evaluating Teachers
![Page 65: Measuring Student Growth](https://reader036.fdocuments.in/reader036/viewer/2022062816/568167a0550346895ddceb44/html5/thumbnails/65.jpg)
Feedback?