Self-Evaluation and School Improvement Using FFT Live
-
Upload
stacy-potts -
Category
Documents
-
view
17 -
download
0
description
Transcript of Self-Evaluation and School Improvement Using FFT Live
Mike TreadawayDirector of ResearchFischer Family Trust
Contents
FFT Live – Key Analyses
SecondarySecondary
FFT Live – Key Reports – Value Added
FFT Live – Key Reports - Estimates
FFT Live – Developments in 2010
KS4 (and KS5) Subject VA and Estimates
Value AddedValue Added
EstimatesEstimates
To Single Grade or Not to Single Grade?
Single Grade Estimates: Database• Single Grade estimate in the database• Calculated for subject groups
Single Grade Estimates: Database
FFTLive• FFTLive introduced a full range of probabilities
FFTLive
FFTLive – Single Grade
G F E D C B A A*Billy Onion 1% 8% 10% 20% 30% 20% 10% 1%
• Highlight the highest probability
Narrowing to the Middle• Let’s imagine a class of 10 pupils with exactly the same
estimates to Billy Onion
• Highlighted grade exported to the school MIS• The subject teacher sees.....
Estimate
Abigail Onion CBilly Onion CChristina Onion CDavid Onion CEmily Onion C
Estimate
Fatima Onion CGeoff Onion CHarry Onion CIsobel Onion CJanice Onion C
G F E D C B A A*Billy Onion 1% 8% 10% 20% 30% 20% 10% 1%
Narrowing to the MiddleEstimates should be averaged across the group
G F E D C B A A*Abigail Onion 1% 8% 10% 20% 30% 20% 10% 1%Billy Onion 1% 8% 10% 20% 30% 20% 10% 1%Christina Onion 1% 8% 10% 20% 30% 20% 10% 1%David Onion 1% 8% 10% 20% 30% 20% 10% 1%Emily Onion 1% 8% 10% 20% 30% 20% 10% 1%Fatima Onion 1% 8% 10% 20% 30% 20% 10% 1%Geoff Onion 1% 8% 10% 20% 30% 20% 10% 1%Harry Onion 1% 8% 10% 20% 30% 20% 10% 1%Isobel Onion 1% 8% 10% 20% 30% 20% 10% 1%Janice Onion 1% 8% 10% 20% 30% 20% 10% 1%Average 1% 8% 10% 20% 30% 20% 10% 1%Estimated no. of pupils 0 0 1 2 3 2 1 0
KS4 Development Report
Feature Details
Highlighting off as default Can be switched on if required
Chances shown as % for each grade
Option to show as cumulative
Option to highlight grades likely for top x% of students removed
Users found it confusing to have selections for both this and for school rank
Additional subjects Specific subjects within Science, MFL and Technology
Two Estimate Grades Est-2 : Grade likely to be achieved by, at most, 25% of studentsEst-1 : Somewhat more complicated …………………..
Est-1 : Explaining the mystery
Est-1 : Pros and Cons
Adding up highest probability grades
Too few A*/A and F/G gradesToo few A*/A and F/G grades
Calculating Points (from ordinal regression)
Issues – Accuracy and FairnessParticularly if used in context of evaluating progress of different teaching groups Issues – Accuracy and FairnessParticularly if used in context of evaluating progress of different teaching groups
Analysing Subject VA• A common approach is to use ordinal regression
• Issues with this are:– Fails (U grades)– Linearity– Granularity– Intervals
Fails and Linearity• KS2 Average English Level
• Similar pattern for over 80% of GCSE subjects at KS4• Using linear regression introduces significant errors for A*, A
and G grades
Granularity
• Regression Analysis (OLS) works well where inputs AND outputs are on a continuous scale
• Inputs are OK (fine grades)• Outputs are not – they are in clusters (grades)
– If we had e.g. UMS points … but we don’t!
Mathematics (random sample of 1000 records)
KS2 -> KS3 KS2 -> KS4
IntervalsIs the difference between A*/A, C/D and F/G grades the same?
YesYes NoNo Not sure, but probably noNot sure, but probably noYesYes
Responses are what we find whenever we ask subject leaders this question
<10% 30% 60%
We can debate whether or not their ‘gut feelings’ are justified
If, though, we can find a method of analysis which doesn’t care whether or not grade intervals are equal………
Solution
Outputs are chances not estimated points
NominalRegression
Likely Developments : Grades for Estimates
Estimates - Example
G F E D C B A A* A*C TM HTM TQ9% 23% 32% 26% 9% 0% 0% 0% 9% E 35% C3% 11% 24% 34% 25% 2% 0% 0% 27% D 27% B0% 0% 0% 2% 23% 38% 30% 7% 97% B 37% A*0% 0% 0% 4% 27% 39% 26% 4% 96% B 30% A*0% 0% 0% 1% 14% 34% 38% 12% 99% A 12% A*2% 8% 21% 35% 30% 3% 0% 0% 33% D 33% B0% 0% 1% 10% 45% 32% 11% 1% 89% C 44% A2% 8% 22% 39% 26% 2% 0% 0% 28% D 28% B0% 0% 0% 1% 13% 33% 39% 14% 99% A 14% A*0% 0% 0% 0% 6% 24% 44% 26% 100% A 26% A*
Estimates - TM
G F E D C B A A* A*C TM HTM TQ9% 23% 32% 26% 9% 0% 0% 0% 9% E 35% C3% 11% 24% 34% 25% 2% 0% 0% 27% D 27% B0% 0% 0% 2% 23% 38% 30% 7% 97% B 37% A*0% 0% 0% 4% 27% 39% 26% 4% 96% B 30% A*0% 0% 0% 1% 14% 34% 38% 12% 99% A 12% A*2% 8% 21% 35% 30% 3% 0% 0% 33% D 33% B0% 0% 1% 10% 45% 32% 11% 1% 89% C 44% A2% 8% 22% 39% 26% 2% 0% 0% 28% D 28% B0% 0% 0% 1% 13% 33% 39% 14% 99% A 14% A*0% 0% 0% 0% 6% 24% 44% 26% 100% A 26% A*
Estimates - TQ
G F E D C B A A* A*C TM HTM TQ9% 23% 32% 26% 9% 0% 0% 0% 9% E 35% C3% 11% 24% 34% 25% 2% 0% 0% 27% D 27% B0% 0% 0% 2% 23% 38% 30% 7% 97% B 37% A*0% 0% 0% 4% 27% 39% 26% 4% 96% B 30% A*0% 0% 0% 1% 14% 34% 38% 12% 99% A 12% A*2% 8% 21% 35% 30% 3% 0% 0% 33% D 33% B0% 0% 1% 10% 45% 32% 11% 1% 89% C 44% A2% 8% 22% 39% 26% 2% 0% 0% 28% D 28% B0% 0% 0% 1% 13% 33% 39% 14% 99% A 14% A*0% 0% 0% 0% 6% 24% 44% 26% 100% A 26% A*
Estimates - HTM
G F E D C B A A* A*C TM HTM TQ9% 23% 32% 26% 9% 0% 0% 0% 9% E 35% C3% 11% 24% 34% 25% 2% 0% 0% 27% D 27% B0% 0% 0% 2% 23% 38% 30% 7% 97% B 37% A*0% 0% 0% 4% 27% 39% 26% 4% 96% B 30% A*0% 0% 0% 1% 14% 34% 38% 12% 99% A 12% A*2% 8% 21% 35% 30% 3% 0% 0% 33% D 33% B0% 0% 1% 10% 45% 32% 11% 1% 89% C 44% A2% 8% 22% 39% 26% 2% 0% 0% 28% D 28% B0% 0% 0% 1% 13% 33% 39% 14% 99% A 14% A*0% 0% 0% 0% 6% 24% 44% 26% 100% A 26% A*
Exports
Issue 7 : Specific Subjects and Subject Types at KS4