Leading Intervention 1

17th September 2009

CPD overview

LI1 17th September 9-12 Finstall• Role of intervention leader• Sources and types of data

twilight1 13th October 4-6 Finstall• Identifying pupils for intervention

LI2 17th November 9-4 Finstall• Tracking and spreadsheets• Looking at data in depth• Intervention models and approaches

twilight2 8th December 4-6 Finstall• Resources for intervention

CPD overview

LI3 12th January 9-12 Pitmaston• Proactive rather than reactive intervention• Quality first teaching• Effective use of TA’s

breakfast 11th February 8-10 WRFC, Warriors Centre

• Behaviour and attendance as part of intervention

LI4 21st June 9-12 Pitmaston• Evaluating the year• Planning ahead

Role of Intervention Leader

Track pupil progressBe a source of data and analysis, and adviceCoordinate resources for staff and pupilsHave an overview of interventionMonitor the impact Liaise with subject leaders, with pastoral leaders,

with Strategy Manager, with school Data Manager etc

Aims of the morning

Be familiar with sources and types of data School data FFT RAISEonline

Begin to identify pupils for intervention

Begin to effectively analyse data

Using data key messages

• Data collection does not in itself solve anything• Data provides questions not answers• Data analysis should be used to promote

discussion, evaluation and planning• Analyses for different groups of pupils, and a

range of indicators, help identify strengths or areas for development/intervention

• Use the past to inform the future

What’s the point?

Historical data• Assessments at key points• Ongoing assessmentsTargets• Where should they be in the future? (externally set,

internally set, adjusting for pupil progress)• Interim targetsMonitoring• Comparing progress with targets (individuals and

overall) and reactingSummaries• Comparisons of overall estimates/targets with

actual results

Sources of data & targets

What are you collecting data for?

What data do you need?

Prioritise!SATs

Formal Teacher Assessments

Ongoing Teacher Assessments

Homeworks

Tests

Exams

Projects

FFT

Raiseonline

Pupil self-assessments

Traffic lighting

School historical data

Expected progress rates

Mocks

Behaviour School attendance

Lessonattendance

Rewards andsanctions

?

Data calendar

What decisions are made as a whole school?What decisions are made within subject/pastoral

teams?

What data is collected centrally by the school?What data is collected by subject/pastoral teams?

Does data inform decisions?Are decisions based on data?Could the process be improved?

Fischer Family Trust

Aims to help schools make effective use of test and TA data

Database of all matched pupilsIncludes past test and TA dataIncludes estimates of future attainment based on

national progress patterns

Using Prior Attainment and School Context

Using Prior Attainment as an indicator of future performance, we know:

• KS3/4 attainment is highly dependent on prior attainment• Girls make different progress to boys• Autumn born pupils have higher attainment than Summer

born pupils• Pupils’ prior attainment in English often has a greater impact

on subsequent progress than attainment in Maths or Science

Taking account of School Context, we know:• Pupils from deprived backgrounds tend to make less progress

(geodemographic data)• The spread of prior attainment for the cohort can have an

impact on estimates of future achievement

FactorsPupil Factors PA SE SX

Mean Test Level (Fine Grade)

Mean TA Level

Subject Variations

Gender

Month of Birth

EAL

FSM

SEN Stage, Statemented

Ethnicity

Mobility (joined late / time in school)

School Factors PA SE SX

Mean Intake Test Level

Spread of Intake Test Level

FSM Entitlement (Percentile Rank)

Geodemographic Data (Percentile Rank)

Value Added Models

Model PA (Prior Attainment)

Model SE (Socio Economic)

3 value-added models have been developed:

Model SX (School EXtended)

Prior Attainment

Gender Month of Birth

GenderSchool Context

Prior Attainment

Month of Birth

GenderSchool Context

Prior Attainment

Month of BirthPupil

Context

Which estimate type?

Advantages Disadvantages

Type A(PA)Similar pupils

Prior Attainment is a major factor upon future performance

Doesn’t take account of context. Doesn’t stretch pupils in advantaged schools

Type B(SE)Similar pupils in similar schools

Is a more accurate reflection of what actually happens

No element of challenge

Type D(SE + Challenge)Similar pupils in similar schools

Stretches pupils in schools with high value-added

May still be too low for schools in the top few % nationally for value-added

FFT Reports – www.fftlive.org

So where do the estimates come from?

English Maths Science Average Points Score

Rebecca Mango 4 4 4 27

Jack Tomato 4 4 4 27

Tim Cumin 4 4 4 27

David Apricot 4 4 4 27

Abigail Garlic 4 4 4 27

Jane Apple 3 5 4 27

Edward Onion 4 4 4 27

Liam Peppercorn 4 4 4 27

Take eight ‘similar’ students at the end of Key Stage 2:

All 8 students have the same overall prior attainment using an Autumn Package points score.

The chances graph Average points score of 26 to 28

Last year, 33% of

students with average

points score of 26-28

achieved grade C

So… estimate of 55% chance of achieving grades A*-C

26<= KS2 Average Point Score <=28

…and estimate of 77% of achieving D+

FFT factors: Prior Attainment Model

Difference between students

KS2 Attainment A* -C Estimate

En Ma Sc Pts Score PA Model

Rebecca Mango 4.2 4.4 4.8 23%

Jack Tomato 4.2 4.4 4.8 23%

Tim Cumin 4.4 4.0 4.5 23%

David Apricot 4.9 4.7 4.6 23%

Abigail Garlic 4.8 4.6 4.3 23%

Jane Apple 3.8 4.4 5.5 23%

Edward Onion 4.5 4.5 4.4 23%

Liam Peppercorn 4.5 4.5 4.4 23%

Gender26%

10%

Month of birth

16%

18%

Marks4%

36%

Subject differences

48%

12%

Student Estimates

Most likely levelModel Used

Prior AttainmentYear 6 Test

& TA

Level achieved bytop 5% - 25% ofsimilar pupils

ProbabilityA* - C

Example Student

NC SS Y6

Y6 Test Levels Y6 Teacher Assessment

Eng Ma Sci Eng Ma Sci

116 5 4 5 5 4 5

% chance of achieving KS4 Grade % chance

G F E D C B A A* A*-C Pass

1% 1% 2% 10% 31% 36% 17% 3% 87% 99%

Prior Attainment

Estimates

Questions:• What would you expect this student to achieve at GCSE? • What targets would you set?• What other information would you need to reach a more informed

decision?

Grade B or C

Example Student

NC SS Y6

Y6 Test Levels Y6 Teacher Assessment

Eng Ma Sci Eng Ma Sci

116 5 4 5 5 4 5

% chance of achieving KS4 Grade % chance

G F E D C B A A* A*-C Pass

1% 1% 2% 10% 31% 36% 17% 3% 87% 99%

Prior Attainment

Estimates

Activity

Highlighted sheet (Pupil Estimates Type D)• Estimated grades highlighted if close to boundary• High percentage chances in yellow• Low percentage chances in pink• Rest in blue

Without additional action, how many A* - C would you expect?

Which students would you target for intervention?What other questions would you want answered?What action would you take next?

School Estimates CHANGE

Range of Estimates

• Matched students only

• A, B, D• Box: 3 year trend

for this school• LA guidance is to

use Type D to build in some challenge

• Includes E, M 2 levels progress

• Targets may be set above these

• Also: Breakdown by gender, upper/ middle/lower, etc

So where do these estimates come from?

Difference between

pupils

KS3 Attainment A* - C Estimate

En Ma Sc Pts Score PA Model

Rebecca MangoGender

4.2 4.4 4.8 23% 26%

Jack Tomato 4.2 4.4 4.8 23% 10%

Tim CuminMarks

4.4 4.0 4.5 23% 4%

David Apricot 4.9 4.7 4.6 23% 36%

Abigail Garlic Subject differences

4.8 4.6 4.3 23% 48%

Jane Apple 3.8 4.4 5.5 23% 12%

Edward Onion Month of birth

4.5 4.5 4.4 23% 16%

Liam Peppercorn 4.5 4.5 4.4 23% 18%

Kim Bolton 24%

Basil Don 6%

How many A* - C would you expect from this list?

For the FFT school estimate:

Add the percentages, divide by 100

26+10+4+36+48+12+16+18+24+6 = 200

200 ÷ 100 = 2

So A* - C estimate is 2 students out of the 10

What are the implications for intervention?

Activity

Highlighted sheet again

What are the A* - C estimates for the sheet, using the

% chances?What are the implications for intervention?

Accessing the FFT data

Website (www.fftlive.org), password from Data Manager

• Updated automatically with validated data• ‘old’ estimates will be overwritten

Know the school policy on the versions:• Which version are the estimates taken from?• Are they fixed for the year or key stage, or are

they flexible?

FFT Key messages

Use the individual pupil estimatesUse the school estimatesBe aware of both when identifying pupils for

intervention

Use the ‘actuals’ reports to review the success of intervention and inform future action

Raiseonline

Match the cardsHow many can you match in 5 minutes?

STOP!

Estimates/Targets

National expectations are set by DCSF

Estimate is based on statistical evidenceAn estimate may be a likely outcome for a typical

school, or a likely outcome for a school performing in the top 25%

Prediction is based on past performance + professional knowledge of a pupil

Target is based on prediction and builds in aspiration

Raiseonline

www.raiseonline.org

Based on each pupil’s prior attainmentCompares top 50% and 25% of schoolsShows estimates (as targets) for pupils, groups,

cohorts Allows moderation of the suggested pupil targets

Two sets of pupil data

Initially these are identical

National provided data

School’s own data

• Oct/Nov• Spring• July• Updates overwrite any

school amendments

• Amended pupil results• School defined pupil

attributes and teaching groups

• Optional test data• Question level data• Moderated pupil targets

Updated/amended by the school Data Manager

Updated by DCSF

Can be shared with Ofsted, SIPs, LA;

sharing requires school action

Used for the full PANDA report,available to

Ofsted, SIPs, LA

RAISEonline

The Report Wizard

view all analyses

Reports with grouping

Use the drop-down boxes to change• graph• year• subject• gender• other groupings

Click this link to save any report you find usefulChange the file name if you wish, then Save

Click on data points to identify groups or pupils

Interactive VA graphs

RAISEonline Key messages

Use the individual pupil resultsUse the school resultsCompare with the national pictureUse these to reflect on the success of previous

intervention and hence to inform future action

FFT v RAISEonline

use historic national pupil data use social context dataprovide summaries of attainment

creates estimates of future attainmentsummaries largely at school level

database includes FFT estimatessummaries include national data for comparisonallows question-level analysis

FFT

both

RAISEonline

So…

Use FFT pupil estimates, FFT school estimates to aid selection of pupils for intervention

Use RAISEonline pupil & school tables & graphs to learn lessons from the past to inform future intervention

Use school data and teacher knowledge to refine selection of pupils & choice of intervention package

National Expectations

Sets out DCSF expectations:KS1 to KS2

all L2 + 45% of L1 L4All to make at least 1 level of progressAll should make at least 2 levels of progress

Reporting:

% achieving L4+ in both Eng and Ma% making 2 levels of progress in Eng% making 2 levels of progress in Ma

National Expectations

Sets out DCSF expectations:KS2 to KS3

all L4 + 50% of L3 L5(and increasing majority L6)all L5 (in both Eng and Ma) L6(and increasing majority L7)All to make at least 1 level of progress

Reporting:

% achieving L5+ in both Eng and Ma, and % L5+ in Sci% making 2 levels of progress in Eng% making 2 levels of progress in Ma

National Expectations

Sets out DCSF expectations:KS3 to KS4

30% of average L5 + all L6 5 A*-C (incl Eng and Ma)All L6 (in Eng and Ma) make 2 levels of progress in bothIncreasing majority of L5 in Eng and Ma make 2 levels of progress in both.

Reporting:% of 5A*-C including Eng and Ma% making 2 levels of progress in Eng% making 2 levels of progress in Ma

Setting targets

Estimates should be used to SUPPORT planning and target setting:

• FFT reports offer estimates not targets• Estimates help us to set targets• A teacher’s professional knowledge of the pupil

is vital in target setting• Targets are not predictions• Targets should be aspirational• Targets need not be fixed• School policy may impact on target-setting

So…

Use the individual student estimatesUse the subject estimatesUse your department’s knowledge of the studentsCreate moderated targetsUse these when identifying students for intervention

Use the ‘actuals’ reports to review the success of intervention and inform future action