Using data from the whole school Perspective

CEM CONFERENCE EXETER

Geoff Davies

Day 2 Final session 28th February 2013

What are schools about?

• Effective Learning

• Opening windows on a world

• Choices

• Realising potential

• Achievement and Standards

• Qualifications

BUT DO YOU RECOGNISE THIS???

You arrive at school one morning and you receive the following information

•Two pupils have misbehaved on the bus upsetting a third child

•A parent has phoned saying her child is being bullied in school and refuses to come to school

•A pupil has told her LSA that her mother has slapped her hard across the face the previous evening and there are finger marks on her face

•Two girls have been fighting in the canteen before school

•A member of staff is very upset after a disagreement with a colleague

•The gym roof is leaking and you have examinations scheduled there for the day

•You receive a serious complaint about a teacher’s conduct

•You are due to attend a meeting in an hour with the Chair of Governors

What are schools about?

• Preventive Detention• For some pupils just keeping them off

the streets is a major achievement

• Social function• Nobody else may take on this function

• Providing security and a safe environment

• Pupils not at ease will not learn

• Effective Learning

• Opening windows on a world

• Choices

• Realising potential

• Achievement and Standards

• Qualifications

Schools are complex organisations

Far too complex to make decisions by hunch!

Can you really know your school without detailed research and surveys?

Carol Fitz-Gibbon 2001 British Psychological Society

It gradually dawned on me that providing the data to schools was the most important outcome of the effort, far more important than writing research papers…..

The provision of data to practitioners meant that they participated in the research. Indeed they were the only ones who knew the surrounding circumstances for their classrooms, their department, each pupil, each family, etc. They were the major players: the ones who could interpret and learn from the detailed data.

25 YEARS OF CEM CENTRE DATA

…..there is a need for teacher researcher posts on the Senior Management team with a brief to develop research that is useful. Given time in the timetable thousands of teachers could become active researchers….

Educational research should be a practical reality contacting scientific enlightenment not a mathematical weight lifting exercise

Necessary knowledge base to use CEM systems to their potential

1. The forms of Value Added Data:• scatter graphs• raw and standardised residuals• SPC charts • tables of data• use of PARIS for further analyses (e.g. by gender, teaching group)

2. Predictive Data:• point and grade predictions• importance of chances graphs• availability of different predictive data

3. Baseline Data• band profile graphs• IPRs• Average GCSE score• Computer adaptive tests4. Attitudinal Data

• Make curriculum changes

• Adjust staffing structure and cater for student needs

• self-evaluation procedures including the analysis of examination results using value added data

• the target setting process

• school and department development plans…….

• Improve your monitoring and reporting procedures

• Provide information to governors

• Have conversations with feeder primary schools

• Etc etc

If you have the tools you can use them to do these

0.2

-0.6 -0.5-0.2 -0.3

-1.3

0.1

-0.1

0.1

-1.1

0.1

-0.1-0.3

0.4

-0.5 -0.5-0.9

-0.5

0.3

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

Art

& D

esig

n

Des

ign

& T

echn

olog

y

Eng

lish

Eng

lish

Lite

ratu

re

Fre

nch

Geo

grap

hy

His

tory

Mat

hem

atic

s

Med

ia S

tudi

es

Mus

ic

Rel

igio

us S

tudi

es

Sci

ence

: G

CS

E

Sci

ence

: G

CS

E A

dditi

onal

SC

Art

& D

esig

n

SC

Geo

grap

hy

SC

His

tory

SC

Rel

igio

us S

tudi

es

Voc

App

lied

Sci

ence

Voc

Hea

lth &

Soc

ial C

are

Ave

rag

e S

tan

dar

dis

ed R

esid

ual

GCSE value added A challenging school

Any result within the outer shaded area decreases the probability that the value added result is down to chance. The probability here is about 1 in 20

Outside the 99.7% confidence limit chance is les than 3 in a1000

Below are the value added charts from YELLIS to GCSE for two contrasting institutions. Which subjects are outside the confidence limits in a ‘negative value added’ way. There must be questions to ask regarding teaching and learning?Which subjects are outside the confidence limits in a ‘positive value added’ way?

0.0 0.0

0.4

0.9

0.10.2

1.4

0.4 0.3

-0.1

0.5 0.6

0.2

0.8

-0.1 -0.2

0.3 0.4

0.9

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

Art

& D

esig

n

Bus

ines

s S

tudi

es

Des

ign

& T

echn

olog

y

Dra

ma

Eng

lish

Eng

lish

Lite

ratu

re

Fre

nch

Geo

grap

hy

Ger

man

His

tory

Hom

e E

cono

mic

s

ICT

Mat

hs

Mus

ic

Phy

sica

l Edu

catio

n

Rel

igio

us S

tudi

es

Dou

ble

Sci

ence

Wel

sh

SC

IC

T

Ave

rag

e S

tan

dar

dis

ed R

esid

ual

GCSE VALUE ADDED A successful school

Sex Band Maths Vocab Pattern YELLIS Female B 46 61 53 54

Weight Subject YELLIS

Predicted Achieved

Grade Raw

Residual Standardised

Residual 1 Drama 5.6 (B/C) 8 (A*) 2.4 1.8 1 English 5.4 (B/C) 7 (A) 1.6 1.6 1 English Literature 5.4 (B/C) 7 (A) 1.6 1.4 1 French 4.9 (C) 8 (A*) 3.1 2.5 1 Geography 5.3 (B/C) 8 (A*) 2.7 2.2 1 Maths 5.2 (C) 7 (A) 1.8 1.7 1 Physical Education 5.4 (B/C) 6 (B) 0.6 0.4 2 Double Science 5.1 (C) 8 (A*) 2.9 2.6 1 Welsh 5.6 (B/C) 7 (A) 1.4 1.0

Weighted Average 5.3 (B/C) 7.4 (A*/A) 2.1 1.8

Student A

Sex Band Maths Vocab Pattern YELLIS Male A 55 71 76 63

Weight Subject YELLIS Predicted Achieved Grade Raw

Residual Standardised

Residual 1 Business Studies 5.7 (B/C) 4 (D) -1.7 -1.4 1 Drama 6.1 (B) 6 (B) -0.1 0.0 1 English 6.0 (B) 5 (C) -1.0 -1.0 1 English Literature 6.0 (B) 5 (C) -1.0 -0.9 1 Geography 6.0 (B) 3 (E) -3.0 -2.4 1 Maths 6.0 (B) 6 (B) 0.0 0.0 1 Physical Education 6.0 (B) 2 (F) -4.0 -3.1 2 Double Science 5.8 (B) 5 (C) -0.8 -0.7 1 Welsh 6.1 (B) 5 (C) -1.1 -0.8

Weighted Average 6.0 (B) 4.6 (C/D) -1.4 -1.1

Student B

Compare the data for Student A and Student B (not Kingston students)

Find students A and B on each of the scatter graphs English and Maths

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

YELLIS Test Score (%)

GC

SE

Gra

de

A*

A

B

C

D

E

F

G

U

Scatter Graph English

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

YELLIS Test Score (%)

GC

SE

Gra

de

A*

A

B

C

D

E

F

G

U

Scatter graph Maths

Is there anything to learn from these scatter graphs?

YELLIS ATTITUDINALYou are looking at the attitudinal feedback from year 10 over time:

Do you notice a pattern between the four charts. There was an initiative in the school that contributed, but was it sustainable?

Extended YELLIS Comparison charts for English and Maths

What concerns you about these charts? Can you suggest which is the stronger department?

This school has a five-year development plan which includes as one its goals:

“To help students prepare for university and the world of work by developing independent learning skills, the ability to reflect and to learn from others and to maximise the benefits to learning offered by emerging technologies.”

The graphs below reflect students’ perceptions of the style of learning that has been adopted in their A-level classes in two broadly similar subjects.

a) If the students’ perceptions are an accurate reflection of what takes place in the classroom, which subject seems more on board with the school’s development plan?

b) How would these perceptions inform the Senior Management Team’s evaluation of progress with its 5-year plan if the subject achieving significantly better value-added results was

i) Subject 1? ii) Subject 2?

Subject 1 Subject 2

YOU ALL HAVE DATA !

TRENDS IN READING SCORES

88

90

92

94

96

98

100

102

104

106

108

1990 1995 2000 2005 2010

YEAR OF ENTRY

AV

ER

AG

E L

ON

DO

N R

EA

DIN

G

SC

OR

E GIRL

BOY

AVERAGE L.R.S

PRIORITIES FOR YOUR SCHOOL

DO YOU SEE THE VARIOUS PICTURES FOR YOUR SCHOOL?

English KS2

3.6

3.7

3.8

3.9

4

4.1

4.2

1996 1998 2000 2002 2004 2006 2008

YEARA

V K

S2

En

glis

h

En KS2

DATA MAY BE COUNTER INTUITIVE?

What action was taken because of data revealing a priority?

• USE ONE YEARS DATA WITH CAUTION!

• Better to use three years data as patterns over time are more significant.

Using data to inform leadership decisions

1. Which data do I need AND which data do I not need? (e.g. MidYIS cohort or Independent Sector)

2. What does the data mean and what does the data not mean? (e.g. staff INSET and support)

3. Who is the data for?4. Storage, retrieval and use of data (e.g.

self-evaluation and preparing for Inspection)

Some key questions:

The use of this data needs to allow us to do our best to help every student to at least achieve if not exceed their potential.

It may challenge• The culture of ‘my’ school/college • Accountability policy• Expectations • Staff training in use of data and ability to cope with

data (data overload)• Integrating the data into school procedures,

storage, retrieval, distribution and access • Roles and Responsibilities

THE MOST IMPORTANT WORD IN EDUCATION???

FINALLY

CULTURE