Beyond the teaching and learning toolkit -putting evidence to work

Inspiring Leadership ConferenceBirmingham 12th June 2014

Professor Rob Coe and Kevan Collins

Kevan.collins@eefoundation.org.uk

www.educationendowmentfoundation.org.uk

The next 60 minutes…

• The case for an evidence led approach

• Putting evidence to work lessons from lesson observation lessons from transition studies

• An invitation to join the ‘quiet revolution’

Teaching and Learning Toolkit

• The Toolkit is an accessible, teacher-friendly summary of educational research. ‘Which?’ for education

• Practice focused: tries to give schools the information they need to make informed decisions and narrow the gap.

• Based on meta-analyses conducted by Durham University.

Using the Toolkit

Use the evidence as a starting point for discussion.

Dig deeper into what the evidence actually says

Understand the ‘active ingredients’ of implementation

The Toolkit is a starting point for making decisions

Do teachers need to know about research?

Lessons from Lesson Observation

Knowledge of research

A. A one-to-one numeracy intervention (two 15-minute sessions per week, delivered by teaching assistants) for Year 2-6 pupils who are struggling with numeracy (outcome: maths)

B. Nine weekly one-hour sessions where Y7 pupils below L4 read and discuss an age-appropriate book, with tools and resources to encourage reading for pleasure (outcome: reading)

C. A four-week summer school programme (between Y6 & 7) for pupils who had been predicted to achieve KS2 below Level 4b in English, focussed on poetry and writing (outcome: writing).

D. Y6 & 7 teachers trained to deliver a programme to help low attaining pupils plan, monitor and evaluate their writing using memorable experiences, eg trips and visitors (outcome: writing).

7

Put these in order of effectiveness:

8

Improving Teaching

• Teacher quality is what matters• We need to focus on teacher learning• Teachers learn just like other people

– Be clear what you want them to learn– Get good information about where they

are at– Give good feedback

Sept 2013: ResearchEd

Do We Know a Successful Teacher When We See One?

• Filmed lessons (or short clips) of effective (value-added) and ineffective teachers shown to– School Principals and Vice-Principals– Teachers– Public

• Some agreement among raters, but unable to identify effective teaching

• No difference between education experts and others• Training in CLASS did help a bit

10

Strong et al 2011 J Tchr Edn

Reliability

Probability that 2nd rater disagrees

1st rater gives % Best caser = 0.7

Worst caser = 0.24

Outstanding 12% 51% 78%Good 55% 31% 43%Req. Impr. 29% 46% 64%Inadequate 4% 62% 90%

Overall 39% 55%

11

Percentages based on simulations

Validity

Probability value-added data disagrees

1st rater gives %Best case

r = 0.4Worst case

r = -0.3

Outstanding 12% 71% 96%Good 55% 40% 45%Req. improv. 29% 59% 79%Inadequate 4% 83% >99%

Overall 51% 63%

12Percentages based on simulations

How can something that feels so right be so wrong?

cem.org/blog

Obvious – but not trueWhy do we believe we can spot good teaching?

• We absolutely know what we like– Strong emotional response to particular

behaviours/styles is hard to over-rule• We focus on observable proxies for learning

– Learning is invisible• Preferences for particular pedagogies are widely

shared, but evidence/understanding of their effectiveness is limited

• We assume that if you can do it you can spot it• We don’t believe observation can miss so much

14

Poor Proxies for Learning

• Students are busy: lots of work is done (especially written work)

• Students are engaged, interested, motivated• Students are getting attention: feedback,

explanations• Classroom is ordered, calm, under control• Curriculum has been ‘covered’ (ie presented to

students in some form)• (At least some) students have supplied correct

answers (whether or not they really understood them or could reproduce them independently)

15

Evidence based advice

Judgements from lesson observation may be used for low-stakes interpretations (eg to advise on areas for improvement) if at least two observers independently observe a total of at least six lessons, provided those observers have been trained and quality assured by a rigorous process (2-3 days training & exam).High-stakes inference (eg Ofsted grading, competence) should not be based on lesson observation alone, no matter how it is done.

What does ‘good teaching’ look like?

18

Improving Teaching

• Teacher quality is what matters• We need to focus on teacher learning• Teachers learn just like other people

– Be clear what you want them to learn– Get good information about where they

are at– Give good feedback

19

Danielson Frameworkhttp://danielsongroup.org/framework/

1. Planning and preparationa. Demonstrating Knowledge of

Content and Pedagogyb. Demonstrating Knowledge of

Studentsc. Setting Instructional Outcomesd. Demonstrating Knowledge of

Resourcese. Designing Coherent Instructionf. Designing Student Assessments

2. Classroom environmenta. Creating an Environment of

Respect and Rapportb. Establishing a Culture for Learningc. Managing Classroom Proceduresd. Managing Student Behaviore. Organizing Physical Space 20

3. Instructiona. Communicating with Studentsb. Using Questioning and Discussion

Techniquesc. Engaging Students in Learningd. Using Assessment in Instructione. Demonstrating Flexibility and

Responsiveness

4. Professional responsibilitiesa. Reflecting on Teachingb. Maintaining Accurate Recordsc. Communicating with Familiesd. Participating in the Professional

Communitye. Growing and Developing

Professionallyf. Showing Professionalism

Evidence-based standards for effective teaching?

• Evidence about relationships between teacher skills, knowledge & behaviours and ‘effectiveness’

• Evidence about what can be changed (and how)• Based on ‘best’ theories of

– Pupil learning– Pedagogy & teaching effectiveness– Behaviour change (individual, institutional, systemic)

• Most important: does focusing on these things lead to improvement?

21

22

Improving Teaching

• Teacher quality is what matters• We need to focus on teacher learning• Teachers learn just like other people

– Be clear what you want them to learn– Get good information about where they

are at– Give good feedback

Why monitor teaching quality?

• Strong evidence of (potential) benefit from – Performance feedback (Coe, 2002)– Target setting (Locke & Latham, 2006)– Intelligent accountability (Wiliam 2010)

• Individual teachers matter most• Everyone can improve• Teachers stop improving after 3-5 years• Judging real quality/effectiveness is very hard

– Multidimensional– Not easily visible– Confounded

23

Monitoring the quality of teaching

• Progress in assessments– Quality of assessment matters (cem.org/blog)– Regular, high quality assessment across curriculum (InCAS, INSIGHT)

• Classroom observation– Much harder than you think! (cem.org/blog)– Multiple observations/ers, trained and QA’d

• Student ratings– Extremely valuable, if done properly (http://

www.cem.org/latest/student-evaluation-of-teaching-can-it-raise-attainment-in-secondary-schools)

• Other– Parent ratings feedback– Student work scrutiny– Colleague perceptions (360)– Self assessment– Pedagogical content knowledge 24

Lessons from transition studies

The level 4 jeopardy…

,

Not achieving 5 A*-C (EM), 82,829

Tracked GCSE outcomes 2013

Achiev-ing L4 + 447,733

Not achiev-ing level 4

93,161

KS2 Outcomes 2008

Focusing on transition

In 2012 with government support the EEF funded 24 transition studies involving 1:10 secondary schools and many of their partner primaries and just under 17,000 pupils.

We asked schools to bring the best of their literacy transition work for evaluation and testing

Funded programmes included commercial products, school grown solutions, in and out of school activities

7 Lessons from transition studies

1.The educational chances of pupils who begin secondary school without having achieved Level 4 in literacy are very poor. In 2013, 72,000 children began secondary school without a Level 4 in reading. If this cohort follows previous trends, approximately 1 in 10 will achieve 5A*-C, including English and Maths at GCSE. 2. Due to the size of the gap, helping struggling readers catch-up with their peers in Year 6 is very challenging. It is highly unlikely that any single approach will be sufficient to close it. 3.Supporting struggling readers is likely to require a concerted effort across the curriculum, and a combination of approaches. 

7 Lessons from transition studies

4. When viewed in isolation some approaches seem to be more promising than others. On average, reading comprehension approaches appear to be more effective than phonics or oral language approaches for low attaining older readers. It may be that children who have not succeeded using phonics previously benefit from approaches which place a greater emphasis on meaning and context.

5.The effectiveness of any approach to reading catch-up is related to the pupil’s stage of reading development, so it is important that staff have the skill and training in diagnostic assessment, as well as in delivering any particular intervention.

 

7 Lessons from transition studies

6. Summer schools can improve reading ability but their effectiveness will be limited by the quality of teaching which takes place. In addition, it is likely that other approaches delivered in school will be more cost-effective.

7. Though some progress is possible, tackling reading catch-up at the transition is extremely difficult. The findings of this summary underline the importance of effective early intervention in Key Stage 1 and the beginning of Key Stage  

Conclusions

1. The new focus on evidence will support informed professional debate – it’s not a panacea

2. Implementation is all and putting evidence to work requires informed and engaged leadership

3. Enduring question - why is the education sector so weak at spreading and sharing lessons from disciplined and informed innovation?

Taking part in EEF research: Kevan.collins@eefoundation.org.uk