Research into research use: A ten-arm cluster RCT 10 th September 2014 Dr Ben Styles, Dr Anneka...
-
Upload
marsha-griffin -
Category
Documents
-
view
218 -
download
1
Transcript of Research into research use: A ten-arm cluster RCT 10 th September 2014 Dr Ben Styles, Dr Anneka...
Research into research use:A ten-arm cluster RCT
10th September 2014
Dr Ben Styles, Dr Anneka Dawson (NFER), Dr Lyn Robinson and Dr Christine Merrell (CEM)
Background on evidence- based education
What research use experts said:• How are you going to measure outcomes?• Can you expect to see any difference?
Research dissemination: competing ideas
Passive methods are rarely effective
Passive methods are cheap i.e. a tiny effect is still cost effective
Measurement of attainment is cheap
(National Pupil Database)
Measurement of research use is complex
and expensive
Passive methods may not require consent
Several interventions
trying to achieve the same goal
Active methods can be expensive
Trial design
Two RCTs focused on improving Key Stage 2 literacy through teachers’ research use:
• Active trial- 780 primary schools randomised to experience one of varied activities from CPD to online communities (or control)
• Passive trial- Most other English primary schools randomised to receive one of various research-based communications (or control)
Outcomes
ACTIVE TRIAL PASSIVE TRIAL
Primary outcome Attainment in literacy at age 11
Attainment in literacy age 11
Secondary outcomes Teachers’ awareness, understanding and action in relation to research use
n/a
How many arms in the active trial?
• Four providers of research evidence, each with an active and passive approach
• Control group• CEM need three arms
Active trial
IEE
Electronic/ paper
materials
Evidence fair
Campaign for Learning/ Train
Visual
Website
Support
CEM
Paper materials
Light CPD
Advanced CPD
ResearchEd/ Nat Cen
Conference
Online community
Control group
780 schools recruited
60
60
60
60
60
60
60
60
60
240
Passive trial
IEE
Electronic/ paper
materials
Campaign for Learning/ Train
Visual
Website
CEM
Paper materials
ResearchEd/ Nat Cen
Conference
Control group
12,500 schools
2500 2500 2500 2500 2500
Timeline
Recruitment and baseline research
use measure
sAutumn
2014
Randomisatio
n
December
2014
Research
dissemination
Spring 2015
New teaching methods
and future
planning
Summer 2015
New teaching methods
Autumn 2015-spring 2016
Research use outcom
es
Spring 2016
Attainment
outcome
Summer 2016
Attainment
outcome
Summer 2017
CEM intervention
LEVEL 3: Evidence-based materials, two CPD sessions, pupil diagnostic information (InCAS), peer observations
LEVEL 2: Evidenced-based materials, one CPD session
LEVEL 1: Evidence-based materials (includes advice booklet, ‘tips for teachers’ cards and monthly classroom posters)
Challenge 1-Recruitment
Starting point- why do schools want to take part in research?:
• Having their say• Doing something good for education• The process being easy and sounding easy• Trust in the organisation approaching them• Security of data • Clear concise communication • Offer of feedback
Our recruitment approach
• Promotion at Research Ed and other provider events
• Word of mouth between schools and providers, NFER, EEF
• Website with detailed information on the trial and what is expected of schools and form to express an interest
• Letters, phoning, emails targeted response• Helping inform schools about RCTs including
our guide for schools
Recruitment (continued)
Challenge 2- Secondary outcome measurement
• Secondary outcomes are ‘awareness’, ‘understanding’ and ‘action’
• Measure using a bespoke survey • Schools involved in other research projects
using this survey cannot be in this trial
Research use outcomes
• Baseline survey developed without pilot• ‘Awareness’, ‘understanding’ and ‘action’
assessed through a series of questions in a survey of teachers
• Follow-up survey will be piloted in autumn• Questions will be retained if they discriminate
well and contribute to reliable measures
Intention-to-treat analysis
• Attainment (multilevel model, attainment at age 7 as covariate)
• Research use is also hierarchical but low numbers per cluster (research use at baseline as a covariate)
Challenge 3- Coordination of the delivery organisations
With 6 delivery partners carrying out different interventions things to consider:
• Effective, timely communication (inc one main point of contact at NFER and one at each provider)
• Funder as mediator for more basic updates• Monitoring information planned in advance
Acknowledgements
• Education Endowment Foundation• NFER Education Trials Unit
For more information
For Literacy Octopus project information:
www.nfer.ac.uk/octopus
For our guide for schools:
http://www.nfer.ac.uk/publications/RESM07/RESM07_home.cfm
For information on RCTs for educational researchers:
http://www.nfer.ac.uk/publications/RCT01/RCT01_home.cfm
For the NFER Education Trials Unit with details of our other trials
http://www.nfer.ac.uk/research/trials-unit/
NFER provides evidence for excellence
through its independence and insights, the
breadth of its work, its connections, and a
focus on outcomes.
National Foundation for Educational Research
The Mere, Upton Park
Slough, Berks SL1 2DQ
T: 01753 574123
F: 01753 691632
www.nfer.ac.uk