SCHOLAR Conference 2011 - Innovative Delivery of Higher and Advanced Higer - Workshop Part 2
SCHOLAR Conference 2011 - Evaluation of SCHOLAR
15
SCHOLAR conference 17 th June, 2011 Edinburgh SQW evaluation of SCHOLAR 2010/11
description
Does SCHOLAR improve exam results? The results of the independent evaluation
Transcript of SCHOLAR Conference 2011 - Evaluation of SCHOLAR
SCHOLAR conference
17th June, 2011
Edinburgh
SQW evaluation of SCHOLAR 2010/11
Background Evaluation,
commissioned and funded
by:
Heriot Watt University
Access to pupil data and
support in kind
Learning and Teaching Scotland
Project management
The evaluation sought to provide:
Evidence of the impact of SCHOLAR on:
Attainment in specific subjects
The breadth of curriculum available in schools
Insights into:
Teachers’ approaches and methods as a result of using SCHOLAR
Students’ use of SCHOLAR
Views and recommendations related to the use of Glow as a platform for SCHOLAR – and SCHOLAR’s future development
Evaluation methodology
Scoping period (included interviews with an
opportunity sample of local Advanced Higher
students)
Designed, tested, hosted and analysed an online
survey of over 2,000 students
Secondary analysis of SCHOLAR data matched
with SQA attainment datasets (from 2008 to
2010)
Four case studies followed by a survey of
teachers (a ‘reality check’)
Interviews with stakeholders
Final analysis and report
Secondary data analysis (SCHOLAR)
Over 6,000,000 ‘hits’ – converted into sessions data
Timing/location Intensity
Use of SCHOLAR
Number of users Number of sessions Sessions per individual
Male 32,521 652,44220.1
Female 33,724 580,33417.2
Unknown 734 3,4054.6
Total 66,979 1,236,18118.5
Pattern of useNumber of hits in session Length of session (mins)
Mean 12.63 20.84
Median 7.00 14.00
Mode 2 Less than 1
Standard Deviation 14.291 24.041
Minimum 2 0
Maximum 102 561
Percentiles
25 3.00 3.00
50 7.00 14.00
75 16.00 30.00
Longest continuous use was 694 minutes (revision for Advanced Higher
Biology) with clicks through a minimum of 23 pages
Pattern of use across years
Subject outcomes
.0
5.0
10.0
15.0
20.0
25.0
30.0
A B C D N
Higher Biology 2010
Percent
Attainment: by intensity of SCHOLAR use
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
A B C D N
Pe
rce
nta
ge a
chiv
ein
g q
ual
ific
atio
n
Higher Biology 2010
All results
Low intensity
Medium intensity
High intensity
Intensity
Attainment: by timing/location of SCHOLAR use
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
A B C D N
Axi
s Ti
tle
Higher Biology 2010
All results
Schoolday
School night
General
Weekend
Timing/locationBlended learning?
Attainment: Higher Physics 2010
.010.020.030.040.0
A B C D N
Attainment
0%
20%
40%
60%
80%
100%
A B C D N
Intensity of SCHOLAR use
All results
Low intensity
Medium intensity
High intensity
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
A B C D N
Axi
s Ti
tle
Timing of SCHOLAR use
All results
Schoolday
Schoolnight
Weekend
General
Conclusions?
Higher attainment associated with:
• More intense SCHOLAR use
• Use of SCHOLAR in school and out (blended learning)
But – is more intense use simply a factor of greater pupil motivation?
• Need for analysis controlling for variables such as sex, ethnicity, first language, deprivation etc.?
Multi level modelling
Local authority
School
Student
Students in one class are more like each other than students in another class
Students in one school are more like each other than students in another school
Schools in one authority are more like each other than schools in another authority
Contact
Marian Morris
Associate Director
SQW t. +44 (0)20 7307 7178
07712 843 295
w. www.sqw.co.uk
Sheila Sim
Senior Consultant
SQW t. +44 (0)131 243 0730
07876 130 217
w. www.sqw.co.uk
