SCHOLAR conference

17th June, 2011

Edinburgh

SQW evaluation of SCHOLAR 2010/11

Background Evaluation,

commissioned and funded

by:

Heriot Watt University

Access to pupil data and

support in kind

Learning and Teaching Scotland

Project management

The evaluation sought to provide:

Evidence of the impact of SCHOLAR on:

Attainment in specific subjects

The breadth of curriculum available in schools

Insights into:

Teachers’ approaches and methods as a result of using SCHOLAR

Students’ use of SCHOLAR

Views and recommendations related to the use of Glow as a platform for SCHOLAR – and SCHOLAR’s future development

Evaluation methodology

Scoping period (included interviews with an

opportunity sample of local Advanced Higher

students)

Designed, tested, hosted and analysed an online

survey of over 2,000 students

Secondary analysis of SCHOLAR data matched

with SQA attainment datasets (from 2008 to

2010)

Four case studies followed by a survey of

teachers (a ‘reality check’)

Interviews with stakeholders

Final analysis and report

Evaluation Team

Marian Morris,

Associate Director, SQW

Sheila Sim,

Senior Consultant, SQW

Walter Patterson,

Principal Consultant, Walter Patterson Consultancy

Secondary data analysis (SCHOLAR)

Over 6,000,000 ‘hits’ – converted into sessions data

Timing/location Intensity

Use of SCHOLAR

Number of users Number of sessions Sessions per individual

Male 32,521 652,44220.1

Female 33,724 580,33417.2

Unknown 734 3,4054.6

Total 66,979 1,236,18118.5

Pattern of useNumber of hits in session Length of session (mins)

Mean 12.63 20.84

Median 7.00 14.00

Mode 2 Less than 1

Standard Deviation 14.291 24.041

Minimum 2 0

Maximum 102 561

Percentiles

25 3.00 3.00

50 7.00 14.00

75 16.00 30.00

Longest continuous use was 694 minutes (revision for Advanced Higher

Biology) with clicks through a minimum of 23 pages

Pattern of use across years

Subject outcomes

.0

5.0

10.0

15.0

20.0

25.0

30.0

A B C D N

Higher Biology 2010

Percent

Attainment: by intensity of SCHOLAR use

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A B C D N

Pe

rce

nta

ge a

chiv

ein

g q

ual

ific

atio

n

Higher Biology 2010

All results

Low intensity

Medium intensity

High intensity

Intensity

Attainment: by timing/location of SCHOLAR use

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A B C D N

Axi

s Ti

tle

Higher Biology 2010

All results

Schoolday

School night

General

Weekend

Timing/locationBlended learning?

Attainment: Higher Physics 2010

.010.020.030.040.0

A B C D N

Attainment

0%

20%

40%

60%

80%

100%

A B C D N

Intensity of SCHOLAR use

All results

Low intensity

Medium intensity

High intensity

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A B C D N

Axi

s Ti

tle

Timing of SCHOLAR use

All results

Schoolday

Schoolnight

Weekend

General

Conclusions?

Higher attainment associated with:

• More intense SCHOLAR use

• Use of SCHOLAR in school and out (blended learning)

But – is more intense use simply a factor of greater pupil motivation?

• Need for analysis controlling for variables such as sex, ethnicity, first language, deprivation etc.?

Multi level modelling

Local authority

School

Student

Students in one class are more like each other than students in another class

Students in one school are more like each other than students in another school

Schools in one authority are more like each other than schools in another authority

Contact

Marian Morris

Associate Director

SQW t. +44 (0)20 7307 7178

07712 843 295

e. mmorris@sqw.co.uk

w. www.sqw.co.uk

Sheila Sim

Senior Consultant

SQW t. +44 (0)131 243 0730

07876 130 217

e.SSim@sqw.co.uk

w. www.sqw.co.uk