Evaluations in OLPCWhat for ? What has been done?

What could be done ?

Pierre Varly, Independent ConsultantOctober 22, 2010

Varlyproject.wordpress.com

Evaluations in OLPCWhat for ?

“An assessment for learning and not an assessment of learning”

Evaluation inputs : What works ? What’s don’t ? Why ?

Knowledge sharing in successful XO implementations

Capacity building of OLPC volunteers on education measurement and education intervention design

Better advocacy for OLPC based on common scientific standards measurement

Evaluations of 1:1 projects outstide the OLPC world

Only 22 researches produce estimates of ICT effects on pupils tests score

Little information is given on the pedagogical supports embedded on the computers

Evaluations mainly in US and Canada

Science or educational belief ??

1:1 reported outcomes

BETHEL E.C. & al (2009)

OLPC deployment contexts IT perspective

In the USA, the average access to home computer is 76% for 3-17 years old in 2003

Haïti RwandaCambodiaEthiopia Ghana Colombia Peru Mongolia Mexico Uruguay0

2

4

6

8

10

12

14

16

0.2 0.2 0.3 0.40.60000000

0000001

4.4

10.3

13.3 13.6 13.6Ratio of computers per 100 inhabitants in ma-

jor OLPC deployment countries (2005)

OLPC deployment contexts educationnal perspective

•High repetition and drop-out rates•Poor literacy environment •Inefficient teaching methods•Bad classrooms equipment•Very little effective learning time•Low social demand from parents•Language of instruction not spoken home•Very low learning outcomes•High gender, rural/urban and ethnic disparities

The reading problem in OLPC deployment contexts

Adapted from GOVE A. & CVELICH P. (2010), Early Reading: Igniting Education for All. RTI International

Afghanistan

Gambia Honduras (rural

schools)

Ethiopia Haiti Uganda (Central province)

Nepal Mali Uganda (Lango

province)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

21%27% 29%

36%48% 53%

79% 83% 88%

Proportion of non readers in the early grades

OLPC deployment actual evaluations

Only one paper published in a scientific review dealing with an OLPC deployment (Hourcade 2009)

As bad or as good as others 1:1 evaluations methods No measurement of effects size

Little information is given on tests used to measure learning outcomes (except for Sri Lanka, Haiti)

Too much emphasis on the attitudes and motivations of pupils in the reporting Impact evaluation reports expected 2010/2011

OLPC reported outcomes (7 countries)

7 countries, see (Varly 2010) for list of countries and data limitations

Motivation Attitudes Attendance Achievement0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0% 85.7% 85.7%

28.6%

50.0%

What is beeing evaluated ?

« That’s the answer, but what was the question ? »

What’s being evaluated ?

•XO ?

•SUGAR .

•SOCIO-CONSTRUCTIVISM THEORIES ?

Developing OLPC evaluations toolsSimple

Cheap

Adapted to OLPC deployments ‘context

Following scientific standards

Manageable by OLPC volunteers

Share experiences in the deployment and problems/solutions in integrating XO in the school and social environment

See (Varly 2010) for a proposal of tools

Case study of Nosy Komba : problems check list

Problem listCommon TICE

project problems

Developing world common problems

OLPC deployment common problems

Specific Nosy Komba

deployment problems

Lost XOs X

Installing school server X

Education authorities forbid XO use in

regular hoursXO ? XO

Teachers are not motivated X

Customs X

Invalid date system XO

Teachers don’t know what to do with XO X

Sugar activities do not match curricula XO

Formative evaluation well underway

Solomon Island framework and others

OLPC France blog

OLPC blogs and wiki reporting problems and solutions with regards to implementation and not just IT issues

Could be more systematic

Could be more standardised to browse info easily

Should be more centered on specific OLPC problems

Impact evaluation : take it or leave it ?

Impact evaluation so far initiated by international agencies and developement banks

Very costly

Long time for reporting but…

Good accessible methodology documentation

OLPC community should catch these powerfull tools further

Thanks for reading

Detailed paper and references on a Blog on Education in the developing countries

ACCESS PAPER HERE