Evaluation of a Teachers’ Incentives scheme

Ministry of Education and TrainingLESOTHO

Accra, Ghana ,14 May 2010

Contextual Factors

• Objective of the MOET outlined in the Education Sector Strategic Plan (2005 2015) and the Medium Term Education Sector Plan (2009-2012) is improve Access and Equity, Quality and Relevance and Efficiency of education at all levels.

Cont… • In 2000 – Government introduced Free Primary

Education

• Heavy investments made in construction of many Primary schools and additional classrooms throughout the country

• Provision of free text books and other learning materials

• Provision of school feeding

Cont...• Impressive improvements realized in the Education

Sector:• Net Enrolment Rates increased from 60% in 2000 to

80% in 2001 and have been sustained at this high level;

• Enrolments of significant numbers of orphans and vulnerable children, which otherwise would not been realized, due to high levels of poverty;

• Gender equity realized, indicated by a gender parity index of around 1

• Lesotho was on track to meet the Millennium Development Goals on access and equity

Cont...• Very important - the public was happy with the

increased access to education and the Government of the day became very popular;

• However, evidence started to show that quality of education is lagging behind;

SACMEC III study (2007)comparing education quality across 15 countries from the Southern and Eastern African region, indicated that Lesotho fell amongst 4 lowest scorers in English reading and mathematical competencies;

National Assessments and EMIS also indicated low net cohort survival rates (62.8%), high repetition rates (20%) in 2009, low completion rates and high drop-outs.

Cont...

• Though a study on the causes of lagging quality was not done, it could be observed that the impressive increase in enrolment were not accompanied by equal increase in the supply of qualified teachers. As a result, a significantly high percentage of teachers were unqualified (40% in 2008).

Problem Statement• Lesotho is largely mountainous, with limited

infrastructure , therefore access to some regions is difficult;

• Out of 1500 existing primary schools, a significant number (estimated around 650) is located in difficult areas;

• Qualified teachers are not willing to teach in these difficult schools;

• There is a high proportion of unqualified teachers in Lesotho (40% in 2008), but the percentage is even higher in difficult schools.

Cont…• On the other hand, unqualified teachers are

willing to upgrade, mainly through enrolment into Distance Teacher Education Programme (DTEP), but are constrained by cost and space.

• Even the few who manage to upgrade, have a tendency to transfer to less difficult schools after graduating.

Impact

• Insufficient qualified teachers in primary education in Lesotho, and particularly in difficult schools

• High number of unqualified teachers in these schools

• Low quality of education indicated by low net cohort survival and completion rates, high repetition and drop-out rates - county wide, but more so in difficult schools.

Intervention

• Government revised a teachers’ salary structure to attract capable people into the teaching profession, that was introduced from April 2008.

• It resulted in significant increases in salaries and benefits, nation- wide.

• Immediately afterwards, in 2009/10, the major source of Lesotho revenue accounting for 40%, namely the Southern African Custom union (SACU) shrunk significantly by almost 50%.

• This threatens the sustainability of these attractive remunerations;

Cont...• MOET is aware that salary increases offered nationwide,

will not necessarily immediately attract qualified teachers into difficult schools.

• The impact of this policy is yet to be realised. Though heavy investments have been made, there were no accompanying measures to evaluate the policy systematically.

• MOET is collaborating with the World Bank to design and implement a teachers incentive scheme for the attraction and retention of teachers in difficult schools.

This workshop, including the country interactions with Marie Helen and Deon have been ceased as a good opportunity to learn and think through design and implementation of both the intervention and the impact evaluation, with considerable progress realised.

The envisaged scheme is constituted as follows:New grandaunts from Teacher Training College offered a monetary incentive of M1000 (+-$133) per month broken down as follows:Transport allowance M500Housing allowance M250Communication allowance M250

Cont...In exchange for the incentive, teachers must

commit to teaching for 3 consecutive years in the difficult schools

Qualified teachers who are already present in the selected 400 schools will be offered the same incentive package as above

Unqualified teachers in the 400 selected schools shall receive a bursary to enrol in DTEP to become qualified

Cont...Those not qualifying for entry into DTEP will be given an

opportunity to upgrade for such entry

In exchange for the bursary, they must commit to teaching in the same school for the 3-4 years duration of training

Qualified teachers from non-selected (low lands or difficult control) schools can take up a place in a selected school, but only at regular remuneration.

This is intended to avoid a situation of qualified teachers, leaving control schools to move to treated schools, thereby contaminating treatment.

Incentive Objective

• Increase supply and equitable distribution of qualified teachers across the country

• To improve quality of education in difficult schools

Target

• 400 primary schools in difficult areas

• 1600 teachers

Schools’ Selection Criteria Schools should meet 2 to 3 of the

following conditions: • More than 3 km away from the main

road• Beyond natural barriers such as big

rivers, mountain ranges or gorges• Poor physical facilities e.g. less than 3

proper classrooms, no functional toilets for teachers and pupils

Cont…

• Beyond 6 km from public services, such as Health centers, police stations and post offices

• History of not attracting qualified teachers, and with less than 4 qualified teachers

• Specifically provide education for the disadvantaged groups in society

Implementation Strategy

• Difficult schools identified based on above criteria (ongoing- a preliminary list of 650 schools exist)

• Selection of 10 schools for piloting in 2010/11 (1 from each of the 10 districts)

• Randomization used to select remaining 390 treatment schools: 190 schools to be phased in 2011/12 200 schools for phase II in 2012/13 200 non-participating schools to serve as a control group

Cont…• The intervention will run through remaining 4 years of project

life span (2010- 2013) and effort will be made to visibly integrate it in annual plans and budget from 2010/11.

• key actors (including involved schools, national teachers associations, churches and communities as school proprietors, teacher training institutions, high level policy makers, the Ministry of Finance and Development Partners) consulted in key stages, including development of implementation timelines to secure their buy-in during the entire period.

• In selected schools, where only unqualified teachers are available, GOL will provide 2 grants for qualified teachers

Cont...

• In non-selected schools with excess grants, where there are vacant posts due to natural occurrences, such grants will be transferred to selected difficult schools.

• Positions will be advertised widely through media and short listed candidates interviewed by school boards;

• School boards’ recommendations submitted to Teaching Service Department for employment;

Motivation for Evaluation

• To identify causal relationship between teachers incentives, skills and qualifications of teachers and students learning outcomes by assessing the impact of teachers incentives on:

Change in numbers of qualified teachers in difficult schools;

skills and qualifications of teachers in difficult school;

Teachers, behaviour Learning outcomes

Evaluation Strategy2010 2011 2012 2013

10 Pilot Schools X - - -

190 Schools - X X X

200 Schools - - X X

200 Control Schools

- X X X

Cont…

• Develop research questionnaires and agree on performance indicators

• Conduct pilot study for 10 schools in 2010, as a basis for roll-out in 2011

• Conduct baseline study on: Treatment schools phased from 2011 and

control schoolsOR (b) Funds permitting, include schools in phase 2 into the baseline for future evaluation

Cont…

• Undertake annual IE from end of 2011 based on agreed IE design

• Equally important, identify and consult the key actors and stakeholders in all the above key stages of impact evaluation (IE) to secure their buy-in from 2010 to 2013.

Conclusion• Lesotho Delegation appreciates the training and support from

Marie Helene and Deon in the design and future implementation of the teachers incentive scheme;

• Hopefully, the collaboration will continue after the conference in moving forward the above outlined implementation and evaluation strategies.

• It is also hoped that when the valuable modifications made during this workshop period, to the initial discussions with the Minister and PS on design and implementation issues are grossly interrogated by this senior management, both experts and Christina (whom we wish quick recovery) will be receptive to video conferencing with us and them to ensure a shared vision.

• We Thank you